The sentence we hear most often in enterprise assistant discussions is that the data must not leave. It sounds like a single technical requirement. Underneath it sit at least three quite different needs:
- The data must not leave the country
- The data must never leave the corporate network
- The system must work at sites with no internet connectivity
These call for different architectures at different costs. The first can be met by a managed deployment hosted domestically. The second requires a deployment inside the customer's own data centre. The third is a separate problem entirely: a client architecture that works without a connection.
Separating the three at the start of a project prevents the "this was not what we meant" conversation six months in.
What a closed-network deployment actually contains
An on-premise enterprise assistant is not, as many assume, a model installed on a server. A working deployment includes at minimum:
- A model serving layer running inference on GPUs
- A document ingestion and indexing pipeline covering file shares, portals, DMS and ticket history
- A search layer with both text and vector indexes
- Authentication through the central identity provider (LDAP or Active Directory)
- Authorisation with role and document-level access control
- An audit and logging layer
- Observability for latency, error rate and usage
The two most underestimated items on that list are authorisation and access control. Getting the model running is a week of work. Getting authorisation right is the longest line item in the project.
The permission filter has to run before the query
This is the single most consequential design decision in enterprise assistant architecture, and it is where the most common mistake lives.
There are two approaches. In the first, the system retrieves results across the whole index, generates an answer, then strips out sources the user is not entitled to see. In the second, the search query itself is narrowed to the document set the user can access before it runs.
The first approach leaks, no matter how carefully it is implemented. The model has read a document the user should not see and has built its answer on it. Hiding the source does not change where the content came from. Sometimes the filename alone carries information: seeing "2026 Restructuring Plan" in a source list tells you something without opening it.
The permission filter therefore belongs in the search layer, applied before the query executes. The cost of that is carrying document access control lists into the index and propagating permission changes to it.
The question to answer on synchronisation is: when an employee's access is revoked, how many minutes until the index reflects it? It does not always have to be real time, but the window needs to be known and accepted. Role changes and departures make this a question auditors ask.
GPU sizing is about concurrency, not headcount
"We have 5,000 employees, how many GPUs do we need" has no direct answer. What matters is how many requests are in flight simultaneously, average prompt length, and generated response length.
Enterprise assistant usage is strongly peaked through the day, with clear spikes at the start of the morning and in the early afternoon. Deployments planned against the average queue during exactly those two hours.
The practical method is to measure the real concurrency distribution during the pilot and plan capacity on p95. Processing short questions and long document summarisation requests in the same queue is another common mistake; moving long jobs to a separate queue noticeably improves response time for short ones.
Model updates are their own process in a closed network
In an environment with no internet access, updating models and components is not a file copy. Working deployments define these steps:
- The update package is prepared and signed outside
- Transfer happens over an approved channel
- The signature is verified inside the environment
- Installation goes to a test environment first, with a regression run
- Rollout is staged, with a rollback plan
The fifth is missing in most organisations. When a new model version turns out to degrade certain answers and there is no version to fall back to, operations stop. Every closed-network deployment should keep the previous working version available locally.
Regression testing needs a fixed, organisation-specific question set. Around a hundred real questions with their expected source documents, run on every update, catches quality loss early. Building that set is a day of work and pays for itself on the first update.
Working in the field, without a connection
Manufacturing plants, energy sites, vessels, mines, remote construction and secure facilities have intermittent connectivity or none. For an assistant to be useful there, the architecture changes.
The core difference is that answers are produced on the device or on a local server, and that brings constraints.
Content selection becomes critical. A central deployment can index everything. A field device has limited storage and limited compute. Which procedures, maintenance manuals and safety instructions get shipped to the device is a scoping decision. Building role-based packages, a maintenance package, a quality package, a safety package, improves both size and retrieval accuracy.
Document version conflicts have to be managed. If a device has been offline for three weeks and a procedure changed in that time, the field worker is reading the old instruction. Working deployments show which document version an answer is based on and when that version was pulled. "This information is from the 12 March version" is far safer than a confidently wrong answer.
Synchronisation is bidirectional. When connectivity returns, new documents come down and usage logs go up. Those logs serve both audit and content improvement. The questions asked most often in the field with no available answer are the clearest work list a documentation team will ever get.
Device loss has to be a planned scenario. The local index contains corporate documents. Without disk encryption, remote wipe and device authentication, an offline deployment is riskier than a central one.
On-premise is not automatically more secure
This point deserves emphasis because it is routinely skipped in decision meetings.
Moving the deployment into your own data centre also moves patch management, monitoring, backup, access control and incident response to you. Without the team and the process to run those, a closed-network system can be more exposed than a managed one. An unpatched component does not become safe by being unreachable from outside.
The question in the decision meeting is not "which is more secure" but "which security burden will which team carry".
Governance has to be built too
Once the technical deployment is done, governance remains: what the model is used for, which risks were assessed, how supplier oversight works, and how the lifecycle is managed.
There is now a published standard to work against. ISO/IEC 42001:2023, released in December 2023, is the first international AI management system standard and defines requirements including risk management, AI system impact assessment, lifecycle management and third-party supplier oversight (iso.org). Paired with ISO/IEC 27001 on the information security side, the coverage is largely complete.
For teams looking for a risk framework, NIST's AI Risk Management Framework is a practical starting point (nist.gov).
One clarification for organisations operating under Turkish data protection law: an on-premise deployment removes the cross-border transfer question. It does not remove processing obligations. Information notices, retention periods, access rights and deletion policies apply exactly as before.
Pre-deployment checklist
- Is the requirement domestic hosting, in-network hosting, or disconnected operation
- Are document ACLs carried into the index, and does the filter run before the query
- Is the propagation delay for permission changes defined and accepted
- Was capacity planned on p95 concurrency rather than the average
- Are update packages signed, verified and reversible
- Does a regression test set exist and run on every version
- Do offline devices have disk encryption, remote wipe and version display
- Which team will run patching, monitoring and incident response
To discuss the architecture against your own environment, see On-Premise Chatbot Solutions and Offline Solutions.
References
- ISO/IEC 42001:2023, Information technology, Artificial intelligence, Management system. iso.org
- ISO/IEC 27001, Information security management systems. iso.org
- NIST AI Risk Management Framework (AI RMF 1.0). nist.gov
- Turkish Data Protection Authority, transfers abroad. kvkk.gov.tr