"Private" covers at least four quite different architectures, and they protect against different things. Teams that pick one before establishing what their actual requirement is tend to discover the mismatch late, during a security review, when rebuilding is expensive.
This is the conversation we have at the start of every sovereign AI engagement, and it takes an hour that saves weeks.
Start by naming the constraint
Before comparing architectures, work out which of these is true. They lead to different answers.
"Our data may not go to a third party." A contractual or policy constraint. Location is not the issue; the presence of an outside processor is. Your own cloud account may satisfy this, depending on how the contract is worded, and it may not.
"Our data may not leave the country." A jurisdictional constraint, typically from a data-localisation rule. An in-country region of a commercial cloud usually satisfies it.
"Our data may not be reachable by a foreign government." A sovereignty constraint rather than a residency one. This is where an in-country region of a foreign-owned provider may not be sufficient, because the question is about legal reach rather than physical location.
"Our network has no outbound connection." An operational constraint, common in defence, industrial control and secure facilities. It rules out anything that needs to call home at any point, including at start-up.
These stack. It is normal to face two or three at once, and the strictest one sets the architecture.
The four architectures
1. Air-gapped on-premises
Models run on servers inside your facility on a network with no route to the internet. Nothing calls out, at any stage, ever.
- Protects against: third-party processing, foreign legal reach, network compromise via the internet, and provider availability.
- Costs you: hardware, facilities, and an operational burden that is entirely yours. Updates follow an agreed manual path.
- Watch for: dependencies that fetch at runtime. Model weights, tokenizers, container images and licence checks all need to be local. Commercial tools with licence servers are frequently disqualified outright, regardless of what their documentation says about offline support.
The strongest guarantee available, and the highest operational cost. Appropriate when the constraint genuinely demands it and over-engineering when it does not.
2. On-premises, connected
The same hardware, on your own network, with controlled outbound access for updates and monitoring.
- Protects against: third-party processing of your data, and data leaving your jurisdiction.
- Costs you: the same hardware and most of the same operational burden, minus the manual update pain.
- Watch for: the outbound path being wider than intended. Telemetry, crash reporting and model-registry lookups can carry more than you expect. Audit what actually leaves.
Usually the pragmatic choice where a contract forbids third-party processing but the network does not need to be isolated.
3. Your own cloud account, in a chosen region
Deployed into your AWS, GCP or Azure tenancy, in a specific region, with private endpoints. You control identity, networking and keys; the provider operates the infrastructure.
- Protects against: data leaving a jurisdiction, and use of a third-party AI service, since you are running your own models on rented compute.
- Costs you: the least operationally, by a wide margin. No procurement, no facilities, elastic capacity.
- Watch for: whether your constraint is actually about location. If it is about foreign legal reach or about any third party at all, a commercial cloud region may not satisfy it however the data is encrypted.
The right answer far more often than teams assume, particularly when the underlying rule turns out to be a residency requirement rather than a sovereignty one.
4. A national or sector sovereign cloud
Infrastructure operated inside a jurisdiction, frequently under domestic ownership or regulatory arrangements designed to address foreign legal reach. Several countries have stood these up specifically for public-sector and critical-industry workloads.
- Protects against: everything a regional deployment does, plus the foreign-reach question, to the degree the arrangement is designed to.
- Costs you: less choice of services, and you are bound to that operator's roadmap and capacity.
- Watch for: the same runtime-dependency question as anywhere else. Sovereign infrastructure with a pipeline that calls an external embedding API is not sovereign.
Where your sector's regulator has effectively nominated a platform, this decision may already be made for you. We cover the regional picture in AI data residency.
Side by side
| Air-gapped | On-prem, connected | Your cloud account | Sovereign cloud | |
|---|---|---|---|---|
| Third-party processing | None | None | Infrastructure only | Infrastructure only |
| Data leaves country | No | No | No, if region chosen | No |
| Foreign legal reach | Not applicable | Not applicable | Possible concern | Addressed by design |
| Works without internet | Yes | No | No | No |
| Operational burden | Highest | High | Lowest | Low |
| Scales elastically | No | No | Yes | Usually |
The mistake that costs the most
Deploying the model privately and leaving the rest of the pipeline elsewhere.
An AI pipeline crosses the boundary in more places than the inference call. Embedding documents into vectors is processing. OCR on a scanned file is processing. Shipping prompt and response logs to a monitoring service carries the content with them. A system with a locally served model and a hosted embedding API is moving every document you index, and the architecture diagram will not show it.
The audit worth doing: list every stage between a document arriving and an answer being returned, and for each one ask where it physically runs and what leaves. Running open-weight models on infrastructure you control closes most of these crossings at once, which is the practical case for sovereign AI over a hosted API.
Hybrid is normal
These are not exclusive, and most mature deployments mix them deliberately:
- Training in a cloud account where GPU capacity is elastic, with inference on-premises where the live data is restricted.
- A language model on an internal server for document work, with vision models at the edge next to the cameras, so video never traverses the network at all.
- Air-gapped production alongside a connected development environment running on synthetic data.
The on-premises versus cloud trade-off covers how this plays out in practice for vision workloads; the reasoning transfers directly to language models.
How AxcelerateAI Helps
We design the deployment around the constraint rather than the other way round:
- Constraint review first, so the architecture matches what your contracts and regulator actually require.
- Deployment to your nominated environment — air-gapped, on-premises, your cloud account or a sovereign platform.
- Full-pipeline audit, so no stage quietly crosses the boundary you set.
