Private knowledge assistant
An internal assistant that answers from your documents, on your own servers
- Answers cite their sources
- Runs air-gapped if required
- You own the model and the index
Illustrative sample. File names and answers are fictional.
Answered in 5.8s on local hardware
Try another question
- On your servers
- No external APIs
- Data stays in
Why build it privately
The documents worth asking about are the ones you can't upload
The material that would benefit most from an assistant is usually the material under the strictest handling rules: contracts, patient records, unreleased research, engineering drawings, pricing. That is exactly what a hosted assistant is the wrong home for.
This is the same argument as sovereign AI generally, applied to one workload. If your constraint is regulatory rather than contractual, the rules on where data may be processed differ by country; see data residency by region.
Handling rules
Contracts and regulations often say where a document may be processed and who may see it. Uploading it to a third party can breach both.Permissions already exist
Your file shares already know who may read what. An assistant that ignores those rules is a data leak with a friendly interface.Answers need provenance
In regulated work, an answer without a source is not usable. Anyone reviewing it has to be able to open the document it came from.Cost that stops scaling
Per-seat pricing grows with headcount, not with value. Your own hardware costs the same whether ten people or a thousand use it.
| Topic | Hosted assistant on your files | Private assistant on your infrastructure |
|---|---|---|
| Where documents sit | Uploaded to the provider's storage | Never leave your file shares and network |
| Permissions | Usually per workspace, not per document | Retrieval filtered by your existing access rules |
| Provenance | Varies by product; often none | Every answer cites the passage it came from |
| Model changes | The provider upgrades on their schedule | You pin a model version and test before changing it |
| Audit | Limited to what the provider exposes | Every question, source and answer logged on your systems |
What we build
The parts a working assistant actually needs
A demo is a model and a vector store. A system people trust enough to use daily needs the six pieces below, and most of the engineering time goes into the ones that are not the model.
Ingestion for your real formats
Connectors for the file shares and systems you already run, handling PDFs, Office files, exports and scanned documents. Scans go through OCR first; rotated and low-quality material is its own engineering problem.Retrieval that finds the right passage
Chunking tuned to how your documents are structured, embeddings generated on your own GPUs, and hybrid keyword-plus-vector search with reranking, because pure vector search misses exact terms like clause numbers and part codes.Generation with citations
An open-weight model, quantized to fit your hardware, that answers from the retrieved passages and cites each one. Prompting and output filters define what it does when nothing relevant was found.Access control at retrieval time
Permissions applied when passages are fetched, not requested in a prompt. Users get answers built only from documents they are already entitled to open.An evaluation set from real questions
A graded set of questions your team actually asks, so changes to chunking, the model or the prompt can be measured rather than argued about. This is what makes the system improvable after launch.Audit logs and monitoring
Every question, the sources retrieved and the answer returned, logged on your systems. Retrieval quality is monitored so you can see when new document types stop being found.
How we build it
Retrieval quality first, model choice second
Most assistants that disappoint do so because the right passage never reached the model, not because the model was too small. We spend the early weeks on retrieval and measurement, which is also where the cheapest gains are.
A useful rule when budgeting: improving what gets retrieved is almost always cheaper than moving to a larger model, and it does not increase your hardware bill.
- Stage 1
Start from the questions, not the documents
We collect the questions your team actually asks and grade what a good answer looks like. That set becomes the evaluation harness, and it decides which document sources are worth ingesting first.Graded question set - Stage 2
Ingest and chunk for your structure
Contracts, manuals and reports each break into passages differently. We tune chunk size and overlap against the evaluation set rather than accepting a default, and add OCR where documents are scanned.Clean passages with source metadata - Stage 3
Index on your own GPUs
An embedding model runs on your hardware and writes into a vector database inside your environment. Nothing is sent out for embedding, which is a step teams often miss when they audit a pipeline.Private vector indexVector databaseLangChain - Stage 4
Hybrid search and reranking
Keyword search catches exact identifiers; vector search catches paraphrase. A reranking pass puts the strongest passages first, which raises answer quality more than a bigger model usually does.Ranked, relevant passages - Stage 5
Size and serve the model
We pick an open-weight model that fits your GPUs at your target latency, quantize it, and benchmark it on your own questions. The model is chosen against your hardware, not the other way round.Served model with measured latencyOpen-weight LLMQuantization - Stage 6
Measure, then harden
We run the evaluation set against the finished pipeline, agree the behaviour when nothing relevant is found, wire in access control and audit logging, then hand over the code and artifacts.A system your team can run and improve
What it takes to run
Hardware sized to the task, not to a headline number
Model size is a sizing decision driven by your GPUs, your users and your latency target. A well-retrieved passage in front of a mid-sized model beats a large model guessing from memory, and it costs considerably less to run.
- Quantization cuts memory use so a larger model fits the GPUs you have
- Concurrent users and target response time drive sizing as much as model size does
- Embedding and generation can run on separate hardware when that is cheaper
- We size it with you during the assessment, before you buy anything
For a worked example of the trade-off, our guide on on-premises versus cloud deployment covers how the same decisions play out for vision workloads.
Where it can run
Air-gapped on-premises
Model, index and logs on servers inside your facility, with no internet connection at any stage.Your own cloud account
Deployed into your AWS, GCP or Azure tenancy and region, with private endpoints your cloud team controls.Managed by us, hosted by you
The same isolated environment, with monitoring and retraining handled by our managed services team.
We have built this
An air-gapped assistant over proprietary medical research
For Atacana we built PharmaBrain, a retrieval assistant over thousands of proprietary medical reports that runs entirely offline on local servers. It uses Mixtral 8x7B, a 46.7B-parameter mixture-of-experts model, quantized to 8-bit so it fits local GPUs, and answers in five to seven seconds with no data leaving the client's environment.
Getting started
From your first questions to a system your team runs
Step 1: Assessment
Scope and feasibility
We review your document sources, access rules and the questions you want answered, then design ingestion, retrieval and serving for your environment.
- NDA on request
- Hardware sized before you buy
Step 2: Proof of concept
4–6 weeks
We build the pipeline on a real subset of your documents and benchmark it against an evaluation set drawn from your own questions.
- Measured on your data
- Runs on your hardware
Step 3: Production
Harden, deploy, hand over
Access control, audit logging and monitoring go in, then we deploy into your environment and hand over the code, models and runbooks.
- You own the IP
- Raw data stays with you
Guides from our engineers
Go deeper on private AI
Choosing a model, sizing the hardware, and what each deployment option actually protects against.
- Which Open-Weight LLM Should You Self-Host?Technical
- What GPUs Do You Need to Self-Host an LLM?Hardware
- RAG or Fine-Tuning? The Wrong Answer Is ExpensiveDeep dive
- What Does It Cost to Run Your Own LLM?Cost
- Air-Gapped, Private Cloud or Sovereign Cloud: Which Do You Need?Architecture
- Deploy an Open-Weight LLM on Your Own Server with vLLMTutorial
- Self-Hosted Coding Assistants: What Works Air-GappedGuide
- Data Residency Rules That Shape AI Projects in 2026Regulation
- On-Premises vs Cloud Computer Vision Deployments – An Enterprise GuideDeep dive
FAQ
Questions teams ask before they start
Where the documents go. A hosted assistant sends your files and every question to a third party, and you are bound by their retention terms, their model changes and their availability. A private assistant keeps the documents, the index, the model and the logs inside your own network. The trade is that you provide the hardware and we build the pipeline, rather than paying per seat forever.
Usually not, and it is rarely the right first move. Retrieval puts the relevant passages in front of the model at question time, which means new documents are searchable as soon as they are indexed and every answer can cite its source. Fine-tuning changes how a model writes and reasons, not what it knows today, and it has to be redone as your content changes. We start with retrieval and fine-tune only where the task needs it.
Three things together. Retrieval constrains the model to passages pulled from your own documents. Citations put the source next to every claim so a reader can check it. Output filters and an evaluation set catch the cases where the model answers anyway when nothing relevant was retrieved. We agree what 'no answer' should look like before launch, because a confident wrong answer is worse than a blank one.
We build custom pipelines, so the practical answer is whatever you have: PDFs, Word files, spreadsheets, slide decks, HTML exports, scanned documents that need OCR first, and records pulled from internal systems through their APIs. Scanned and rotated material needs an OCR stage, which is its own engineering problem rather than a setting to switch on.
Retrieval respects your existing access rules. Passages a user is not entitled to see are filtered before they reach the model, so permissions are enforced at retrieval time rather than requested in a prompt. Every question, the sources it retrieved and the answer it produced are logged on your systems for review.
A proof of concept on your own documents typically runs 4–6 weeks. That covers ingestion for one or two document sources, a working retrieval pipeline, an evaluation set built from questions your team actually asks, and benchmarks on your hardware. Production adds access control, monitoring and the handover of code and model artifacts.
Related
Keep exploring
- LLMs & agentsSovereign AIPrivate LLMs and vision models on your own servers, VPC or air-gapped network.
- LLMs & agentsOpen-weight model fine-tuningAdapt open models to your domain on your own infrastructure, and keep the weights.
- Document AIDocument AI (IDP)Invoices, contracts and forms read, validated and routed into your systems.
- LLMs & agentsAI data residencyHow localisation rules shape an AI deployment, and how to architect inside them.
Book a strategy session
Scope a private knowledge assistant
Tell us which documents you want answerable, where they live and any rules on where they may be processed. An engineer will reply within one business day, under NDA if you need it.
- Send the form, it takes 2 minutes
- We reply within 1 business day, under NDA if you need it
- A 30-minute call to scope feasibility and next steps