{"slug":"local-models-for-private-documents","id":"local-models-private-documents","type":"guide","title":"Use Local Models for Private Documents","description":"A practical recipe for using local or private AI models with sensitive documents, including retrieval, redaction, access control, and evaluation.","last_updated":"2026-04-24","last_verified":null,"verification_status":"unverified","markdown_url":"/content/guides/local-models-for-private-documents.md","html_url":"/guides/local-models-for-private-documents","api_url":"/api/v1/guides/local-models-for-private-documents.json","content_hash":"b1c7d4960470756b809c247a281e52e4193aeba96d3f9a27e142e0bdb15359e1","sha256":"b1c7d4960470756b809c247a281e52e4193aeba96d3f9a27e142e0bdb15359e1","tags":["recipe","local","privacy","documents","retrieval"],"relationships":{"links":[{"text":"Choose a Local Model","href":"/guides/choose-a-local-model","html_path":"/guides/choose-a-local-model","target_id":"choose-a-local-model","target_type":"guide","target_title":"Choose a Local Model"}],"related":[{"id":"choose-a-local-model","title":"Choose a Local Model","type":"guide","html_url":"/guides/choose-a-local-model","markdown_url":"/content/guides/choose-a-local-model.md","shared_tags":["local","privacy"],"score":4},{"id":"choose-model-for-customer-support","title":"Choose a Model for Customer Support","type":"guide","html_url":"/guides/choose-model-for-customer-support","markdown_url":"/content/guides/choose-model-for-customer-support.md","shared_tags":["recipe","retrieval"],"score":4},{"id":"choose-a-private-model","title":"Choose a Private Model","type":"guide","html_url":"/guides/choose-a-private-model","markdown_url":"/content/guides/choose-a-private-model.md","shared_tags":["privacy","local"],"score":4},{"id":"build-a-coding-agent-stack","title":"Build a Coding Agent Stack","type":"guide","html_url":"/guides/build-a-coding-agent-stack","markdown_url":"/content/guides/build-a-coding-agent-stack.md","shared_tags":["recipe"],"score":3},{"id":"build-agent-readable-docs-site","title":"Build an Agent-Readable Docs Site","type":"guide","html_url":"/guides/build-an-agent-readable-docs-site","markdown_url":"/content/guides/build-an-agent-readable-docs-site.md","shared_tags":["recipe"],"score":3},{"id":"build-ai-research-assistant","title":"Build an AI Research Assistant","type":"guide","html_url":"/guides/build-an-ai-research-assistant","markdown_url":"/content/guides/build-an-ai-research-assistant.md","shared_tags":["recipe"],"score":3}],"explicit":{}},"metadata":{"title":"Use Local Models for Private Documents","type":"guide","id":"local-models-private-documents","description":"A practical recipe for using local or private AI models with sensitive documents, including retrieval, redaction, access control, and evaluation.","last_updated":"2026-04-24","tags":["recipe","local","privacy","documents","retrieval"]},"content_text":"# Use Local Models for Private Documents\n\nPrivate document workflows are usually retrieval systems with model reasoning on top. The model is only one part of the privacy design.\n\n## Minimal Stack\n\n- Document store with access controls\n- Parser and chunker\n- Local embedding model or private embedding API\n- Vector index or keyword search\n- Local/private generation model\n- Audit logs\n- Evaluation set\n\n## Workflow\n\n1. Classify documents by sensitivity.\n2. Decide whether embeddings can leave the environment.\n3. Build retrieval with access checks.\n4. Use a local/private model for answers.\n5. Require citations to retrieved chunks.\n6. Log prompts, retrieved documents, and outputs.\n\n## Model Guidance\n\nUse [Choose a Local Model](/guides/choose-a-local-model) for model selection. Prefer the smallest model that passes document QA evals.\n\n## Guardrails\n\n- Do not retrieve documents the user cannot access.\n- Do not answer without retrieved evidence.\n- Redact secrets before logs leave the environment.\n- Keep evaluation documents separate from training or tuning.\n\n## Failure Mode\n\nThe common mistake is focusing only on model weights. Private document systems fail when retrieval ignores permissions or logs leak sensitive content.","content_length":1593,"generated_at":"2026-04-24"}