Building private AI search.

What private AI search has to guarantee

Private AI search is retrieval-augmented generation over internal documents with one extra guarantee: a user only ever gets answers built from content they could open themselves. Everything else (chunking, ranking, citations) matters, but a leak of one restricted HR file outweighs a thousand good answers.

The system has five parts: connectors that read source systems and their permissions, an index that stores chunks with access metadata, a retrieval step that filters by the asking user’s identity, a model that writes an answer with citations, and logs that record what was retrieved for whom. Each part can leak. Our explainer on retrieval augmented generation covers the basics; this guide covers the permission layer.

Where to enforce permissions

Enforce permissions inside the retrieval query, so restricted content is never a candidate. The alternatives fail in predictable ways.

OWASP’s 2026 Top 10 for LLM applications lists cross-tenant leakage through shared similarity search under vector and embedding weaknesses: when access control runs after the search, an attacker can infer the existence and topic of other tenants’ documents from result counts, score distributions and timing. The 2025 entry recommends permission-aware vector stores and strict logical partitioning between user groups.

Syncing permissions from source systems

Permissions live in the source systems (SharePoint, OneDrive, Google Drive, Confluence, file shares), and each models them differently: inheritance from folders and sites, sharing links, external guests, nested groups. Your connector has to translate all of that into a list of principals per document.

Design rules that hold up:

• Store group IDs on documents; expand the user’s groups at query time. Copying group membership into the index goes stale every time someone joins or leaves a team.
• Decide what “anyone with the link” means. Most organisations exclude link-shared files from search unless the link is organisation-scoped.
• Sync deletions and revocations first. A new document arriving late is an inconvenience; a revoked one lingering is a breach.
• Set a freshness target and measure it. For example, revocations reflected in search within 15 minutes, with the actual lag reported daily.
• Re-check at source for the most sensitive collections. A live permission check against the source API before returning a chunk costs latency but removes sync lag from the risk.

Oversharing in the source is the largest practical risk. If a payroll spreadsheet is shared with the whole company, a correct permission-aware search will faithfully surface it to everyone. Off-the-shelf assistants work the same way: Microsoft states that Microsoft 365 Copilot only surfaces organisational data to which each user has at least view permission, so existing oversharing becomes visible the day it is switched on. Audit sharing before you index, especially in SharePoint and Google Drive.

def private_search(user, query, index, acl, k=8):
    """Pseudo-code: the filter syntax depends on your vector database."""
    principals = acl.principals_for(user)        # user id plus current group ids
    hits = index.query(
        vector=embed(query),
        top_k=k * 2,
        filter={"allowed_principals": {"any_of": sorted(principals)}},  # inside the search
    )
    if index.collection_is_restricted:
        hits = [h for h in hits if acl.source_allows(user, h.doc_id)]  # live re-check
    log_retrieval(user_id=user.id, query=query, doc_ids=[h.doc_id for h in hits])
    return [h.without_score() for h in hits[:k]]  # never return raw similarity scores

How filters behave under load differs by product; see choosing a vector database for filtered-search performance.

The leakage test suite

You prove privacy with tests run as real user identities, on every release and nightly in production. The core technique is canary documents: restricted files containing a unique string that must never appear anywhere an unauthorised user can see.

CANARY = "CANARY-7f3a91c2"  # appears only in hr/restructure-2026.docx

def test_canary_never_reaches_unauthorised_users(app, directory):
    for user in directory.users_without_access("hr/restructure-2026.docx"):
        for q in ("restructure plans", "who is affected by the restructure", CANARY):
            result = app.ask(as_user=user, question=q)
            assert CANARY not in result.answer
            assert all(CANARY not in c.text for c in result.retrieved_chunks)
            assert "hr/restructure-2026.docx" not in result.citation_ids

Injection cases deserve their own suite; see prompt injection testing. Retrieval quality is a separate question with its own metrics, covered in RAG retrieval evaluation.

The leaks people forget

The index is not the only store of your documents. Every derived copy needs the same protection as the source.

• Embeddings. OWASP warns that stored embeddings can be inverted to recover source text, so vector backups belong at the same sensitivity tier as the documents.
• Semantic caches. Key cached answers by the permission set as well as the question, or do not cache restricted collections.
• Agent memory. Summaries of past conversations can carry restricted facts into new sessions; see agent memory design.
• Logs and traces. Retrieval logs contain the retrieved text. Restrict who can read them and set retention.
• Evaluation sets. Questions and gold answers drawn from restricted documents inherit their access rules.
• Model providers. Retrieved text is sent to the model provider on every call. Where that processing happens is covered in data residency for AI systems.

In New Zealand, IPP5 of the Privacy Act 2020 requires safeguards that are reasonable in the circumstances against unauthorised access and disclosure. A search tool that shows personal information to the wrong employee is a privacy breach, and one likely to cause serious harm must be notified to the Privacy Commissioner, ideally within 72 hours of becoming aware of it.

Where private search goes wrong

We build these controls into RAG development projects and run canary tests as part of release checks. Policy and procedure search in regulated firms, such as internal policy retrieval in financial services, is where we see the strictest requirements.

• Permissions are enforced inside the retrieval query, never by filtering answers
• Documents carry group and user principals; user groups are expanded at query time
• Link-shared and externally shared files have an explicit indexing policy
• Revocations and deletions sync first, with a measured freshness target
• The most sensitive collections use a separate index or a live check against the source
• Canary documents are tested as real unauthorised users in CI and nightly
• Caches, memory, logs and evaluation sets are keyed or restricted by permission
• Raw similarity scores are never returned to clients
• Vector stores and backups are protected at the same tier as source documents
• Source-system oversharing was audited before the first index build

Written by the Levy Street team in Wellington. Last updated 19 October 2026.