Security Trimming

Applies a user's security permissions as a mandatory filter before the search query executes against the index. The system modifies the query to include only authorized document identifiers or access control lists (ACLs).

• Mechanism: Injects a filter clause containing the user's group memberships or clearance level into the search query.
• Performance: Highly efficient for large indexes, as the retrieval algorithm only scores documents the user is already allowed to see.
• Trade-off: Requires the index to maintain a real-time, high-cardinality mapping of every document's authorized principals.

Executes the search query broadly against the full index first, then strips unauthorized documents from the result set before presenting them to the user.

• Mechanism: A Policy Enforcement Point (PEP) intercepts the raw results and checks each document's ACL against the user's security context.
• Use Case: Ideal for systems where security policies are too complex to express as a simple query-time filter, such as Attribute-Based Access Control (ABAC).
• Risk: Can lead to pagination issues where a page of 10 results is reduced to 3 after filtering, requiring backfill logic.

Two distinct strategies for resolving a user's security group membership during the trimming process.

• Early Binding: The user's group memberships are resolved at session login and cached. The cached groups are used for filtering. Fast, but doesn't reflect real-time permission changes.
• Late Binding: The system queries the identity provider (e.g., Active Directory) for current group memberships at query time. Ensures Zero Trust compliance but adds latency to every request.

Security trimming relies on a synchronized Access Control List (ACL) stored alongside the document vector or metadata.

• Indexing Phase: The ingestion pipeline must extract ACLs from the source system (e.g., SharePoint, Google Drive) and attach them to the chunk.
• Stale Permissions: A critical failure mode occurs if the ACL in the search index becomes out of sync with the source-of-truth document store, leading to data leakage.
• Solution: Implement a real-time ACL change notification system to trigger immediate re-indexing of affected documents.

In Retrieval-Augmented Generation (RAG), security trimming must occur during the retrieval phase to prevent the language model from grounding its answer on unauthorized data.

• Critical Step: The retriever must apply the user's permissions before passing the top-k chunks to the generator.
• Failure Mode: If trimming happens only on the final answer, the model may still hallucinate sensitive information it saw in the unauthorized context window.
• Best Practice: Combine pre-retrieval filtering with a final post-generation audit to redact any residual sensitive entities.

For systems with millions of documents and thousands of users, naive ACL checking is computationally prohibitive. Roaring bitmaps provide a high-performance solution.

• Mechanism: Each document is assigned a bit in a bitmap. A user's authorized document set is a pre-computed bitmap of all accessible document IDs.
• Trimming: The search result set (a bitmap of candidate doc IDs) is logically ANDed with the user's permission bitmap.
• Efficiency: This reduces complex ACL traversal to a single CPU-level bitwise operation, enabling sub-millisecond trimming.

The most performant security trimming strategy. User permissions are applied as a pre-filter before the vector or keyword search executes. The query is augmented with an access control list (ACL) clause, ensuring the index scan only considers documents the user is authorized to see.

• Mechanism: Query-time ACL injection
• Advantage: Guarantees no unauthorized documents enter the candidate set
• Trade-off: Requires highly performant index-level filtering

A security trimming fallback where the search executes broadly, and results are filtered after scoring. Each document's ACL is checked against the user's identity, and unauthorized items are removed before the final result set is returned.

• Mechanism: Result-set ACL validation
• Risk: Can lead to empty or sparse result pages if top-N results are all unauthorized
• Use case: Complex permission models that cannot be expressed as simple index filters

The foundational permission model that security trimming enforces. Access is granted or denied at the entire document boundary. A user either sees a document in its entirety or not at all.

• Contrast with: Field-level security, which masks specific attributes within a visible document
• Implementation: Typically enforced via ACLs stored as document metadata
• Challenge: Maintaining sync between source system permissions and the search index

The architectural component that evaluates access requests against applicable policies during security trimming. The PDP consumes user attributes, resource metadata, and environmental context to issue a PERMIT or DENY decision.

• Protocol: Often implements XACML or OPA-style policy evaluation
• Separation of concerns: Decouples policy logic from enforcement, enabling centralized policy management
• Latency impact: PDP call must be sub-millisecond to avoid degrading search response times

The policy paradigm that enables dynamic security trimming. Access is granted based on attributes of the user, resource, and environment evaluated against rules, rather than static role memberships.

• Example policy: 'Allow READ if user.department == document.department AND user.clearance >= document.classification'
• Advantage: Eliminates role explosion and enables context-aware trimming
• Standard: NIST SP 800-162 provides the architectural framework

The specific application of security trimming to RAG pipelines. Ensures that a language model only grounds its answer on documents the end-user is permitted to see, preventing indirect data leakage through generated text.

• Critical failure mode: A model synthesizing an answer from a document the user cannot view directly
• Implementation: Trimming occurs at the retrieval stage, before documents enter the LLM context window
• Audit requirement: Every retrieved chunk must be logged with its authorization decision