Document-Level Security

The core mechanism that checks a user's identity against a document's Access Control List (ACL) before retrieval. When a query is issued, the system filters the searchable corpus to only include documents where the user has explicit read permissions. This is typically implemented via metadata filtering in vector databases, where each document chunk inherits the ACL from its parent file. The authorization decision is binary: the entire document is either fully accessible or completely invisible to the RAG pipeline.

Documents are tagged with hierarchical sensitivity labels such as Public, Internal, Confidential, or Restricted. The retrieval engine compares the user's clearance level against the document's classification before allowing any chunks to be surfaced. This model is common in government and defense sectors using mandatory access control (MAC). Unlike discretionary ACLs, classification labels enforce a lattice-based security model where users cannot downgrade sensitivity or share documents outside their clearance band.

Document-level security can be applied at two stages:

• Pre-Retrieval Filtering: The search space is scoped before the vector similarity query runs. This is efficient but requires ACLs to be indexed as metadata filters.
• Post-Retrieval Filtering: The full search runs first, then unauthorized documents are stripped from results. This ensures high recall but risks leaking document existence through side channels.

Most production systems combine both approaches, using pre-filtering for efficiency and post-filtering as a safety net.

Document-level security depends on synchronizing user attributes from Identity Providers (IdPs) like Azure AD, Okta, or PingFederate. Groups, roles, and attributes are mapped to document permissions through SCIM provisioning or LDAP queries. When a user's department or role changes, the vector store must reflect these updates in near real-time to prevent stale access grants. This requires robust entitlement propagation pipelines that handle join/leave/move lifecycle events without blocking retrieval operations.

Document-level security alone is insufficient for complex enterprise needs:

• Granularity Gap: A user with access to a 200-page report can retrieve any sentence within it, even if some sections contain sensitive financial data.
• Context Aggregation Risk: Multiple low-sensitivity documents, when combined in a prompt, may reveal classified information.
• No Field-Level Awareness: PII embedded within an authorized document passes through unfiltered.

These gaps are addressed by complementary controls like field-level security, chunk-level authorization, and data masking applied downstream.

The architectural component that intercepts retrieval requests and enforces the PDP's access decision by filtering or blocking results before they reach the LLM. The PEP acts as a security gate within the RAG pipeline, ensuring no document bypasses authorization.

• Sits between the retriever and the generator in a RAG stack
• Can implement pre-retrieval filtering, post-retrieval redaction, or both
• Must operate with minimal latency to avoid degrading the user experience

A method of regulating access based on the roles of individual users within an enterprise. Permissions are assigned to roles rather than directly to subjects, and documents inherit access rules based on the role's clearance level.

• Simplifies permission management at scale by grouping users
• Common roles: viewer, editor, admin, auditor
• Less granular than ABAC; may grant overly broad retrieval access if not combined with document labels

An access control paradigm that evaluates attributes of the user, resource, and environment against a set of policies to grant or deny retrieval access. Unlike RBAC, ABAC supports dynamic, context-aware decisions.

• Attributes include: department, clearance level, document classification, geolocation
• Policies are expressed as logical rules: "Allow if user.clearance >= document.sensitivity AND user.department == document.owner"
• Essential for implementing context-aware retrieval in zero-trust architectures

A technique used in vector databases to restrict search results by applying boolean conditions to the key-value attributes associated with each document chunk. Metadata filtering is the primary enforcement mechanism for document-level security in RAG.

• Filters are applied pre-query to scope the vector search space
• Common metadata fields: classification, owner, project_id, expiry_date
• Supported natively by vector stores like Pinecone, Weaviate, and Milvus

The systematic recording of every retrieval event to create an immutable record for forensic analysis and compliance. Each log entry captures the user identity, query, documents accessed, and the authorization decision.

• Essential for SOC 2, HIPAA, and GDPR compliance
• Enables detection of anomalous access patterns or privilege escalation attempts
• Should be stored in a tamper-proof append-only ledger separate from the retrieval pipeline