Metadata Filtering

A strategy where structured constraints are applied to the index before the vector similarity search. The ANN algorithm only traverses nodes that satisfy the filter criteria.

• Mechanism: Reduces the search space upfront.
• Guarantee: Always returns a full top-k set if enough documents match the filter.
• Performance: Highly efficient when the filter is selective, but can be slow if the filter matches a massive portion of the index.
• Use Case: Enforcing strict access controls where a user must only see documents from their own department.

A strategy where the ANN search is executed first to find the top semantic matches, and then metadata constraints are applied to prune the results.

• Mechanism: Filters are applied to the final candidate list.
• Risk: Can return an empty result set if none of the semantically similar documents match the filter.
• Performance: Very fast initial search, but the final result count is unpredictable.
• Use Case: A product search where semantic relevance is paramount, but out-of-stock items are filtered from the final display.

The most basic form of metadata filtering, targeting a single attribute in a document's payload.

• Operators: Supports exact matches (==), range queries (>, <), and set membership (IN).
• Data Types: Typically applied to strings, integers, floats, and booleans.
• Example: { "filter": { "year": { "$gte": 2023 } } } retrieves only documents from 2023 or later.
• Indexing: Requires the filtered field to be indexed for high performance, avoiding a full scan.

Combines multiple filter conditions using logical operators to express complex business rules.

• Operators: $and, $or, $not allow for nested conditional logic.
• Granularity: Enables precise scoping, such as documents that are either published reports or internal memos, but not drafts.
• Example: { "$and": [ { "status": "published" }, { "$or": [ { "type": "report" }, { "type": "memo" } ] } ] }.
• Optimization: Query planners reorder clauses to execute the most selective filters first, minimizing the candidate set early.

Restricts results to documents associated with a physical location within a specified radius of a central point.

• Function: Uses $geoWithin or $geoRadius operators on geospatial indexes.
• Coordinates: Requires documents to store location data as GeoJSON points ([longitude, latitude]).
• Example: { "location": { "$geoWithin": { "$centerSphere": [[-73.97, 40.77], 10/3963.2] } } } finds documents within 10 miles of a New York City point.
• Use Case: Powering "find near me" features in local search or logistics applications.

A multi-tenancy pattern where a top-level metadata field logically isolates data for different customers or domains within a single physical index.

• Mechanism: Every document is tagged with a tenant_id or namespace string.
• Enforcement: A mandatory pre-filter is injected into every query to ensure strict data isolation.
• Benefit: Prevents accidental cross-tenant data leakage in a shared infrastructure.
• Trade-off: Can lead to unbalanced index shards if one tenant's data volume dwarfs others.

A retrieval strategy where structured metadata constraints are applied to the index before the vector similarity search is executed. This ensures that only documents meeting the filter criteria are considered during the ANN traversal, guaranteeing that all returned results satisfy the non-semantic constraints. Pre-filtering is ideal when the filter is highly selective and you need a guaranteed number of results, but it can introduce latency if the filter evaluation is complex.

A retrieval strategy where an ANN search is performed first to find the top semantic matches, and then metadata constraints are applied to prune the results. While computationally simpler, this approach can lead to empty result sets if the initial semantic matches do not satisfy the filters, especially when the filter is highly restrictive. Post-filtering is best suited for scenarios where semantic relevance is paramount and metadata constraints are broad or advisory.

An algorithm that merges multiple ranked result lists into a single ranking by assigning a score based on the reciprocal of each document's rank position. RRF effectively balances the contributions of different retrieval systems—such as dense vector search, sparse keyword matching, and metadata-filtered result sets—without requiring score calibration. This makes it a robust choice for combining structured and unstructured retrieval signals.

The process of scaling the uncalibrated relevance scores from disparate retrieval sources onto a common, comparable scale before they can be meaningfully merged. Common methods include:

• Min-Max Normalization: Rescales scores to a [0, 1] range.
• Z-Score Normalization: Standardizes scores based on mean and standard deviation. Proper normalization is critical when combining a metadata relevance score with a cosine similarity score from a vector search.

A fundamental data structure for sparse retrieval that maps each unique term to a postings list containing the identifiers of all documents where the term appears. In the context of metadata filtering, inverted indexes are often used to efficiently resolve structured constraints—such as date:[2023-01-01 TO 2023-12-31] or doc_type:contract—before or during the retrieval process, enabling rapid Boolean and faceted search over document attributes.

A strategy that combines results from multiple heterogeneous retrieval systems—such as dense vector search, sparse keyword matching (BM25), and metadata filtering—to improve overall recall and robustness. By leveraging the strengths of each system, ensemble retrieval mitigates the weaknesses of any single approach. For example, a metadata filter can restrict by date range, while a vector search finds semantically relevant documents within that window.