Query API

The core function is the k-Nearest Neighbor (k-NN) or Approximate Nearest Neighbor (ANN) search. Given a query vector, the API returns the k most similar vectors from the database. This is measured by a distance metric like cosine similarity, Euclidean distance (L2), or inner product. The API abstracts the complexity of traversing high-dimensional index structures like HNSW or IVF to deliver results in milliseconds.

Modern Query APIs support hybrid search by combining vector similarity with structured metadata filters. A filter expression (e.g., category = 'news' AND date > '2024-01-01') is applied before or during the vector search to narrow the candidate set. This enables precise retrieval, such as finding similar products only within a specific price range. Advanced implementations use pre-filtering, post-filtering, or single-stage filtered-ANN algorithms for optimal performance.

To tune the trade-off between speed (latency) and accuracy (recall), the API exposes search parameters. Common knobs include:

• ef or efSearch: Controls the size of the dynamic candidate list in HNSW graphs.
• nprobe: Determines how many Voronoi cells to search in IVF indexes.
• limit / top_k: The number of results to return.
• include_values / include_metadata: Flags to control data returned in the response. Engineers adjust these based on application requirements.

For high-throughput scenarios, the API supports batch queries, where multiple query vectors are submitted in a single request. This reduces network overhead and allows for internal optimizations. Some APIs also support multi-vector queries, where a single search is performed using an average or weighted combination of several input vectors, useful for complex semantic fusion.

For queries that match thousands of vectors, the API provides pagination mechanisms. This can be offset/limit-based (limit=10, offset=20) or, more efficiently, cursor-based. A cursor is an opaque token returned with a page of results that points to the next page, ensuring stability when the underlying index is being updated concurrently.

Beyond returning the nearest neighbors, the API allows setting a distance or similarity score threshold. Only vectors with a score better than the threshold are returned, filtering out weak matches. The API also returns the computed distance or similarity score for each result, enabling clients to perform confidence-based ranking or post-processing.

The core search operation executed by a Query API. It finds the most similar vectors in a database to a given query vector. Key parameters include:

• k: The number of nearest neighbors to return.
• Distance Metric: The function (e.g., cosine similarity, Euclidean distance) used to measure vector similarity.
• Query Vector: The embedding representation of the search input. This operation leverages the underlying vector index (like HNSW or IVF) for sub-linear search time, making it the fundamental action of semantic search.

A conditional statement applied to metadata during a vector search to narrow results, enabling hybrid search. It allows the Query API to combine semantic similarity with structured filtering.

• Example: WHERE department = 'engineering' AND date > '2024-01-01'.
• Implementation: Filters are typically applied pre-search (to reduce the candidate set) or post-search (to filter the top results).
• Use Case: Retrieving only documents relevant to a specific user, product version, or time period while maintaining semantic relevance.

The mathematical function used by the Query API to quantify the similarity or dissimilarity between two vectors. The choice of metric is crucial for result relevance and is often configurable per query or collection.

• Cosine Similarity: Measures the cosine of the angle between vectors, ideal for text embeddings where magnitude is less important. Range: -1 to 1.
• Euclidean Distance (L2): Measures the straight-line distance between points in vector space. Range: 0 to infinity.
• Inner Product (Dot Product): Related to cosine similarity but affected by vector magnitude. Requires normalized vectors for consistent nearest neighbor results.

The class of algorithms that enable fast similarity search at scale by accepting a trade-off between perfect accuracy (100% recall) and speed. Query APIs are built on top of ANN indices.

• Trade-off: Controlled by parameters like ef (search scope) in HNSW or nprobe (cells searched) in IVF, balancing latency against recall.
• Scalability: Allows query time to grow sub-linearly (e.g., O(log N)) with the number of vectors, making billion-scale searches feasible.
• Core Algorithms: HNSW (Hierarchical Navigable Small World), IVF (Inverted File Index), and PQ (Product Quantization) are common index types that accelerate the Query API.

A technique used by the Query API to split large result sets into manageable chunks or pages. This is essential for user interfaces and batch processing where returning millions of vectors in one response is impractical.

• Methods:
  • Limit/Offset: limit=10, offset=20 returns results 21-30. Simple but can be inefficient for deep pages on large indices.
  • Cursor-based: Uses a stable token (cursor) from the previous result to fetch the next page. More performant and consistent for iterating through changing datasets.
• API Design: Pagination parameters are typically part of the query request payload.

The practice of tuning Query API performance for lower latency and higher accuracy. This involves configuring both the query parameters and the underlying index.

• Latency vs. Recall: Increasing search parameters (e.g., HNSW's ef) improves recall at the cost of higher latency.
• Pre-filtering vs. Post-filtering: Deciding when to apply metadata filters significantly impacts performance. Pre-filtering reduces the search space but requires indexed metadata.
• Parallel Query Execution: Distributing a single query across index shards or partitions to reduce response time.
• Caching: Storing frequent or recent query results to serve subsequent identical requests from memory.