Collection API

The API provides endpoints to create, delete, and list collections, which are the fundamental containers for vector data.

• Create: Define a new collection with a schema specifying the vector dimensionality, distance metric (e.g., cosine similarity, L2), and optional metadata field types.
• Delete: Permanently remove a collection and all its contained vectors.
• List/Describe: Retrieve metadata about existing collections, including their configuration and statistics like vector count.

This is the core data manipulation layer, enabling Create, Read, Update, and Delete operations on individual vectors within a collection.

• Insert/Add: Ingest new vectors along with their associated metadata (e.g., {"id": "doc_123", "text": "...", "category": "finance"}).
• Upsert: A combined insert-or-update operation that is idempotent, ensuring data consistency.
• Fetch/Get: Retrieve a specific vector and its full metadata by its unique ID.
• Delete: Remove a specific vector by its ID.

For high-throughput scenarios, the API supports submitting large numbers of vectors in a single request.

• Bulk Insert: Significantly reduces network overhead and latency compared to sequential single-vector inserts.
• Atomicity: Operations are often atomic at the batch level—either all vectors in the batch succeed or the entire operation fails, ensuring data integrity.
• Asynchronous Processing: For massive batches, the API may provide an async endpoint, returning a job ID for later status polling.

The API allows control over the underlying vector index, the data structure that enables fast Approximate Nearest Neighbor (ANN) search.

• Index Creation: Trigger the building of an index (e.g., HNSW, IVF) after data ingestion. This can be a background operation.
• Parameter Tuning: Configure index-specific parameters like ef_construction for HNSW or nlist for IVF, balancing build time, search speed, and recall accuracy.
• Rebuild/Reindex: Update the index after significant data changes to maintain optimal search performance.

The primary read operation: finding vectors semantically similar to a query vector.

• Nearest Neighbor Search: Submit a query vector and receive the k most similar results.
• Hybrid Search: Combine vector similarity with filter expressions on metadata (e.g., category = 'finance' AND date > '2024-01-01').
• Pagination: Handle large result sets using limit and offset or cursor-based tokens.
• Distance Metrics: Specify the function (cosine, L2, inner product) used to calculate similarity, which must match the collection's configuration.

Monitoring endpoints provide visibility into the state and performance of a collection.

• Metrics: Retrieve counts of total vectors, index status (e.g., BUILDING, READY), and memory usage.
• Health Checks: Verify the collection is operational and ready to serve queries.
• Performance Sampling: Some APIs offer endpoints to sample query latency or index integrity, aiding in Query Optimization and capacity planning.

A REST API is a programmatic interface for a vector database that uses standard HTTP methods (GET, POST, PUT, DELETE) and resource-oriented URLs to perform operations. It is the most common interface for web-based applications.

• Uses JSON as the primary data interchange format.
• Stateless by design; each request contains all necessary information.
• Ideal for public-facing services and integrations due to its simplicity and wide support across programming languages and tools.

A gRPC API is a high-performance, schema-driven interface that uses the HTTP/2 protocol and Protocol Buffers for efficient, low-latency communication.

• Provides strongly-typed contracts via .proto files, enabling automatic client and server code generation.
• Supports bi-directional streaming, allowing for real-time data flow during bulk operations.
• Typically preferred for internal, service-to-service communication within microservices architectures where performance is critical.

A Client SDK is a software development kit that provides a language-specific library, abstracting the underlying API calls to simplify interactions for developers.

• Handles low-level details like HTTP connection pooling, serialization, and authentication.
• Implements retry logic and circuit breakers to improve application resilience.
• Offers idiomatic APIs (e.g., collection.insert() in Python) that feel native to the language, reducing boilerplate code and accelerating development.

The Query API is the primary interface for executing similarity searches, such as k-nearest neighbor (k-NN) queries, against indexed vectors.

• Accepts a query vector and returns the most similar vectors from a collection.
• Supports hybrid search by combining vector similarity with metadata filter expressions.
• Allows configuration of search parameters, including the distance metric (e.g., cosine, L2), the number of results (top_k), and search ef or nprobe values for tuning the speed-accuracy trade-off.

Vector upsert is a combined insert/update operation. It inserts a new vector with a given ID or updates the existing vector if that ID is already present in the collection.

• Idempotent: Executing the same upsert operation multiple times yields the same final state, making it safe for retries.
• Critical for real-time applications where data is constantly changing, as it ensures a single source of truth for each vector ID.
• The operation must often be followed by an index refresh for the changes to be visible to subsequent queries.

A filter expression is a conditional statement applied to vector metadata during a search to narrow results, enabling precise hybrid search.

• Expressed using a query language (e.g., metadata["category"] == "science" AND price < 50).
• Executed before or after the vector similarity search, depending on the database's implementation and the filter's selectivity.
• Allows for complex, multi-field filtering, combining equality, range, and set operations to retrieve vectors that are both semantically relevant and meet specific business criteria.