Pagination

The most straightforward method, where the client specifies a limit (page size) and an offset (starting position).

• Simple Implementation: Easy for clients to understand and implement; the offset is simply the number of records to skip.
• Performance Drawback: On large datasets, high offset values can cause performance degradation as the database must count through many rows to find the start point.
• Result Inconsistency Risk: If data is modified (inserted/deleted) between page requests, the offset can become misaligned, causing skipped or duplicate results.

A more robust method that uses a stable pointer (cursor) to a specific record, rather than a positional offset.

• Stable Cursors: The cursor is typically an opaque token representing the last returned item's unique ID or a timestamp. The client passes this token to fetch the next page.
• Consistent Results: Immune to insertions or deletions in the dataset, as the cursor points to a specific record, not a position.
• Performance at Scale: Databases can use indexed lookups on the cursor value, making page retrieval efficient regardless of how deep into the result set you are.

Defines where the pagination state is managed.

• Stateless (Client-Managed): The client is responsible for tracking the offset or cursor. The API treats each request independently. This is standard for RESTful APIs and scales well with distributed clients.
• Stateful (Server-Session): The server creates a temporary session or query context that holds the pagination state (e.g., a server-side cursor). The client receives a session ID. This is more common in SQL databases and can be more efficient for complex, multi-stage queries but adds server-side resource overhead.

Pagination must work seamlessly with filter expressions and hybrid search.

• Filter-First Pagination: Filters are applied to the entire dataset before similarity scoring and pagination. This ensures paginated results are consistent subsets of the filtered view.
• Ordering Guarantees: Pages are returned in the order defined by the similarity score (or another specified sort). The pagination mechanism must preserve this order across all pages.
• Complex Query Support: Pagination tokens must encode enough context to correctly resume a query that includes multiple filters, distance metrics, and sorting criteria.

Pagination directly impacts query latency and system load.

• Page Size Tuning: The limit parameter controls network payload size. Smaller pages reduce latency for the first byte but increase total round trips. Optimal size is often 50-100 vectors.
• Index-Friendly Cursors: Cursor-based pagination allows the database to use the vector index efficiently for "seek" operations, rather than scanning.
• Concurrent Page Access: Well-designed pagination allows different clients or threads to fetch different pages of the same large result set concurrently without blocking.

The JSON response format communicates pagination metadata clearly.

• Data Array: Contains the page of vector results with IDs, embeddings, scores, and metadata.
• Pagination Metadata: Includes fields like next_cursor, has_more, or total (if calculable without performance penalty).
• Opaque Tokens: Cursors should be opaque strings that clients treat as black boxes, not parsing them. This gives the server flexibility to change the underlying implementation.
• Error Handling: APIs should define clear behavior for expired or invalid cursor tokens, typically returning a 400 Bad Request or 404 Not Found error.

The most basic pagination method where limit specifies the maximum number of results per page and offset indicates how many initial records to skip. While simple, it has significant drawbacks for mutable data.

• Simple Implementation: Easy for clients to understand and implement.
• Performance Issues: High offsets force the database to scan many skipped records, degrading performance.
• Data Consistency Risk: If records are inserted or deleted between pages, the offset can become inaccurate, causing missed or duplicate results.

An advanced technique that uses a stable pointer (cursor) to a specific record's position, rather than a numerical offset. The cursor is often an opaque token derived from a unique, sortable field (like a timestamp or vector ID).

• Performance: Enables constant-time lookups regardless of page depth.
• Data Stability: Resilient to insertions and deletions in the dataset, preventing skipped or duplicate items.
• Implementation: The API returns a next_cursor token with each page, which the client uses to fetch the subsequent page. This is the preferred method for high-throughput vector search.

An opaque, often encrypted, string returned by an API that encodes the state needed to retrieve the next or previous page of results. It is the client-facing manifestation of a cursor.

• Opaque to Client: The token's internal structure (e.g., a timestamp, ID, and sort order) is an implementation detail hidden from the API consumer.
• Stateless Server: Allows the server to be stateless; all pagination context is contained within the token.
• Security: Can be signed to prevent tampering with pagination parameters.

The complete, ordered collection of items matching a query before pagination is applied. In vector databases, this is typically the list of nearest neighbors sorted by similarity score.

• Virtual Collection: The full result set may not be materialized in memory; it's often generated lazily as pages are requested.
• Deterministic Order: Pagination requires a deterministic sort order (e.g., by distance metric). Changes to the sort order between requests break pagination.
• Size Estimation: Some APIs provide the total size of the result set, which can be useful for UI progress indicators.

A specific type of cursor-based pagination where the cursor is a value from a unique, sortable column. The query fetches records where the sort key is greater than (or less than) the cursor value.

• Query Pattern: SELECT * FROM vectors WHERE id > :last_seen_id ORDER BY id LIMIT 10.
• Efficiency: Can use a database index on the sort key for very fast seeks.
• Common in Vector DBs: Often used with monotonically increasing vector IDs or immutable timestamps as the stable cursor.

A synonym for a page token, emphasizing its role in continuing a sequential read operation. It is critical for long-running queries or streaming large vector search results.

• Session-like Behavior: Allows resumption of a query after a client disconnect or timeout.
• Time-Bound Validity: Tokens may expire after a period to prevent indefinite storage of query state on the server.
• Use in Async APIs: Essential for retrieving partial results from asynchronous vector index queries or bulk embedding jobs.