FAISS

FAISS provides native GPU implementations that dramatically accelerate vector search operations. The library supports single-GPU and multi-GPU configurations with near-linear scaling, enabling queries over billion-vector indices in sub-millisecond latency. Key capabilities include:

• Batched query processing for throughput optimization
• Half-precision (float16) computation to maximize memory bandwidth
• Multi-GPU sharding where indices are distributed across devices
• Direct integration with CUDA streams for asynchronous execution

This GPU acceleration makes FAISS the default choice for production legal retrieval systems where query latency directly impacts user experience.

FAISS implements a comprehensive taxonomy of Approximate Nearest Neighbor (ANN) index structures, each optimized for different accuracy-speed-memory tradeoffs:

• IndexFlatL2: Exact brute-force search using L2 distance, serving as the ground-truth baseline
• IndexIVFFlat: Inverted file index with k-means clustering for coarse quantization, reducing search scope
• IndexIVFPQ: Combines inverted files with Product Quantization (PQ) for compressed vector storage
• IndexHNSW: Hierarchical Navigable Small World graph-based index achieving logarithmic search complexity
• IndexLSH: Locality-Sensitive Hashing for cosine similarity search

This diversity allows legal embedding systems to select the optimal index for their specific precision-recall requirements.

FAISS pioneered the practical implementation of Product Quantization (PQ) for vector compression, enabling billion-scale indices to fit in RAM. The technique:

• Decomposes high-dimensional vectors into M sub-vectors
• Quantizes each sub-vector independently using k-means codebooks
• Achieves compression ratios of 8x-32x with minimal recall degradation
• Supports asymmetric distance computation where queries remain uncompressed for higher accuracy

For legal document retrieval with 768-dimensional embeddings, PQ can reduce memory from 3 KB per vector to under 100 bytes, making entire legal corpora searchable on a single machine.

FAISS provides a declarative index factory that constructs complex indexing pipelines from simple string descriptions. This abstraction enables rapid experimentation:

• 'IVF4096,PQ64' builds an inverted file with 4096 centroids and 64-byte PQ compression
• 'HNSW32' creates a hierarchical navigable small world graph with 32 neighbors per node
• 'OPQ64,IVF4096,PQ64' chains Optimized Product Quantization preprocessing before IVF+PQ indexing
• 'PCA80,Flat' applies dimensionality reduction to 80 dimensions before exact search

The factory pattern allows legal AI engineers to benchmark multiple index configurations without writing custom construction code.

FAISS supports direct index serialization to disk and memory-mapped loading, critical for production legal retrieval pipelines:

• write_index() and read_index() for full index persistence
• Memory-mapped IO allows multiple processes to share a single index without duplication
• Incremental index building supports adding vectors to existing indices without full rebuilds
• IndexIDMap wrapper maintains mapping between FAISS internal IDs and external document identifiers

This serialization architecture enables zero-downtime index updates in legal retrieval systems where new case law and contracts must be continuously ingested.

FAISS natively supports multiple distance metrics essential for different legal embedding spaces:

• L2 distance (Euclidean): Standard for many dense embedding models including Legal-BERT fine-tunings
• Inner product: Used when embeddings are normalized and cosine similarity is required
• Cosine similarity: Directly supported through vector normalization preprocessing
• L1 distance (Manhattan): Available for sparse or specialized embedding spaces
• Custom distances: Extensible framework for domain-specific similarity functions

This metric flexibility ensures FAISS can serve as the retrieval backend regardless of the legal embedding model architecture or training objective.