Quantization

A method where a pre-trained model is converted to a lower precision format after training is complete, without any retraining. This is the most common and straightforward approach.

• Process: The full-precision model's weights and activations are analyzed to determine optimal scaling factors (calibration) and then mapped to integers.
• Types: Includes weight-only quantization (only weights are quantized) and weight-and-activation quantization (both weights and activations during inference are quantized).
• Use Case: Ideal for rapid deployment where retraining is impractical. It provides significant memory savings with a manageable, predictable drop in accuracy.

A process where quantization is simulated during the training or fine-tuning phase, allowing the model to learn to compensate for the precision loss.

• Process: Forward passes use fake (simulated) quantization operations. The gradients are computed with respect to these quantized values, but the underlying full-precision weights are updated (Straight-Through Estimator).
• Advantage: Typically achieves higher accuracy than PTQ for the same bit-width, as the model adapts to the quantization error.
• Cost: Requires additional compute and time for the training cycle. Essential for aggressive quantization to very low bit-widths (e.g., 4-bit or lower).

A form of PTQ where the scaling factors for activations are calculated on-the-fly at runtime based on the observed data range for each input.

• Mechanism: Weights are quantized ahead of time (statically), but activations pass through a dynamic range calculation for each inference batch.
• Benefit: Handles inputs with varying ranges more effectively than static quantization, which uses a fixed, pre-calibrated range.
• Trade-off: Introduces minor runtime overhead for computing scaling factors. Commonly used for models like LSTMs and transformers where activation ranges can vary.

A form of PTQ where scaling factors for both weights and activations are determined once during a calibration step and remain fixed for all inferences.

• Calibration: Requires a small, representative dataset to observe the range of activations and determine optimal scaling factors (min/max values).
• Performance: Eliminates the runtime overhead of dynamic quantization, offering the fastest inference speed.
• Constraint: Accuracy can degrade if the calibration data is not representative of real inference data, as activation ranges are locked.

A strategy that applies different quantization bit-widths to different parts of a model based on their sensitivity to precision loss.

• Principle: Not all layers contribute equally to error. Sensitive layers (e.g., attention mechanisms) are kept at higher precision (e.g., 8-bit), while robust layers (e.g., certain embeddings) are pushed to lower precision (e.g., 4-bit).
• Optimization Goal: Achieves a better trade-off between compression ratio and model accuracy than uniform quantization.
• Method: Sensitivity is analyzed via heuristics, profiling, or automated neural architecture search (NAS) techniques.

Extreme forms of quantization where weights are constrained to just two values (-1, +1) or three values (-1, 0, +1).

• Binary Quantization: Represents weights with 1 bit. Enables highly efficient bitwise operations (XNOR, popcount) instead of floating-point multiplications.
• Ternary Quantization: Introduces a zero value, offering more representational capacity and often higher accuracy than binary, while still enabling significant computational savings.
• Challenge: Causes severe information loss, requiring specialized training techniques (e.g., BinaryConnect) and is typically applied to smaller models or specific hardware.

A specialized database designed to store, index, and query high-dimensional vector embeddings. It enables efficient semantic search and similarity retrieval, forming the primary memory backend for many agentic systems by allowing rapid lookup of relevant past experiences or knowledge.

• Key Function: Approximate Nearest Neighbor (ANN) search.
• Common Use: Storing embeddings of text, images, or other modalities for agent recall.
• Examples: Pinecone, Weaviate, Qdrant, and Milvus.

A structured semantic network representing real-world entities (nodes) and their interrelationships (edges). Unlike vector similarity, it enables logical, rule-based reasoning and provides explicit context, making it crucial for maintaining factual consistency and complex state in agent memory.

• Structure: Uses ontologies to define entity types and relationship properties.
• Query Language: Typically queried using SPARQL (for RDF graphs) or Cypher (for property graphs).
• Advantage: Provides deterministic, explainable links between pieces of information.

The core data structure within a vector store that is optimized for the rapid retrieval of vectors. It uses algorithms like HNSW (Hierarchical Navigable Small World) or IVF-PQ (Inverted File with Product Quantization) to perform Approximate Nearest Neighbor (ANN) search, trading perfect accuracy for massive speed and scalability gains in high-dimensional spaces.

The broader field of encoding information using fewer bits than the original representation. Quantization is a form of lossy compression specific to numerical data. Other techniques relevant to memory storage include:

• Lossless Compression (e.g., GZIP, LZ4): No data loss; used for logs, text.
• Deduplication: Eliminates duplicate data blocks.
• Erasure Coding: Provides data redundancy for durability.
• Columnar Formats (e.g., Apache Parquet): Compress data efficiently for analytical workloads.

An umbrella term for techniques that reduce the size and computational requirements of neural networks. Quantization is a primary method, but it operates alongside other strategies:

• Pruning: Removing unnecessary weights or neurons from a model.
• Knowledge Distillation: Training a smaller "student" model to mimic a larger "teacher" model.
• Low-Rank Factorization: Approximating weight matrices with lower-rank decompositions.
• Goal: Enable deployment on resource-constrained devices (edge, mobile) and reduce inference latency and cost.

The engineering discipline focused on reducing the cost, latency, and resource consumption of executing trained models. Quantization directly serves this goal by reducing the precision of calculations. It is part of a larger toolkit that includes:

• Kernel Fusion: Combining multiple GPU operations into one.
• Graph Optimization: Simplifying the model's computational graph.
• Caching: Reusing intermediate results (e.g., key-value caches in transformers).
• Batching: Processing multiple inputs simultaneously to improve hardware utilization.