Memory Metrics

These metrics measure the speed and efficiency of memory operations, directly impacting user experience and system scalability.

• Latency (P50, P95, P99): The time delay for read/write operations, critical for real-time agent responsiveness.
• Throughput (Ops/sec): The rate of successful operations the system can handle, indicating overall capacity.
• Cache Hit Rate: The percentage of queries served from a fast cache vs. slower primary storage. A low rate suggests poor indexing or cache sizing.
• Query Planning Time: The time taken by the memory query planner to devise an execution strategy before retrieval begins.

These metrics track resource consumption and limits, essential for capacity planning and preventing system overload.

• Vector Store Size: The total number of embeddings or memory items stored, often measured in millions.
• Index Size on Disk: The physical storage footprint of search indices.
• Concurrent Connections/Operations: Current load versus the configured memory concurrency limit.
• Context Window Usage: For systems managing LLM context, the percentage of the available token window consumed by retrieved memories.

These metrics assess the effectiveness and correctness of the memory system's core function: retrieving relevant information.

• Retrieval Precision/Recall: Measures the relevance of returned items. Precision is the fraction of retrieved items that are relevant; recall is the fraction of all relevant items that were retrieved.
• Mean Reciprocal Rank (MRR): Evaluates the ranking quality by measuring how high the first relevant result appears in the list.
• Retrieval Score Distribution: Analysis of the similarity scores (e.g., cosine similarity) for returned results. A tight, high-scoring cluster indicates confident matches.
• Hallucination Reduction Rate: In RAG systems, the measurable decrease in model fabrications when grounded by retrieved memory.

These metrics monitor system stability, error conditions, and overall operational health for SLOs and alerting.

• Error Rate: The percentage of operations resulting in failures (5xx errors, timeouts, connection errors).
• Uptime / Availability: The proportion of time the memory API is operational and responding to memory health checks.
• Mean Time Between Failures (MTBF): A measure of system reliability for hardware-backed stores.
• Backend Dependency Health: Status of connected systems like vector databases (e.g., Pinecone, Weaviate) or embedding model endpoints.

These metrics link system activity to infrastructure costs and operational overhead, crucial for CTO oversight.

• Cost per Query: Calculated from cloud infrastructure costs (compute, storage, networking) divided by query volume.
• Embedding Generation Cost: Expense associated with calling embedding model APIs (e.g., OpenAI, Cohere) for indexing new memories.
• Eviction Rate: The frequency at which items are removed from cache/storage by the memory eviction policy, indicating churn.
• Data Ingestion Throughput: The rate at which new memories can be processed, embedded, and indexed, affecting data freshness.

Metrics derived from memory telemetry and logging systems that provide deep visibility into request lifecycles and system behavior.

• Trace Duration & Spans: From OpenTelemetry for Memory, the breakdown of time spent in different stages (query, retrieval, ranking) for a single request.
• Log Volume & Patterns: Analysis of aggregated logs to detect anomalies, such as spikes in error messages or unusual query patterns.
• Alert Volume: The frequency of triggered memory alerting rules, indicating periods of instability.
• Correlation ID Coverage: The percentage of logs and traces that include a memory correlation ID, enabling effective request tracking.

The automated collection, transmission, and analysis of operational data from an agentic memory system. This is the foundational process that produces raw data, which is then aggregated into the memory metrics displayed on a dashboard. It involves instrumenting the memory system to emit traces, logs, and low-level performance counters.

The time delay between a request to an agentic memory system and the completion of that request. This is a critical performance metric typically measured in milliseconds (ms) or percentiles (p50, p95, p99). High latency directly impacts agent response time. Factors include:

• Network round-trip time to the memory store
• Query complexity and index utilization
• Embedding model inference time for vector searches
• Cache hit/miss status

The rate at which an agentic memory system can process operations, measured in operations per second (ops/sec). This metric defines the system's capacity under load. Key throughput measurements include:

• Queries per second (QPS) for read operations
• Writes per second for ingestion and updates
• Vector similarity searches per second Throughput is often tested against increasing concurrency limits to find the system's breaking point.

A performance efficiency metric measuring the percentage of memory read requests served from a fast-access cache versus requiring a slower primary storage lookup. A high hit rate (e.g., >95%) indicates effective caching and reduces average latency. This metric is crucial for systems using LRU (Least Recently Used) or other eviction policies. A low hit rate may signal poorly configured cache size or non-temporal access patterns.

A reliability metric quantifying the frequency of failed operations within the memory system. This is expressed as a percentage of total requests or as an absolute count per time interval. Common errors tracked include:

• Timeout errors from slow queries
• Connection errors to vector databases
• Internal server errors (5xx)
• Validation errors from malformed API requests Monitoring error rates is essential for SLA (Service Level Agreement) compliance and system health.

A diagnostic API endpoint or routine that verifies the operational status of the memory system and its dependencies. It performs synthetic tests, such as:

• Connectivity verification to the backing vector store
• Schema validation for memory collections
• Read/write capability of a test record
• Dependency status (e.g., embedding model API) Health checks are used by load balancers and orchestrators (like Kubernetes) to determine if a service instance is ready to receive traffic.