Anomaly Detection

These foundational techniques model data using statistical distributions to identify outliers.

• Z-Score / Standard Deviation: Flags data points that fall beyond a set number of standard deviations from the mean.
• Interquartile Range (IQR): Defines a 'normal' range between the 25th and 75th percentiles (Q1 and Q3). Points outside Q1 - 1.5IQR or Q3 + 1.5IQR are considered anomalies.
• Grubbs' Test: A statistical test used to detect a single outlier in a univariate dataset assumed to come from a normally distributed population.

Best For: Univariate data, datasets where a normal distribution can be assumed, and providing a simple, explainable baseline.

Uses labeled datasets containing both 'normal' and 'anomalous' examples to train a classifier.

• Binary Classification Models: Algorithms like Logistic Regression, Random Forests, or Gradient Boosting Machines (XGBoost) are trained to distinguish between the two classes.
• Key Challenge: Requires a significant number of verified anomaly examples, which are often rare and costly to obtain.
• Example Use Case: Fraud detection in credit card transactions, where confirmed fraud cases are used to label training data.

Best For: Domains where labeled anomaly data is available and the characteristics of anomalies are relatively stable and known.

The most common approach for anomaly detection, which identifies outliers in data without pre-existing labels by learning the structure of 'normal' data.

• Clustering-Based (e.g., DBSCAN): Identifies dense regions of data; points that do not belong to any cluster are flagged as anomalies.
• Nearest-Neighbor Based: Measures the distance of a point to its k-nearest neighbors; isolated points have large distances.
• Isolation Forest: Explicitly isolates anomalies by randomly selecting features and split values to build trees. Anomalies are easier to isolate and require fewer splits.

Best For: Novel anomaly detection, where the types of anomalies are not known in advance, and labeled data is unavailable.

Leverages neural networks to model complex, high-dimensional data and detect subtle, non-linear anomalies.

• Autoencoders: Neural networks trained to reconstruct normal data with low error. High reconstruction error on a new data point indicates an anomaly.
• Variational Autoencoders (VAEs): Learn a probabilistic latent space; anomalies are points with low probability under the learned distribution.
• Generative Adversarial Networks (GANs): Can be used where the generator learns the normal data distribution, and the discriminator helps identify data that doesn't fit.

Best For: Complex data types like images, video, sensor sequences, and high-dimensional datasets where patterns are non-linear.

Specialized techniques for data where points are sequentially ordered and correlated over time.

• Seasonal-Trend Decomposition: Models data as Trend + Seasonality + Residual. Large residuals indicate anomalies.
• Exponential Smoothing (ETS) & ARIMA Models: Forecast the next point based on past values; significant deviations from the forecast are anomalies.
• Deep Learning Models: LSTMs and Temporal Convolutional Networks (TCNs) are particularly effective at learning complex temporal dependencies for forecasting and anomaly scoring.
• Change Point Detection: Identifies points in time where the statistical properties of the sequence change.

Best For: Monitoring server metrics (CPU, latency), financial market data, IoT sensor streams, and any sequential business process data.

Combines multiple detection algorithms to improve robustness, accuracy, and reduce false positives.

• Voting Ensembles: Runs multiple anomaly detectors (e.g., Isolation Forest, Local Outlier Factor) and flags a point as anomalous if a majority or weighted majority agree.
• Stacking: Uses the output scores of several base detectors as features to train a meta-model that makes the final anomaly decision.
• Contextual vs. Collective Anomalies: Hybrid systems often separate detection of individual point anomalies (contextual) from sequence or group anomalies (collective).

Best For: Production systems requiring high precision and stability, where the failure mode of any single algorithm is mitigated by the consensus of others.

An immutable, chronological record of all events and actions taken within a system. In the context of AI tool use, this provides a verifiable history of every API call, parameter passed, and result returned. It is the foundational data source for anomaly detection.

• Primary Use: Security forensics, compliance evidence, and root cause analysis.
• Key Property: Events are timestamped and sequentially ordered to reconstruct exact execution flows.
• Example: A log entry recording that an AI agent invoked a process_payment tool with specific transaction details at a given time.

A write-once, append-only data store where entries cannot be altered or deleted after creation. This property is critical for audit logs to ensure data integrity and serve as trustworthy evidence for compliance (e.g., SOX, GDPR) and security investigations.

• Technical Enforcement: Often implemented via Write-Once Read-Many (WORM) storage or cryptographic techniques like chained hashing (e.g., Merkle Trees).
• Contrast with Mutable Logs: Prevents tampering that could hide malicious activity or tool misuse.
• Foundation for Trust: Enables non-repudiation, where an actor cannot deny having performed a logged action.

The practice of writing log messages as machine-readable, key-value pairs (typically JSON) instead of unstructured plain text. This is essential for automated anomaly detection systems to parse and analyze tool invocation data at scale.

• Key Benefit: Enables precise querying, aggregation, and correlation of log events. For example, filtering all tool calls by user_id or error_code.
• Standard Fields: A well-defined log schema for tool use includes timestamp, agent_id, tool_name, parameters, response_status, latency, and error_message.
• Enables Log Enrichment: Contextual metadata (e.g., user role, session ID) can be consistently added to improve analytical value.

A security platform that performs real-time monitoring and analysis of events aggregated from various sources, including AI audit logs. SIEM systems apply correlation rules and machine learning to detect anomalous patterns indicative of security threats.

• Core Function: Log Aggregation from distributed agents and tools into a central pane of glass.
• Anomaly Detection Role: Uses statistical baselines and threat intelligence to flag deviations, such as an AI agent suddenly accessing tools outside its normal permission scope or at anomalous times.
• Output: Generates alerts for security operations center (SOC) analysts to investigate potential incidents.

A method of profiling requests as they flow through a distributed system. For AI agents making sequential or parallel tool calls, tracing provides a causal graph of execution, which is vital for detecting performance anomalies and logic errors.

• Key Concepts: A Trace represents an entire transaction; Spans represent individual operations (e.g., a single tool call).
• Standard: OpenTelemetry (OTel) is the vendor-neutral framework for instrumenting applications to generate traces, metrics, and logs.
• Anomaly Detection Use: Identifying unexpectedly long latencies in a specific tool call (span) or failure cascades across a chain of dependent tool invocations.

The proactive capability to reliably collect, preserve, and analyze digital evidence in anticipation of an incident. For AI systems, this means designing audit logging for tool use with anomaly detection and investigation as primary goals.

• Key Components: Establishing a chain of custody for logs, defining log retention policies, and ensuring PII redaction/data masking where required.
• Dependent on: Tamper-evident logs (using cryptographic hashes) and secure, centralized storage.
• Enables: Effective root cause analysis (RCA) after an anomaly is detected, allowing teams to understand the who, what, when, and how of a security or operational event.