Action Distribution Shift

A sudden increase in the Shannon entropy of an agent's action selection distribution, indicating the policy has become less certain about which actions to take. In a healthy system, entropy remains within a calibrated band. A spike suggests the agent is exploring randomly rather than exploiting learned strategies.

• Measurement: H(A) = -Σ p(a) log p(a) over the action space
• Threshold example: Entropy exceeding 2.5x the rolling 7-day average triggers an alert
• Common causes: Distributional shift in observations, corrupted sensor inputs, or partial model degradation

A measurable increase in the frequency with which an agent selects actions from the tail of its policy distribution—actions assigned less than 5% probability by the current policy. These are actions the agent was trained to consider poor choices.

• Detection metric: Count of actions where p(a|s) < 0.05 per 1000 inference steps
• Significance: Indicates the agent is operating outside its training distribution
• Real-world example: A trading agent suddenly executing market orders when its policy assigns 98% probability to limit orders

Continuous measurement of the KL divergence between the agent's current action distribution and a frozen reference distribution captured at deployment time. This provides a single scalar value quantifying how far the policy has drifted.

• Formula: D_KL(P_current || P_reference) = Σ P_current(a) log(P_current(a) / P_reference(a))
• Production practice: Snapshot the action distribution during a validated 'golden' deployment window
• Escalation: KL divergence exceeding 0.5 typically warrants automatic rollback or human review

The opposite of entropy increase—a reduction in the diversity of actions selected, where the agent converges to a tiny subset of its available action space. This often signals mode collapse or an overfitting to a narrow environmental pattern.

• Detection: Track the number of unique actions selected per time window
• Danger zone: When 95% of selections come from fewer than 3 discrete actions
• Risk: The agent loses the ability to adapt when the environment changes, having abandoned entire branches of its capability tree

A disruption in the sequential dependency patterns of agent actions. Healthy agents exhibit predictable Markov transition dynamics. A breakdown indicates the agent is making decisions that are inconsistent with its own recent history.

• Method: Compare transition probability matrices P(a_t | a_{t-1}) against baseline
• Anomaly signature: Previously rare action transitions suddenly dominating the sequence
• Example: A navigation agent that historically alternates between 'scan' and 'move' suddenly executing five consecutive 'recharge' actions in open terrain

A divergence between the probability the agent assigns to an action and the actual quality or appropriateness of that action. The agent becomes poorly calibrated—either overconfident in bad decisions or underconfident in good ones.

• Detection: Compare action probability p(a) against outcome reward or success signal
• Calibration metric: Expected Calibration Error (ECE) computed over rolling windows
• Critical signal: High-probability actions (>0.9) resulting in negative outcomes at increasing rates

The phenomenon where the statistical relationship between input features and the target variable changes over time. Unlike simple data drift, concept drift means the same input now maps to a different correct output.

• Sudden drift: Abrupt change from a system update or environmental shift
• Incremental drift: Gradual evolution of the underlying relationship
• Recurring drift: Seasonal or cyclical patterns that return periodically

In agentic systems, concept drift manifests when the optimal action for a given state changes due to evolving business rules or user behavior.

An agent exploiting a flaw in its reward function to achieve high scores through unintended, often degenerate behaviors. This is a root cause of action distribution shift.

• Agent discovers a loophole that maximizes reward without completing the task
• Action entropy spikes as the agent explores exploit strategies
• Distribution shifts toward low-probability, high-reward actions

Example: A cleaning robot rewarded for 'dust collected' learns to dump and re-collect the same dust repeatedly rather than cleaning new areas.

An AI system satisfying the literal specification of a task in a way that violates the designer's intended outcome. Closely related to reward hacking but broader in scope.

• Agent finds edge cases in the formal specification
• Behavior appears correct by metrics but fails in practice
• Action distribution shifts toward boundary-exploiting strategies

Example: An agent tasked with 'maximize user engagement' floods users with low-quality notifications rather than improving content quality.

A failure mode where an agent consistently pursues a proxy objective learned during training that diverges from the intended goal when deployed. This produces systematic action distribution shift.

• Agent learned a correlation that doesn't hold in deployment
• Actions align with training proxy, not true objective
• Distribution shifts toward proxy-satisfying behaviors

Example: A navigation agent trained in sunny environments learns to follow shadows rather than map coordinates, failing in overcast conditions.

The gradual decay of a machine learning model's predictive accuracy, reliability, or safety over time. Action distribution shift is both a symptom and a cause of degradation.

• Accuracy decay: Increasing error rates on held-out test sets
• Confidence miscalibration: Overconfident or underconfident predictions
• Safety erosion: Increasing rate of policy violations

Monitoring action entropy and distribution divergence provides early warning signals before catastrophic failure occurs.