Failure Mode and Effects Analysis (FMEA)

A Failure Mode is the specific manner in which a component, subsystem, or process could potentially fail to meet its intended design function. It is a clear, concise description of the failure itself.

• Examples: A sensor provides a constant zero reading (failure to sense), a valve sticks in the closed position (failure to open), a software module enters an infinite loop (failure to proceed).
• The description should be specific and observable, avoiding vague terms like "breaks" or "fails."

The Effect of Failure describes the immediate consequences of the failure mode on the system's operation, performance, or safety at the local, subsystem, and end-user level.

• Local Effect: Impact on the specific component or its immediate function.
• System/Subsystem Effect: Impact on the operation of the larger subsystem or assembly.
• End Effect: The ultimate consequence for the user, customer, or environment.
• A single failure mode can have multiple cascading effects, all of which must be documented.

The Cause of Failure is the identified root reason why the failure mode could occur. It focuses on design weaknesses, process errors, or environmental factors that lead to the failure.

• Examples: Corrosion due to improper material selection, software bug in error-handling logic, excessive mechanical load due to miscalculation, human error in assembly.
• Effective FMEA requires drilling down to a root cause that can be addressed, not just a symptom. Identifying causes enables the development of targeted corrective actions.

Severity (S) is a numerical rating (typically on a 1-10 scale) that estimates the seriousness of the effect of the potential failure mode. It is assigned without regard for the likelihood of occurrence.

• Scale: A rating of 1 indicates a negligible effect (e.g., minor inconvenience), while a 10 indicates a catastrophic effect without warning (e.g., loss of life, system destruction).
• Severity is often defined by a standardized ranking table tailored to the industry (e.g., automotive, aerospace, medical devices). It is the first factor in calculating the Risk Priority Number (RPN).

Occurrence (O) is a numerical rating (typically 1-10) that estimates the probability or frequency that a specific cause of the failure mode will happen.

• This rating is based on historical data, test results, or predictive reliability models.
• Scale: A rating of 1 indicates failure is extremely unlikely (e.g., < 1 in 1,000,000), while a 10 indicates failure is almost inevitable (e.g., > 1 in 2).
• The goal of corrective actions is often to reduce the Occurrence rating by eliminating or controlling the root causes.

Detection (D) is a numerical rating (typically 1-10) that estimates the ability of the current design controls or process checks to detect the failure mode or its cause before the product reaches the customer or the system is deployed.

• Scale: A rating of 1 indicates detection is almost certain (e.g., automated proof test with 100% coverage), while a 10 indicates absolute uncertainty or no detection method exists.
• Detection evaluates the effectiveness of inspections, tests, simulations, or monitoring systems in place. Improving detection capability lowers the Detection rating.

The Risk Priority Number (RPN) is a calculated numerical value used to prioritize potential failure modes for corrective action. It is the product of the three ratings: RPN = Severity (S) × Occurrence (O) × Detection (D).

• Range: RPN values range from 1 (lowest risk) to 1000 (highest risk).
• Purpose: The RPN provides a relative risk ranking. Failure modes with higher RPNs are typically addressed first. However, high Severity ratings often mandate action regardless of the overall RPN.
• The RPN is a tool for prioritization, not an absolute measure of risk.

This component documents the proposed corrective or preventive actions to reduce the risk associated with a high-priority failure mode. It assigns ownership and tracks implementation.

• Actions target reducing Severity, Occurrence, or Detection ratings. Examples include design changes (reduce S or O), adding redundant systems (reduce S), improving testing (reduce D), or revising procedures.
• Each action is assigned to a responsible individual or team with a target completion date.
• After actions are implemented, the Severity, Occurrence, and Detection ratings are re-evaluated to calculate a Resultant RPN, verifying risk reduction.

The quantitative heart of FMEA is the Risk Priority Number (RPN), calculated as: RPN = Severity (S) × Occurrence (O) × Detection (D). Each factor is rated on a scale (e.g., 1-10).

• Severity: Impact of the failure effect.
• Occurrence: Likelihood of the failure cause.
• Detection: Ability to detect the failure before impact. High RPN failures are prioritized for mitigation. In simulation, this framework guides where to focus domain randomization and fault injection testing.

FMEA is performed on the simulation environment and the trained policy before physical deployment. Key analysis targets include:

• Simulation-to-Reality Gap: Failures due to inaccurate physics (e.g., friction, contact dynamics).
• Sensor Simulation Failures: Noise, latency, or distortion not modeled in simulated cameras/LiDAR.
• Actuator Dynamics: Differences between idealized motor models and real hardware saturation/backlash. This analysis directly informs the parameter ranges for domain randomization and the design of adversarial scenarios in training.

Beyond traditional engineering faults, FMEA for autonomous systems must analyze algorithmic and data-centric failures:

• Distributional Shift: The policy encounters Out-of-Distribution (OOD) states not seen in simulation.
• Adversarial Robustness: Sensitive to visual or physical perturbations (e.g., deceptive patterns).
• Reward Hacking: The agent exploits loopholes in the reward function, achieving high score without performing the intended task.
• Catastrophic Forgetting: Performance degrades on previously mastered skills after new training.
• Uncertainty Miscalibration: The model is overconfident in its erroneous predictions.

FMEA findings are operationalized through safety architectures:

• Constrained MDPs: Used to formulate Safe Reinforcement Learning problems where policies must respect cost constraints identified as high-severity failures.
• Safety Critics: Neural networks trained to predict constraint violation probability, acting as a learned detection mechanism.
• Runtime Monitoring & Shields: Control Barrier Functions or rule-based shields override unsafe actions in real-time, addressing failures with high severity and occurrence but low detection scores.
• Recovery Policies: Pre-computed policies to return the system to a safe state after a failure is detected.

A modern, iterative FMEA process for AI robotics:

1. Concept & Requirements: Define system boundaries and safety goals.
2. Simulation FMEA: Analyze failure modes in the digital twin. Use fault injection to probe robustness.
3. Mitigation in Training: Apply domain randomization, adversarial training, and safe RL algorithms to reduce RPN.
4. Hardware-in-the-Loop FMEA: Integrate physical components (sensors/actuators) with simulation to test interfaces.
5. Field Testing & Validation: Deploy with runtime monitors active. Log anomalies as new failure modes for the next cycle. This creates a closed-loop, Evaluation-Driven Development process for safety.

FMEA does not exist in isolation; it is part of a broader safety engineering ecosystem:

• ISO 21448 (SOTIF): Focuses on mitigating risks from performance limitations of perception and decision systems, a perfect complement for AI FMEA.
• ISO 26262 / IEC 61508: Define Safety Integrity Levels (SIL). FMEA helps argue for and achieve a target SIL.
• Hazard and Operability Study (HAZOP): A broader, more qualitative hazard analysis that often precedes a detailed FMEA.
• Formal Verification: Provides mathematical proofs for specific high-criticality failure modes identified by FMEA (e.g., proving a barrier function is invariant).

A complementary testing technique to FMEA's theoretical analysis. Fault injection actively induces errors—such as bit flips, network delays, or sensor failures—into a running system to empirically evaluate its robustness and fault tolerance. While FMEA predicts potential failures, fault injection validates the system's actual response.

• Types: Software-implemented fault injection (SWIFI), hardware fault injection, and simulation-based fault injection.
• Application: Critical for testing runtime monitors, fail-safe modes, and recovery policies identified during FMEA.

A structured, team-based risk assessment methodology often used in conjunction with FMEA. HAZOP systematically examines a process or system using guide words (e.g., "NO," "MORE," "LESS," "REVERSE") applied to parameters to identify potential deviations from design intent and their hazardous consequences.

• Scope: HAZOP often focuses on process flow and operational hazards, while FMEA is typically more component- or function-oriented.
• Synergy: HAZOP can identify high-level hazards, which are then decomposed for detailed analysis via FMEA.

A mathematical approach to proving the correctness of a system's design with respect to a formal specification. Unlike the qualitative/quantitative analysis of FMEA, formal verification uses model checking or theorem proving to offer mathematical guarantees that certain failure modes (e.g., deadlock, violation of a safety property) cannot occur under defined assumptions.

• Relationship to FMEA: Formal verification can be used to rigorously validate the safety constraints and control logic identified as critical in an FMEA.

A reactive safety layer that operates alongside a deployed system. Runtime monitoring continuously observes system execution (e.g., state variables, actuator commands) to detect violations of pre-defined safety properties or constraints in real-time. It acts as a last line of defense for failures not mitigated during design.

• Implementation: Often uses Control Barrier Functions (CBFs) or safety critics to evaluate risk.
• Link to FMEA: The monitors and trip points for runtime systems are frequently derived from the high-severity/high-occurrence failure modes identified in the FMEA.

The primary mathematical framework for Safe Reinforcement Learning (Safe RL). A CMDP extends a standard MDP by adding cost functions and constraints on the expected cumulative cost. This formalizes the problem of learning a policy that maximizes reward while respecting safety limits.

• FMEA Integration: The failure modes and their severity from an FMEA can be encoded as constraints or cost functions within a CMDP formulation to guide safe policy learning in simulation.

A specific Safe RL technique where a protective "shield" intervenes during an agent's operation. The shield is a pre-computed or learned safety module that overrides any action proposed by the learning agent that would lead to a constraint violation or unsafe state.

• Mechanism: The shield acts as a runtime monitor and action masker combined.
• Design Basis: The shield's logic is fundamentally informed by a prior safety analysis, such as FMEA, which defines the unsafe states and prohibited actions.