Explainable AI (XAI)

A game-theoretic approach to explain the output of any machine learning model by computing the marginal contribution of each feature to a specific prediction. In pharmacovigilance, SHAP values are used to interpret adverse event classification decisions, showing precisely which tokens in a clinical narrative drove a model to flag a case as serious.

• Based on Shapley values from cooperative game theory
• Provides both local (single prediction) and global (model-wide) explanations
• Example: A model flags a report for Stevens-Johnson Syndrome; SHAP highlights the tokens "rash," "mucosal involvement," and "lamotrigine" as the highest contributors

A technique that explains individual predictions by perturbing the input and learning a simpler, interpretable surrogate model locally around that prediction. For drug safety, LIME can explain why a specific Individual Case Safety Report (ICSR) was flagged for manual review.

• Model-agnostic: Works with any black-box classifier
• Generates explanations by creating synthetic samples near the instance of interest
• Useful for causality assessment workflows where reviewers need to understand automated triage decisions
• Example: Perturbing words in a clinical note to see which removals change the predicted seriousness criteria classification

A gradient-based attribution method that assigns importance scores to input features by accumulating gradients along a path from a baseline input to the actual input. This technique satisfies the completeness axiom, ensuring that feature attributions sum to the difference between the model's output and the baseline.

• Particularly effective for transformer-based models used in medical NLP
• Identifies which words in an Adverse Event Mention most influenced a signal detection model
• Example: For a disproportionality analysis flag, Integrated Gradients reveals that the temporal phrase "three days after starting" was the decisive factor in linking the drug to the event

A method specific to transformer architectures that visualizes the attention weights between tokens, showing which parts of the input the model focused on when making a prediction. In pharmacovigilance, attention maps can reveal whether a model correctly attended to drug-event pairs or was distracted by irrelevant text.

• Displays multi-head attention patterns across layers
• Helps validate that models focus on clinically relevant relationships
• Can expose spurious correlations where the model attends to formatting artifacts rather than clinical content
• Example: Visualizing that a model correctly attends to "atorvastatin" and "myalgia" while ignoring boilerplate disclaimer text

A technique that generates minimal changes to an input that would alter the model's prediction, answering the question: "What would need to be different for the outcome to change?" For drug safety officers, this provides actionable insight into decision boundaries.

• Generates "what-if" scenarios by modifying input features
• Identifies the smallest set of changes required to flip a classification
• Example: A model classifies a case as non-serious; a counterfactual shows that adding "patient hospitalized" would change it to serious, confirming the model's alignment with regulatory seriousness criteria
• Supports causality assessment by revealing which clinical details are decision-critical

A model-agnostic technique that produces high-precision rules called anchors, which are sufficient conditions that guarantee a prediction with high confidence. Unlike LIME's local approximations, anchors provide if-then rules that are easy for pharmacovigilance reviewers to validate.

• Generates human-readable decision rules
• An anchor "anchors" the prediction: if the rule is present, the prediction is almost certainly fixed
• Example: For an expectedness classifier, an anchor might be: "IF 'dyspnea' AND 'anaphylaxis' are present THEN predict 'serious' with 98% precision"
• Useful for creating auditable documentation of model behavior for regulatory submissions

A game-theoretic approach to model interpretability that assigns each feature an importance value for a particular prediction. SHAP values quantify the marginal contribution of each input token—such as a specific drug mention or symptom descriptor—to the final adverse event classification. This allows safety reviewers to see exactly which words in a clinical narrative drove a model to flag a case as a potential signal, moving beyond correlation to causal feature attribution.

A technique that explains individual predictions by perturbing the input and learning a simpler, interpretable surrogate model locally around that prediction. In pharmacovigilance, LIME can generate explanations for a single Individual Case Safety Report (ICSR) by identifying which combinations of symptoms and temporal phrases most influenced the model's causality assessment, regardless of the underlying algorithm's complexity.

A method specific to transformer-based architectures that visualizes the attention weights between tokens in an input sequence. For a clinical language model processing a narrative, attention maps can reveal that the model linked the phrase 'rash developed' with 'Drug X' while ignoring a negated mention of 'no headache.' This provides a direct window into the model's contextual reasoning path for entity linking and relation extraction.

An explanation method that answers the question: 'What minimal change to the input would alter the prediction?' For a causality assessment model, a counterfactual might state: 'If the temporal relationship was negative instead of positive, the predicted causality would change from Possible to Unlikely.' This helps drug safety officers understand the decision boundary and the specific clinical criteria driving the model's logic.

A gradient-based attribution method that satisfies the axioms of sensitivity and implementation invariance. It computes the average gradient of the model's output relative to the input along a path from a baseline to the actual input. For an adverse event classifier, this highlights the specific tokens in a clinical document that cumulatively contribute to a positive signal detection, offering a mathematically rigorous alternative to raw gradient saliency maps.

Testing with Concept Activation Vectors (TCAV) provides explanations in terms of high-level, human-friendly concepts rather than raw input features. A pharmacovigilance team can define a concept like 'anaphylaxis severity' using a set of example texts. TCAV then quantifies the sensitivity of a signal detection model to that clinical concept, enabling validation that the model's internal representation aligns with established medical ontology definitions.