Training Data Attribution

A classic statistical method that estimates the impact of a single training point by calculating the change in model parameters if that point were removed. Influence functions use the Hessian of the loss to approximate this counterfactual without expensive retraining.

• Up-weighting: Measures how a prediction changes when a training sample is slightly up-weighted.
• Computational Cost: High for deep neural networks due to the inverse Hessian-vector product calculation.
• Use Case: Identifying mislabeled examples or outliers that disproportionately skew the decision boundary.

A scalable gradient-based method that estimates training example influence by tracking the inner product of gradients during the training process. TracIn approximates the cumulative reduction in loss on a test point attributable to a specific training point across checkpoints.

• Checkpoint Averaging: Aggregates influence over multiple saved model states to improve robustness.
• Self-Influence: Identifies training points that are hard to memorize or are inherently noisy.
• Provenance: Effectively traces which batches contributed most to a specific factual recall in LLMs.

An application of cooperative game theory to machine learning that assigns a fair value to each training datum based on its marginal contribution to model performance. The Shapley value is the weighted average contribution of a data point across all possible subsets of the training set.

• Equitability: Satisfies axioms of fairness, including symmetry and additivity.
• Monte Carlo Approximation: Exact calculation is NP-hard; practical implementations use truncated permutation sampling.
• Valuation: Used to compensate data providers or prune low-value, noisy samples from the training corpus.

A direct counterfactual approach that retrains the model on subsets of data to measure impact. Data removal studies (leave-one-out or leave-subset-out) provide ground-truth attribution but are computationally prohibitive for large models.

• Gold Standard: Serves as the empirical benchmark for validating approximate methods like influence functions.
• Sub-sampling: Randomly dropping large cohorts of data to measure aggregate impact on specific model capabilities.
• Domain Specificity: Often the only reliable method for complex emergent behaviors in generative models where gradient approximations fail.

A retrieval-based attribution technique that identifies influential training samples by finding nearest neighbors in the model's embedding space. This method assumes that a test input is influenced by training points that activate similar internal representations.

• Embedding Indexing: Uses vector databases to store and query intermediate layer activations.
• Non-parametric: Does not require access to model gradients or loss functions, making it suitable for black-box API models.
• Copyright Detection: Effective for identifying verbatim memorization and near-duplicate training images in diffusion models.

Techniques specifically designed to identify when a model has verbatim memorized training sequences rather than learned generalizable patterns. This is crucial for mitigating regurgitation of Personally Identifiable Information (PII) or copyrighted code.

• Canary Extraction: Inserting unique, out-of-distribution strings into training data to test if the model can be prompted to output them.
• Sequence Likelihood: Measuring the perplexity difference between a sequence and its counterfactual to flag memorized content.
• Differential Testing: Comparing outputs against the training corpus using suffix arrays to detect exact substring matches.