Side Effect Evaluation

The locality hypothesis is the foundational assumption that a neural network's knowledge is stored in specific, localized parameters. A successful edit should change behavior only for the target input (e.g., 'The CEO of Apple is Tim Cook') while preserving the model's original outputs for all unrelated inputs (e.g., 'The capital of France is Paris'). Side effect evaluation rigorously tests this hypothesis by measuring performance drift across a broad suite of tasks.

Side effects are measured using curated benchmark datasets. Key metrics include:

• Edit Success Score: Accuracy on the specific edited fact.
• Neighborhood Score: Performance on inputs semantically related to the edit (tests for edit generalization).
• Paraphrase Score: Performance on differently phrased versions of the edit target.
• Zeroshot Score: Performance on completely unrelated tasks from benchmarks like MMLU or BIG-bench. A significant drop in the zeroshot score indicates catastrophic side effects.

Causal tracing and activation patching are interpretability techniques used to diagnose potential side effects. By identifying the specific attention heads and feed-forward neurons (or knowledge neurons) responsible for a fact, researchers can predict if an edit targeting that circuit will interfere with other knowledge stored in overlapping parameters. This mechanistic understanding guides the development of more precise editing algorithms like ROME and MEMIT.

A core challenge in side effect evaluation is balancing two opposing goals:

• Edit Specificity: The edit should not affect unrelated inputs (minimizing side effects).
• Edit Generalization: The edit should apply to all valid paraphrases and logical entailments of the target fact. Algorithms must navigate this trade-off. An overly specific edit may fail on simple rephrasings, while an overly generalized edit may incorrectly alter unrelated facts, a phenomenon known as overwriting.

SERAC (Scalable Efficient Retrieval-Augmented Counterfactuals) represents a paradigm that minimizes parameter-side effects by design. Instead of modifying the base model's weights, it stores edits in an external memory. A scope classifier routes queries; if a query matches an edit, it's answered from memory. This architecture inherently limits side effects to the performance of the classifier, making side effect evaluation more modular.

Edit robustness is evaluated over time and across contexts. Key tests include:

• Consistency: Does the edit remain effective after thousands of forward passes?
• Portability: Can the edit be applied successfully to different model checkpoints or sizes (edit portability)?
• Compositionality: What happens when hundreds of edits are applied via batch editing? Do side effects compound? This long-term view is essential for considering model editing as a sustainable alternative to full retraining in production systems.

The locality hypothesis is the foundational assumption in model editing that a neural network's knowledge is stored in specific, localized parameters. This principle enables targeted edits that change behavior for a narrow set of inputs without degrading general performance. Techniques like causal tracing are used to validate this hypothesis by identifying critical circuits.

Edit specificity is the desirable property where a model edit's effect is confined to the intended set of inputs. It is the direct goal of side effect evaluation. High specificity means the edit successfully changes the answer for "Who is the CEO of Company X?" but does not alter the model's knowledge about other CEOs or unrelated facts. It is often in tension with edit generalization.

Causal tracing is a mechanistic interpretability technique used to identify the specific computational paths (e.g., neurons, attention heads) within a neural network that are causally responsible for a particular output. In model editing, it is used to:

• Locate where knowledge is stored to guide edits.
• Diagnose the root cause of side effects by tracing how an edit altered information flow for unrelated tasks.

Knowledge neurons are specific, often sparse, sets of neurons within a transformer's feed-forward layers that activate strongly for and are causally important for specific factual knowledge. For example, a small group of neurons may fire for the subject "Paris." Editing methods like ROME target these neurons to update facts. Side effect evaluation must test if editing these neurons inadvertently affects other knowledge they may be involved in.

Edit robustness measures the stability and persistence of a model edit over time and across varied input phrasings. A robust edit remains effective when queried with synonyms, paraphrases, or in different contexts. Side effect evaluation must assess robustness not just for the edit target, but also ensure that unrelated capabilities remain stable and do not regress due to the edit.

Batch editing is the process of applying hundreds or thousands of model edits simultaneously (e.g., updating an entire knowledge base). Algorithms like MEMIT are designed for this. Side effect evaluation becomes exponentially more critical and complex in batch editing, as the cumulative impact of many small parameter changes must be assessed to ensure global model performance is preserved.