Locality Hypothesis

The hypothesis directly enables model editing techniques like ROME and MEND, which make constrained updates to a tiny subset of weights. Instead of retraining the entire network on a new dataset, engineers can target specific knowledge neurons or feed-forward layers identified via causal tracing. This is analogous to performing surgery on a model's memory, correcting a single fact (e.g., 'The CEO is Alice') without altering its general reasoning capabilities.

A core engineering objective derived from the hypothesis is achieving high edit specificity. A successful edit should change the model's output only for the intended input scope (e.g., queries about a specific person's title). It should not cause catastrophic forgetting or side effects on unrelated tasks. Evaluation suites for model edits rigorously test this by measuring performance on thousands of held-out prompts to ensure the edit's effect is locally contained.

The search for local representations drives mechanistic interpretability methods. Techniques like activation patching are used to empirically test the hypothesis by identifying causal circuits. For instance, researchers might patch activations from a forward pass containing 'Paris' into a pass about 'France' to isolate which neurons store the 'capital-of' relation. This research validates the hypothesis and provides a map for where to apply edits.

The hypothesis suggests that future dynamic neural architectures could be designed for better locality from the start. This includes models with:

• Sparse expert networks (e.g., Mixture of Experts) where knowledge is routed to specialized sub-networks.
• More modular components that are easier to interpret and edit.
• Explicit external memory systems (like in SERAC) that separate editable factual knowledge from core reasoning parameters.

If knowledge is local, then multiple independent facts should be stored in distinct parameter sets. This underpins batch editing algorithms like MEMIT, which apply hundreds of edits simultaneously. The algorithm assumes edits are largely non-interfering—updating the parameter 'location' for 'Capital of France' should not overlap with the location for 'Inventor of the telephone'. This is essential for maintaining edit robustness at scale.

A key tension arises between locality and edit generalization. While the hypothesis suggests knowledge is local, a useful edit often needs to generalize correctly to semantically related inputs (e.g., editing 'capital of France' should apply to 'What is the French capital?'). Advanced editing methods must navigate this, ensuring locality (no side effects) while achieving sufficient generalization within the intended conceptual scope, a central challenge in the field.

Model editing is a family of techniques for making precise, targeted updates to a neural network's knowledge or behavior without performing full retraining on a new dataset. It operationalizes the locality hypothesis by seeking to localize changes.

• Goal: Correct errors, update facts, or adjust behavior for a specific input class.
• Methods: Range from direct parameter updates (ROME, MEMIT) to external memory systems (SERAC).
• Contrast with Fine-Tuning: Unlike fine-tuning, which broadly updates all parameters, editing aims for minimal, surgical intervention.

Knowledge editing is a specific application of model editing focused on updating factual associations stored within a model's parameters, such as correcting an outdated fact or injecting a new piece of information. It directly tests the locality hypothesis for factual recall.

• Example: Changing a model's answer from "The CEO of Company X is Jane Doe" to "The CEO of Company X is John Smith."
• Challenge: Ensuring the edit is localized (doesn't affect other facts) and generalizes (works for related phrasings of the query).

Edit specificity is the desirable property of a model edit where the updated behavior is confined to the intended set of inputs, preventing unintended side effects on unrelated model capabilities. It is the core measurable goal of the locality hypothesis.

• Evaluation: Measured by testing model performance on a broad, unrelated benchmark (e.g., MMLU, GLUE) after an edit. A successful, specific edit shows no performance drop.
• Failure Mode: An edit with poor specificity causes catastrophic side effects, degrading general reasoning or knowledge.

Causal tracing and activation patching are mechanistic interpretability techniques used to identify the localized circuits implied by the locality hypothesis. They find where knowledge is stored.

• Causal Tracing: Identifies the specific components (neurons, attention heads) causally responsible for a behavior.
• Activation Patching: An intervention where activations from one forward pass are surgically replaced with those from another to isolate a circuit's effect.
• Role in Editing: These techniques guide editing algorithms (like ROME) by pinpointing the exact layers and neurons to modify.

Knowledge neurons are specific neurons or units within a neural network, particularly in the feed-forward layers of transformers, that are found to activate strongly in response to and are causally important for specific factual knowledge. They provide empirical evidence for the locality hypothesis.

• Discovery Method: Often identified via causal tracing or gradient-based attribution.
• Implication: Facts may be encoded in sparse, localized subnetworks, making selective editing plausible.
• Example: A specific neuron that fires for "The capital of France" and directly influences the output "Paris."

Side effect evaluation is the rigorous process of testing a model edit to ensure it has not negatively impacted the model's performance on tasks or knowledge unrelated to the intended edit. It is the primary method for quantifying an edit's adherence to the locality hypothesis.

• Standard Benchmarks: Use diverse evaluation suites like:
  • Factual Recall: zsRE, CounterFact datasets.
  • General Capability: MMLU, BIG-bench Hard.
  • Reasoning: GSM8K, LogiQA.
• A successful edit shows high accuracy on the edited fact while maintaining baseline scores on all other evaluations.