Data Sharding

The dataset is divided into disjoint shards where no data point exists in more than one shard. This strict isolation ensures that a deletion request affects only a single subset. The partitioning strategy often uses a deterministic hash function on a primary key to guarantee consistent assignment and prevent data leakage between shards during incremental unlearning operations.

Each shard trains a completely separate model instance with its own weights and parameters. This architectural choice means that when a user requests data deletion, only the model associated with the contaminated shard requires retraining or unlearning. The other replicas remain untouched, preserving their learned knowledge and avoiding catastrophic forgetting across the entire system.

Sharding transforms the computational cost of unlearning from O(n) to O(1) relative to total dataset size. Instead of retraining a monolithic model on the entire remaining corpus, engineers retrain only the single affected shard model on its cleaned subset. This makes compliance with the Right to be Forgotten economically feasible for large-scale production systems.

The Sharded, Isolated, Sliced, and Aggregated framework formalizes this approach:

• Sharding: Partition data into disjoint subsets
• Isolation: Train independent models per shard
• Slicing: Further divide each shard into incremental slices for checkpointing
• Aggregation: Combine outputs via ensembling at inference time This structure limits retraining to the smallest affected slice when a deletion request arrives.

Since multiple independent models exist, a voting or averaging mechanism combines their outputs at inference time. Common strategies include:

• Majority voting for classification tasks
• Arithmetic mean for regression outputs
• Confidence-weighted averaging based on each model's validation score The aggregation layer must be stateless and fast to avoid introducing latency penalties.

Sharding requires robust data lineage tracking to map every training point to its exact shard and slice. This demands:

• Immutable manifest files recording hash-to-shard mappings
• Tombstone records to prevent accidental re-ingestion of deleted data
• Versioned shard snapshots for audit compliance The provenance infrastructure is as critical as the model architecture itself for verifiable unlearning.

The Sharded, Isolated, Sliced, and Aggregated framework is the primary architecture that operationalizes data sharding for unlearning. It partitions data into mutually exclusive shards, trains independent constituent models on each shard, and aggregates their outputs. When a deletion request arrives, only the model trained on the shard containing the target data requires retraining, reducing computational cost from O(n) to O(1) relative to dataset size.

A strategy that processes deletion requests sequentially by isolating affected shards. Key characteristics:

• Targeted retraining: Only the sub-model for the contaminated shard is rebuilt
• Ensemble continuity: Unaffected shard models continue serving predictions without interruption
• Linear scaling: Unlearning cost scales with shard size, not total dataset size This approach transforms unlearning from a batch crisis into a routine maintenance operation.

The prerequisite for effective sharding-based unlearning. Data lineage tracks the origin, movement, and transformation of every data point throughout the ingestion and training pipeline. Without precise lineage:

• Shard contamination becomes undetectable
• Deletion requests cannot be mapped to specific shards
• Audit trails break, violating GDPR accountability principles Lineage metadata must include shard assignment timestamps and model version identifiers.

After data is deleted from a shard, a tombstone record persists as a metadata marker. This serves critical functions:

• Re-ingestion prevention: Prevents accidentally reintroducing deleted data into the training pipeline
• Audit trail integrity: Provides immutable proof that a deletion occurred at a specific timestamp
• Distributed consistency: In federated sharding architectures, tombstones propagate deletion state across nodes Tombstones are the persistent memory of forgetting.

The primary risk when retraining a shard model in isolation. Catastrophic forgetting occurs when a neural network abruptly loses previously learned knowledge upon learning new information. In the sharding context:

• Retraining a single shard model on the remaining data can shift its decision boundaries
• The aggregation layer must be recalibrated to prevent the ensemble from degrading
• Techniques like Elastic Weight Consolidation apply quadratic penalties to preserve important weights during shard retraining

The empirical process of proving that a shard model no longer retains influence from deleted data. Methods include:

• Membership Inference Attacks: Test whether the model behaves differently on deleted vs. non-deleted samples
• Shadow Model Testing: Train surrogate models on similar distributions to predict the target model's behavior
• Backdoor Triggers: Inject known patterns during training and verify they no longer activate post-unlearning Verification is the only way to transform a sharding architecture from a technical claim into a compliance guarantee.