SISA Training

The training dataset is partitioned into multiple disjoint shards before any training begins. Each shard contains a non-overlapping subset of the data, ensuring that any individual record influences only a single shard's model. When a deletion request arrives, only the shard containing the target data requires retraining.

• Shards are created using deterministic hashing or random assignment
• Typical configurations use 4-16 shards depending on dataset size
• Each shard trains an independent model replica from scratch
• Shard boundaries are fixed and immutable throughout the lifecycle

Each shard trains its model in complete isolation from other shards, with no gradient sharing, parameter synchronization, or data leakage between partitions. This isolation is the architectural guarantee that limits the blast radius of any single data point.

• No cross-shard communication during training
• Each shard maintains its own optimizer state and learning rate schedule
• Isolation prevents membership inference signals from propagating across shards
• Models can be trained in parallel on separate hardware for throughput

Within each shard, data is further divided into temporal slices that are introduced incrementally during training. The model checkpoints after each slice, creating a sequence of intermediate states that enable fine-grained unlearning without restarting from scratch.

• Slices are processed sequentially within a shard
• Each slice boundary produces a checkpoint saved to persistent storage
• Typical slice sizes range from 1-5% of the shard's data
• Enables partial rollback to the checkpoint immediately preceding the deleted record's slice

At prediction time, outputs from all shard models are combined into a single prediction through an aggregation mechanism. The ensemble nature provides both the final prediction and a natural measure of uncertainty that can be used to detect distribution shift.

• Common aggregation methods: majority voting for classification, averaging for regression
• Aggregation weights can be uniform or learned via a meta-learner
• Provides built-in uncertainty quantification through inter-shard disagreement
• Inference latency scales linearly with shard count but can be parallelized

SISA provides provable exact unlearning, meaning the final model state after a deletion request is mathematically identical to a model that was never trained on the deleted data. This is stronger than approximate unlearning methods that only bound the statistical influence.

• Retraining is scoped to a single shard and its affected slices
• Computational cost of unlearning is O(1/k) where k is the number of shards
• No residual influence remains in other shards due to isolation
• Satisfies Article 17 GDPR right to erasure requirements for ML models

The primary cost of SISA is the storage overhead from maintaining per-slice checkpoints for every shard. This creates a direct tradeoff between unlearning granularity and infrastructure cost that practitioners must tune for their deployment context.

• Storage scales as O(shards × slices) per model
• Finer slicing reduces retraining cost but increases checkpoint storage
• Checkpoint compression and delta storage can reduce overhead
• Typical production deployments balance at 8 shards with 10-20 slices each