Model Sharding

Unlike a centralized model server where breaching one host yields the entire model, sharding distributes parameter partitions across isolated enclaves. An attacker must simultaneously compromise all shard-holding nodes to reconstruct the complete model. Each shard contains only a partial weight matrix, rendering individual shard theft cryptographically insufficient for model extraction. This raises the attacker's cost from O(1) to O(n) where n is the shard count.

Each shard computes its portion of the forward pass using only its local parameters. Intermediate activations passed between shards represent incomplete transformations of the input data. An adversary observing inter-shard communication sees activation tensors but cannot derive the originating weights without solving a computationally hard matrix factorization problem. This property is analogous to secret sharing schemes where individual shares reveal nothing about the secret.

Production sharding deployments typically bind each shard to a dedicated Trusted Execution Environment (TEE) or secure enclave. This ensures:

• Shard memory is encrypted and inaccessible to the host OS
• Remote attestation verifies shard integrity before computation begins
• Side-channel protections prevent weight leakage via timing or power analysis The combination of logical sharding with hardware isolation creates a defense-in-depth posture against both remote and physical attackers.

Sharding naturally amplifies differential privacy guarantees. When each shard processes only a subset of parameters, the sensitivity of any single shard's output to a specific training example is reduced. Combined with noise injection at shard boundaries, the system achieves stronger privacy bounds than a monolithic model with equivalent noise parameters. This property is critical for federated learning deployments where shards map to separate data silos.

Sharded architectures can incorporate Byzantine fault tolerance (BFT) protocols to detect and mitigate malicious or faulty shards. Techniques include:

• Redundant computation across multiple shard groups with output comparison
• Zero-knowledge proofs verifying that a shard correctly executed its assigned layer operations
• Threshold voting on intermediate results before proceeding to subsequent layers This transforms the sharding layer from a passive distribution mechanism into an active adversarial detection fabric.

Model sharding is a natural substrate for Secure Multi-Party Computation (SMPC) protocols. When shards are held by mutually distrusting parties, SMPC enables joint inference without any party revealing its shard to others. The computation proceeds via additive secret sharing and oblivious transfer primitives, ensuring that even colluding parties cannot reconstruct the full model unless they control a threshold of shards exceeding the protocol's security parameter.

A cryptographic protocol that distributes a model's computation across multiple parties who jointly compute an inference result without revealing their private input shares to one another. SMPC is the mathematical foundation upon which many sharding strategies are built.

• Each shard holder performs local computation on their fragment
• Intermediate results are exchanged via obfuscated communication rounds
• No single party ever reconstructs the complete model weights
• Provides information-theoretic security when properly implemented

SMPC protocols like SPDZ and ABY3 are commonly used to coordinate sharded inference across mutually distrusting enclaves.

A hardware-enforced secure area within a main processor that guarantees the confidentiality and integrity of code and data loaded inside it. TEEs provide the isolated runtime for individual model shards.

• Intel SGX and AMD SEV are dominant TEE implementations
• Each shard executes within its own encrypted memory region
• The host operating system cannot inspect shard contents
• Remote attestation verifies enclave integrity before shard deployment

TEEs prevent even privileged users from extracting shard parameters during active computation.

A cryptographic method that allows computation to be performed directly on encrypted data, enabling a model to generate predictions without ever decrypting the user's input or the model's output.

• Fully Homomorphic Encryption (FHE) supports arbitrary computation on ciphertext
• Pairs naturally with sharding: each shard operates on encrypted intermediate tensors
• Eliminates the need to trust individual shard hosts with plaintext data
• Current limitations include high computational overhead and latency

Schemes like CKKS are optimized for the approximate arithmetic common in neural network inference.

A class of defenses that thwart attacks aiming to steal model functionality through black-box querying. Sharding raises the bar for extraction by ensuring no single node can answer queries independently.

• Attackers must compromise multiple shard hosts simultaneously
• Query rate limiting per shard prevents systematic probing
• Differential privacy noise can be injected at shard boundaries
• Threshold cryptography requires consensus among shards to produce output

Extraction becomes exponentially harder as the number of shards increases.

Defenses that eliminate or mask physical information leakage—timing, power consumption, or electromagnetic emanations—from processors running model inference on individual shards.

• Constant-time execution prevents timing-based weight inference
• Power analysis countermeasures mask computational intensity patterns
• Memory access obfuscation randomizes DRAM read/write sequences
• Bus encryption protects data in transit between CPU and external memory

Each shard host must implement these mitigations independently to prevent an attacker from exploiting the weakest node.

Techniques for securing decentralized training against gradient leakage and malicious nodes. While sharding focuses on inference-time distribution, federated learning addresses the training-time equivalent.

• Secure aggregation protocols ensure the central server sees only aggregated updates
• Gradient clipping and noising bound the information leaked per participant
• Byzantine-robust aggregation rejects anomalous updates from compromised nodes
• Sharded models can be fine-tuned via federated learning without reassembly

Both paradigms share the core principle that raw data and complete models should never be centralized.