Homomorphic Encryption Inference

HE schemes are built on lattice-based cryptography, specifically the Ring Learning With Errors (RLWE) problem. They replace standard arithmetic with operations on polynomials in a ciphertext space. Addition and multiplication gates in a neural network are mapped to corresponding homomorphic operations. The key constraint is managing noise—a random error that grows with each multiplication. If noise exceeds a threshold, decryption fails. This necessitates bootstrapping, a procedure that refreshes a ciphertext by homomorphically evaluating the decryption circuit itself, enabling unlimited computations.

The field distinguishes between two primary capabilities:

• Leveled HE (LHE): Supports computation of a pre-determined, finite-depth circuit. It avoids the extreme overhead of bootstrapping, making it practical for fixed neural network architectures where the multiplicative depth is known at compile time.
• Fully HE (FHE): Supports arbitrary computation of unbounded depth. This is achieved through bootstrapping, which resets the noise level. While mathematically elegant, FHE is significantly slower and is an active area of optimization for general-purpose inference.

Standard neural networks cannot run directly on encrypted data. They must be adapted to the constraints of HE:

• Polynomial Activation Replacement: Non-linear functions like ReLU or Sigmoid are incompatible with HE's polynomial arithmetic. They are replaced with low-degree polynomial approximations, such as x² or x⁴.
• Discrete Quantization: Floating-point weights and inputs must be quantized to fixed-precision integers to operate within the plaintext modulus of the HE scheme.
• Average Pooling Substitution: Max-pooling is a non-polynomial comparison operation; it is replaced by average pooling or structured striding.

HE inference operates under a specific security assumption: the server executing the model is honest-but-curious. It follows the protocol correctly but attempts to learn information from the data it processes. HE cryptographically guarantees that the server cannot extract the user's plaintext input or the resulting inference. It does not, however, protect against a malicious server that deviates from the protocol, returns incorrect results, or performs a side-channel attack on the execution environment.

The primary trade-off for privacy is computational cost. A single homomorphic multiplication is orders of magnitude slower than a plaintext operation. Additionally, ciphertext expansion is massive: an encrypted scalar can become a multi-megabyte object. For a typical neural network, this translates to:

• Latency: Inference can take seconds to minutes, compared to milliseconds in plaintext.
• Memory: RAM requirements can be 10,000x the model size.
• Bandwidth: Transmitting large ciphertexts between client and server introduces significant network bottlenecks.

Different HE schemes are optimized for different workloads:

• CKKS (Cheon-Kim-Kim-Song): Supports approximate arithmetic on real numbers, making it the preferred choice for neural network inference where perfect precision is not required.
• BFV (Brakerski-Fan-Vercauteren): Designed for exact integer arithmetic, suitable for computations requiring modular precision.
• TFHE (Fast Fully Homomorphic Encryption over the Torus): Optimized for very fast bootstrapping of individual bits or small integers, ideal for evaluating binary circuits and non-polynomial functions via lookup tables.

The strongest form of homomorphic encryption that supports arbitrary computation on ciphertexts, including both addition and multiplication operations without limit. Unlike partially homomorphic schemes, FHE enables the evaluation of deep neural networks entirely in the encrypted domain.

• Supports unlimited circuit depth through bootstrapping
• Enables arbitrary polynomial and non-polynomial function approximation
• Current schemes: CKKS (approximate arithmetic), BGV/BFV (exact integer arithmetic), TFHE (fast bitwise operations)
• Primary bottleneck: computational overhead of 10,000x to 1,000,000x compared to plaintext

A restricted form of homomorphic encryption that supports only one type of operation—either addition or multiplication—on ciphertexts, but not both. PHE schemes are significantly faster than FHE and practical for specific inference tasks.

• Paillier cryptosystem: supports additive homomorphism, used for secure aggregation
• ElGamal: supports multiplicative homomorphism
• Suitable for linear layers and privacy-preserving federated averaging
• Cannot evaluate non-linear activation functions without decryption

A cryptographic protocol where multiple parties jointly compute a function over their private inputs without revealing those inputs to each other. In the context of model inference, SMPC distributes the model across non-colluding servers.

• Based on secret sharing (additive, Shamir) or garbled circuits
• Requires multiple non-colluding servers for security guarantees
• Communication overhead dominates latency in WAN settings
• Often combined with HE in hybrid protocols for optimal performance

A hardware-enforced secure enclave within a processor that isolates code and data from the host operating system, hypervisor, and other applications. TEEs provide an alternative to cryptographic approaches for confidential inference.

• Implementations: Intel SGX, AMD SEV, ARM TrustZone
• Orders of magnitude faster than HE but requires hardware trust assumptions
• Vulnerable to side-channel attacks (cache timing, power analysis)
• Provides attestation to verify enclave integrity remotely

A cryptographic primitive where a decryption key reveals only a specific function of the encrypted data rather than the plaintext itself. For inference, a server holding a functional decryption key can compute the model's prediction without learning the input.

• Stronger privacy guarantee than HE: server never sees plaintext
• Key generation is function-specific and computationally intensive
• Inner-product functional encryption used for linear classifiers
• Less mature than HE; limited support for deep neural networks

A mathematical framework that provides provable privacy guarantees by adding calibrated noise to computation outputs, ensuring that the presence or absence of any single individual in the dataset cannot be reliably inferred.

• Defined by the privacy budget parameter ε (epsilon)
• Complements HE by protecting against output-based inference attacks
• Applied during training (DP-SGD) or inference (noisy predictions)
• Trade-off: lower ε means stronger privacy but reduced model accuracy