Confidential Inference

The model and data are placed inside a hardware-enforced encrypted enclave that isolates them from the host operating system, hypervisor, and cloud provider. This creates a hardware root of trust that ensures no external process—even with root privileges—can inspect memory or exfiltrate weights during inference. The CPU verifies the enclave's integrity via remote attestation before releasing secrets.

Unlike standard encryption that protects data at rest (storage) and in transit (TLS), confidential inference closes the final gap by protecting data in use. Model weights, intermediate activations, and user inputs remain encrypted within the CPU cache and memory. This prevents memory-scraping attacks, cold-boot attacks, and privileged insider threats from accessing sensitive computation.

Before a client sends sensitive data for inference, the TEE generates a cryptographic attestation report signed by the hardware manufacturer. This report proves:

• The enclave is running on genuine, trusted hardware
• The exact model hash and runtime code loaded
• No tampering or debugging interfaces are active This establishes verifiable trust without relying on the cloud provider's word.

Proprietary model weights remain encrypted outside the CPU and are only decrypted inside the enclave. The cloud operator cannot:

• Snapshot or clone the model
• Perform offline analysis on weights
• Extract architectural details from memory dumps This enables secure model monetization where customers can use the model without ever possessing the underlying intellectual property.

Confidential computing enables scenarios where multiple distrusting parties contribute encrypted data to a joint inference without revealing their individual inputs. For example, two banks can run fraud detection on combined transaction data without exposing customer records to each other. The TEE acts as a neutral computation zone that all parties can cryptographically verify.

Confidential inference provides technical guarantees that satisfy data residency and sovereignty requirements under regulations like GDPR, HIPAA, and the EU AI Act. Because the cloud provider has no access to plaintext data, organizations can demonstrate verifiable data protection rather than relying solely on contractual agreements and audit paperwork. This is critical for processing PII and PHI in regulated industries.

A cryptographic process by which a relying party verifies the identity and integrity of a TEE before releasing secrets or sending sensitive data. The TEE produces a signed attestation report containing a measurement of its internal state.

• Hardware Root of Trust: The attestation chain anchors to a manufacturer-provisioned key fused into the silicon.
• Measurement Verification: The client compares the TEE's cryptographic hash against a known-good golden measurement of the inference software stack.
• Freshness Guarantees: Nonces prevent replay attacks on attestation evidence.

This ensures the model is running on genuine, unmodified hardware in a trusted configuration.

The hardware mechanism that transparently encrypts and integrity-protects the memory pages belonging to a TEE. The memory encryption engine sits on the processor's memory bus, encrypting data as it leaves the CPU cache and decrypting it upon return.

• Granularity: Operates at the cache-line level with unique keys per enclave or VM.
• Protection Scope: Defends against cold-boot attacks, DRAM probing, and malicious DMA access.
• Performance Impact: Modern AES-XTS hardware acceleration minimizes encryption overhead to single-digit percentage latency increases.

Without memory encryption, model weights and intermediate activations would be exposed in plaintext in system RAM.

A cryptographic scheme that allows computation directly on ciphertexts, generating an encrypted result that decrypts to the correct plaintext output. Unlike TEEs, HE provides mathematical privacy guarantees without trusting hardware.

• Fully Homomorphic Encryption (FHE): Supports arbitrary computations on encrypted data but incurs 10,000x-1,000,000x compute overhead.
• Partial HE: Supports only addition or multiplication, useful for specific linear algebra operations.
• Hybrid Approaches: Combine HE for input privacy with TEEs for efficient computation on decrypted data within the enclave.

Current HE performance remains prohibitive for large neural network inference, making TEEs the pragmatic choice for confidential inference today.

A cryptographic protocol where multiple parties jointly compute a function over their private inputs without revealing those inputs to each other. In the context of confidential inference, SMPC can split a model across non-colluding servers.

• Secret Sharing: Model weights and inputs are split into random shares distributed across parties.
• Garbled Circuits: Boolean circuits representing the neural network are encrypted and evaluated obliviously.
• Communication Overhead: Requires significant network bandwidth between parties for each multiplication gate.

SMPC eliminates hardware trust assumptions entirely but introduces latency measured in seconds per inference, making it suitable only for low-throughput, high-privacy scenarios.

Software-level defenses applied to model artifacts to resist extraction and reverse engineering when hardware TEE protections are unavailable or as a defense-in-depth layer.

• Weight Encryption: Model parameters are stored encrypted and decrypted on-the-fly only during computation.
• Code Obfuscation: The inference runtime is transformed to resist static and dynamic analysis.
• Watermarking: Imperceptible patterns are embedded into model weights to prove ownership if extraction occurs.
• Query Rate Limiting: Restricts API access to prevent the systematic probing required for black-box model stealing.

These techniques raise the cost of extraction but do not provide the formal guarantees of hardware-enforced TEE isolation.