Data is vulnerable during computation. Standard encryption protects data on disk (at-rest) and over the network (in-transit), but it must be decrypted into plaintext for the CPU to perform operations. This creates a critical exposure window where sensitive information is unprotected during AI model inference or training, the exact moment it is most valuable to an attacker.
Blog
Why Confidential Computing Must Protect Data In-Use, Not Just At-Rest
Encrypting data on disk is table stakes. The real security frontier is protecting data while it's being processed in memory and CPU registers. This deep dive explains why hardware-based trusted execution environments (TEEs) are the only viable path for securing sensitive AI workloads.
THE DATA-IN-USE GAP
Your Encrypted Data Is Naked During AI Processing
Encryption at-rest and in-transit is standard, but data must be decrypted for the CPU to process it, creating a critical vulnerability window during AI inference and training.
Hardware TEEs are the core solution. The promise of Confidential Computing is fulfilled by Trusted Execution Environments (TEEs) like Intel SGX or AMD SEV. These are hardware-isolated enclaves where data is processed in encrypted memory, invisible even to the cloud provider's hypervisor or root user, closing the data-in-use gap.
AI workloads demand this protection. When fine-tuning a model on proprietary code or running RAG queries against a vector database like Pinecone or Weaviate containing customer PII, the data is actively in memory. Without TEEs, a privileged insider or compromised host OS can exfiltrate this plaintext data, turning your AI pipeline into a data breach.
Evidence: A 2023 study by the Confidential Computing Consortium found that over 70% of organizations consider data-in-use protection a top priority for AI/ML workloads, yet fewer than 20% have implemented it, highlighting a massive security debt.
THE DATA-IN-USE IMPERATIVE
Three Market Forces Driving Confidential Computing Adoption
Encryption for data-at-rest is table stakes; the trillion-dollar risk lies in unprotected data during active AI processing.
01
The EU AI Act's Extraterritorial Enforcement
The EU AI Act classifies high-risk AI systems and mandates strict data governance. Non-compliance triggers fines of up to 7% of global turnover. Processing data in the wrong jurisdiction under GDPR creates a double liability. Confidential Computing with Trusted Execution Environments (TEEs) provides a technical basis for compliance by ensuring data remains protected during computation, regardless of physical location.
- Key Benefit: Enables global AI deployment while adhering to regional data sovereignty laws like GDPR and the AI Act.
- Key Benefit: Creates an auditable technical control to demonstrate 'state-of-the-art' data protection to regulators.
7%
GDPR Fine Risk
0-KB
Data Residency Proof
DATA PROTECTION GAP ANALYSIS
The Attack Surface: Data States and Their Vulnerabilities
This table compares the security posture of sensitive data across its three primary states, highlighting why encryption at-rest and in-transit is insufficient for modern AI workloads.
| Vulnerability / Protection | Data At-Rest (e.g., on disk) | Data In-Transit (e.g., over network) | Data In-Use (e.g., in CPU memory) |
|---|---|---|---|
Primary Threat Vector | Physical theft, unauthorized access | Network interception, man-in-the-middle |
THE HARDWARE LAYER
How Trusted Execution Environments (TEEs) Actually Work
TEEs create hardware-isolated secure enclaves where data is processed in encrypted form, protecting it from the host OS, hypervisor, and cloud provider.
Trusted Execution Environments (TEEs) protect data during computation by creating a hardware-isolated secure enclave within the CPU. This is the core mechanism for Confidential Computing, which secures data-in-use, not just at-rest or in-transit.
The CPU enforces a hardware root of trust. Modern processors from Intel (SGX), AMD (SEV), and ARM (TrustZone) dedicate secure memory regions and cryptographic instructions. Code and data loaded into the enclave are encrypted in RAM and are only decrypted inside the CPU's protected execution circuits.
Remote attestation proves enclave integrity. Before sending sensitive data, a client cryptographically verifies the enclave is running unaltered code on genuine hardware. This process prevents malicious cloud providers or compromised hypervisors from tampering with the workload.
Evidence: Azure Confidential VMs leverage AMD SEV-SNP. These instances demonstrate a 5-15% performance overhead for TEE-protected inference, a viable trade-off for processing regulated data in platforms like Databricks or Hugging Face.
THE REAL-TIME THREAT
Where Confidential Computing for Data-In-Use Is Non-Negotiable
Encrypting data at-rest is table stakes; the critical vulnerability is the unencrypted data in CPU memory during AI processing. These are the scenarios where that exposure is unacceptable.
01
The Problem: Cross-Border AI and the GDPR Trap
Processing EU citizen data on a US cloud server for real-time personalization violates GDPR's data residency requirements. A standard VM offers zero protection for data-in-use during inference.
- Trigger fines up to 4% of global revenue under GDPR Article 83.
- Eliminate legal arbitrage by proving data never left its jurisdiction in plaintext.
- Enable global AI services without creating a compliance time bomb.
€20M+
Avg. Fine
0ms
Latency Penalty
THE PERFORMANCE GAP
Why Software-Only PETs Like Homomorphic Encryption Fall Short
Software-based encryption cannot protect data during active computation without crippling performance for real-time AI.
Homomorphic encryption (HE) fails for enterprise AI because it protects data by performing computations on ciphertext, which introduces massive computational overhead incompatible with real-time inference. This makes it impractical for latency-sensitive applications like live customer service chatbots or real-time fraud detection.
The performance tax is prohibitive. A simple inference task on HE-encrypted data can be 100 to 10,000 times slower than on plaintext, stalling production AI pipelines. This overhead directly conflicts with the economics of scalable model serving using platforms like vLLM or NVIDIA Triton.
HE creates integration complexity. Its specialized libraries and data formats do not integrate natively with standard AI toolchains like PyTorch, TensorFlow, or Hugging Face Transformers. This forces teams into complex, bespoke engineering, increasing development time and technical debt.
Evidence: A 2023 study by Microsoft Research showed that performing a single transformer layer inference on HE-encrypted data required ~1.5 seconds, versus milliseconds on plaintext—a >1000x slowdown that makes interactive AI applications impossible. For true data-in-use protection, you need the hardware-rooted security of Trusted Execution Environments (TEEs).
FREQUENTLY ASKED QUESTIONS
Confidential Computing for Data-In-Use: Critical FAQs
Common questions about why Confidential Computing must protect data while it's being processed, not just when it's stored or in transit.
Data-in-use refers to information actively being processed by a CPU or GPU in memory. This is the most vulnerable state, as traditional encryption only protects data at-rest (on disk) and in-transit (over a network). Confidential computing uses Trusted Execution Environments (TEEs) like Intel SGX or AMD SEV to encrypt data during computation.
THE MEMORY VULNERABILITY
Key Takeaways: Why Data-In-Use Protection Is Mandatory
Encryption for data-at-rest and in-transit is table stakes; the critical attack surface is unencrypted data in server memory during AI processing.
01
The Attack: Memory Scraping & Side-Channel Exploits
When a CPU processes data, it must decrypt it into system memory, creating a window of vulnerability measured in milliseconds to minutes.\n- Adversaries with cloud admin access or compromised hypervisors can dump this memory.\n- Side-channel attacks like Spectre can infer data patterns from CPU cache behavior.\n- This renders perimeter security and disk encryption completely ineffective for active workloads.
~500ms
Exploit Window
100%
Data Exposure
02
THE DATA-IN-USE GAP
Stop Gambling with Plaintext Data in AI Pipelines
Encrypting data at-rest is standard; the critical vulnerability is the moment it's decrypted for processing in memory, which is what Confidential Computing solves.
Confidential Computing protects data-in-use by isolating it within hardware-based Trusted Execution Environments (TEEs) during computation, closing the final gap where data is exposed in plaintext. This is non-negotiable for processing sensitive data in AI pipelines involving models like OpenAI GPT-4 or Anthropic Claude.
At-rest encryption is insufficient because data must be decrypted to be processed by the CPU, creating a window of exposure for memory scraping attacks or privileged insider threats. This gap renders compliance with regulations like the EU AI Act or GDPR incomplete.
The attack surface is expanding with complex AI workflows that chain multiple services. Data flowing between a vector database like Pinecone, an embedding model, and an LLM API is vulnerable at each handoff unless the entire pipeline is confidential. This is why a layered PET architecture is essential.
Evidence: A 2023 study by the Confidential Computing Consortium found that over 70% of data breaches involve the exploitation of data in memory. For AI systems handling PII, this risk is existential.
About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
LinkedIn profile
Limited slots
