Inference Data Masking

Intercepts and transforms sensitive data in-flight during the inference request/response cycle without adding perceptible latency. The masking engine operates as a transparent proxy within the serving pipeline.

• Entity Detection: Identifies PII like names, emails, and credit card numbers using regex and NER models
• Format-Preserving Masking: Replaces [email protected] with j***.d**@e****.com while maintaining data structure
• Sub-5ms Latency: Optimized C++/Rust implementations ensure masking doesn't bottleneck inference throughput

Employs Named Entity Recognition (NER) and custom regex patterns to distinguish between sensitive and benign tokens based on semantic context, not just pattern matching.

• Contextual Disambiguation: Understands that 'Apple' in a tech context is an organization, not a fruit
• Custom Entity Types: Define domain-specific sensitive entities like patient IDs or internal project codes
• Multi-Language Support: Handles PII across 50+ languages using language-specific NER models

Administrators define granular masking policies that specify which entity types to redact, the masking algorithm to apply, and the scope of application (logs, responses, or both).

• Role-Based Visibility: Full PII visible to authenticated users, masked for anonymous access
• Algorithm Selection: Choose between redaction, substitution, encryption, or tokenization per entity type
• Policy as Code: Define masking rules in YAML/JSON, version-controlled and deployed via CI/CD pipelines

Automatically scrubs sensitive data from inference audit trails before they are written to persistent storage, ensuring compliance with GDPR, HIPAA, and PCI-DSS logging requirements.

• Pre-Write Filtering: Masks data in the logging pipeline before it reaches disk or SIEM systems
• Immutable Audit Records: Maintains a tamper-proof log of all access attempts while protecting PII
• Compliance Reporting: Generates attestations that no raw PII exists in inference logs

Replaces sensitive values with consistent, reversible tokens using a vault-based architecture, allowing downstream systems to perform analytics on masked data without exposing raw PII.

• Consistent Mapping: The same input always produces the same token, preserving referential integrity
• Vault Integration: Securely stores the mapping between original values and tokens in an encrypted vault
• De-Tokenization API: Authorized services can reverse the tokenization for legitimate business needs

Handles token-by-token streaming responses from LLMs, applying masking logic incrementally as each token is generated without buffering the entire response.

• Incremental Processing: Masks each token as it arrives, maintaining the streaming user experience
• Partial Entity Handling: Detects and buffers incomplete PII entities until the full entity is streamed
• Backpressure Awareness: Respects streaming flow control to prevent memory pressure under high load

A chronological, tamper-proof record of all access and query events stored in WORM-compliant storage. When inference data masking is applied, the audit trail must log both the original and masked values to maintain non-repudiation while proving compliance:

• Captures pre-masking and post-masking payloads
• Provides cryptographic proof that masking policies were enforced
• Essential for GDPR Article 30 record-keeping requirements
• Enables forensic reconstruction without exposing raw PII in logs

The practice of encrypting request and response bodies at the application layer, providing defense-in-depth beyond transport-level TLS. Data masking and payload encryption are complementary controls:

• Masking redacts sensitive fields within the payload structure
• Encryption protects the entire payload from eavesdropping
• Masking operates on plaintext after decryption at the endpoint
• Combined, they ensure that even if encryption is compromised, sensitive data remains obfuscated

The enforcement of a strict data contract, often defined by an OpenAPI Specification, on all inference requests. Schema validation ensures that data masking rules can be applied predictably by guaranteeing the structure of incoming data:

• Rejects malformed inputs before they reach masking logic
• Defines which fields contain PII requiring redaction
• Prevents type confusion attacks that could bypass masking
• Enables deterministic masking policies based on known field schemas

The execution of model inference within a hardware-based Trusted Execution Environment (TEE) that isolates the model and data from the host OS. Data masking within a TEE ensures that sensitive data is never exposed in cleartext to the underlying infrastructure:

• Masking policies execute inside the encrypted enclave
• Prevents cloud provider administrators from viewing raw PII
• Attestation verifies that masking code has not been tampered with
• Combines with differential privacy for mathematical guarantees

A mechanism defined by RFC 7662 that allows a resource server to query the authorization server about an access token's active state and metadata. This metadata can dynamically inform masking policies:

• Token claims may specify the caller's data access tier
• High-privilege tokens receive unmasked responses
• Low-privilege tokens trigger aggressive PII redaction
• Enables context-aware masking without hardcoding policies per endpoint