Inferensys

Glossary

Message Layer Security (MLS)

An IETF standard for end-to-end encryption in group messaging, providing forward secrecy and post-compromise security for asynchronous communication.
Stylish WeWork-like workspace with hot desks and document wall, professional searching through enterprise knowledge base on a mounted ultrawide display, warm industrial pendants overhead.
IETF STANDARD

What is Message Layer Security (MLS)?

Message Layer Security (MLS) is an IETF standard (RFC 9420) for end-to-end encryption in group messaging, providing forward secrecy and post-compromise security for asynchronous communication.

Message Layer Security (MLS) is a cryptographic protocol standardized by the IETF to provide efficient, scalable end-to-end encryption for groups ranging from two to thousands of participants. Unlike legacy group chat protocols that rely on a central server to re-encrypt messages for each recipient, MLS uses an asynchronous ratcheting tree to enable members to derive shared group keys independently, drastically reducing computational overhead and eliminating the server as a single point of trust.

MLS delivers critical security properties including Forward Secrecy (compromised keys cannot decrypt past messages) and Post-Compromise Security (a compromised member is healed out of the group automatically). By operating at the application layer, it is transport-agnostic, making it the foundational encryption layer for secure inter-agent communication in federated systems, WebRTC conferencing, and enterprise messaging platforms.

IETF STANDARD FOR GROUP MESSAGING

Core Cryptographic Properties of MLS

Message Layer Security (MLS) provides a modern, formally verified protocol for end-to-end encryption in groups, designed to overcome the scalability and security limitations of older schemes. Its architecture delivers critical cryptographic guarantees essential for secure inter-agent communication.

01

Asynchronous Operation

MLS is designed for asynchronous messaging, meaning participants do not need to be online simultaneously to exchange messages or perform key updates. This is achieved through a Delivery Service (DS) that stores and forwards handshake messages and ciphertexts. An agent can publish a cryptographic update to the DS, and other members process it when they next come online. This decouples communication from real-time coordination, making it ideal for autonomous agent meshes where agents operate on independent schedules and network connectivity is intermittent.

02

Forward Secrecy (FS)

MLS guarantees forward secrecy by continuously ratcheting the symmetric encryption keys used to protect messages. Each time a sender commits a new message, they derive a fresh symmetric key from an evolving key schedule. If an attacker later compromises a device's long-term identity key, they cannot retroactively decrypt previously recorded ciphertexts because the ephemeral key material used to encrypt those past messages has already been securely deleted. This property ensures that the confidentiality of historical agent communications remains intact even after a future key breach.

03

Post-Compromise Security (PCS)

Beyond forward secrecy, MLS provides post-compromise security (also known as future secrecy or self-healing). If an attacker compromises an agent's current keying material, they can decrypt subsequent messages only until the next honest agent performs a key update. When any group member sends a message with an UpdatePath, they inject new, uncompromised entropy into the group's shared secret state. This cryptographic action heals the group, permanently evicting the passive attacker and restoring confidentiality for all future messages without requiring a full group re-initialization.

04

Group Key Agreement via TreeKEM

MLS uses a TreeKEM (Tree-based Key Encapsulation Mechanism) for scalable group key agreement. Members are arranged as leaves in a ratchet tree, a binary tree structure where each node holds a public-private key pair. The root node's private key is the shared group secret. When membership changes, only log(n) nodes along the path to the root need to be updated, rather than the entire group. This logarithmic scaling allows MLS to efficiently handle groups with thousands of agents, making it suitable for large-scale agent swarms.

05

Formal Verification & Security Proofs

The MLS protocol was designed with formal security analysis as a core requirement. The specification includes a detailed, machine-verified proof of its security properties in the Universal Composability (UC) framework. This rigorous mathematical treatment provides high assurance that the protocol resists a wide range of attacks, including insider threats and complex key compromise scenarios. For security architects deploying agentic systems, this formal verification reduces the risk of subtle protocol flaws that have historically plagued custom or ad-hoc group messaging solutions.

06

Scalable Dynamic Groups

MLS natively supports highly dynamic groups with efficient member addition, removal, and update operations. Adding a new agent requires only O(log n) cryptographic operations, not O(n) as in many legacy protocols. Critically, member removal provides cryptographic eviction: a removed agent is cryptographically excluded from decrypting any future messages, even if they collude with other removed members. This is essential for agentic systems where compromised or decommissioned agents must be securely and permanently expelled from the communication mesh without re-keying the entire network.

MLS PROTOCOL CLARIFIED

Frequently Asked Questions

Clear, technical answers to the most common questions about the Message Layer Security (MLS) protocol, its cryptographic foundations, and its role in securing modern group communication.

Message Layer Security (MLS) is an IETF standard (RFC 9420) for end-to-end encryption in group messaging applications. It provides forward secrecy and post-compromise security for asynchronous communication. MLS operates by maintaining a shared, continuously evolving group state called a ratchet tree. When a member sends a message, they derive symmetric encryption keys from the tree's root secret. When membership changes—a user joins or leaves—the tree is updated via a commit operation, which distributes new public keys to the group. This process ensures that a new member cannot decrypt messages sent before they joined, and a removed member cannot decrypt messages sent after their removal. Unlike older protocols like Signal's Double Ratchet optimized for two parties, MLS scales efficiently to groups of thousands by using logarithmic tree operations, making it the modern standard for secure large-scale group chat, voice, and video calling.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.