JSON Web Token (JWT)

JWTs are designed to be small enough to be sent through a URL, POST parameter, or inside an HTTP header. The token is Base64Url encoded, making it safe for transmission without additional encoding. A typical JWT is just three Base64Url strings separated by dots (header.payload.signature), minimizing overhead compared to verbose XML-based tokens like SAML. This compactness is critical for high-throughput APIs where every byte counts.

The payload carries claims—statements about an entity (typically the user) and additional data. There are three claim types:

• Registered claims: Predefined keys like iss (issuer), exp (expiration time), and sub (subject), which provide interoperable metadata.
• Public claims: Custom names defined in the IANA JSON Web Token Registry or using collision-resistant namespaces.
• Private claims: Custom claims agreed upon by the parties exchanging the token, such as role: "admin" or department: "engineering". This self-contained nature eliminates the need for a server-side session store, enabling true stateless authentication.

JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair (using RSA or ECDSA). The signature is computed over the encoded header and payload, ensuring that the token has not been tampered with in transit. A recipient can independently verify the signature without contacting the issuer. For example, a resource server can validate a JWT signed with an RSA private key by using the corresponding public key, trusting the token's claims without a centralized session lookup. This is the foundation of Zero Trust Architecture and Policy Enforcement Point patterns.

Because the token itself contains all necessary session data, the server does not need to maintain a session store or query a database on every request. This is a fundamental enabler for horizontally scaling APIs and microservices. When a user authenticates, the authorization server issues a JWT. Subsequent requests include the token in the Authorization: Bearer <token> header. Each microservice can independently validate the token and extract user context, such as OAuth 2.0 Scopes or Role-Based Access Control attributes, without a centralized state bottleneck.

The exp (expiration time) and nbf (not before) claims provide precise temporal control over token validity. This enables patterns like short-lived access tokens (e.g., 15 minutes) paired with longer-lived refresh tokens. If a token is compromised, the damage window is strictly limited. Combined with Just-In-Time (JIT) Access principles, systems can issue tokens with minimal privilege and immediate expiry, reducing the risk of privilege escalation. The iat (issued at) claim further allows servers to reject tokens that are too old, enforcing strict session lifetimes.

JWTs are the standard token format for OAuth 2.0 and OpenID Connect. The scope claim within a JWT defines the precise permissions granted to the bearer. For instance, a token might contain scope: "read:documents write:comments", enabling fine-grained Attribute-Based Access Control (ABAC). In a Retrieval-Augmented Generation pipeline, these scopes can be mapped to Document-Level Security filters, ensuring that a language model only retrieves documents the user is explicitly authorized to see. This is the mechanism behind Pre-Retrieval Filtering in secure search architectures.

The enforcement of data access controls during the retrieval phase of a RAG pipeline. When a user queries an AI system, their JWT is passed to the retriever, which applies pre-retrieval filtering to the vector index. This ensures the language model only grounds its answer on documents the user is permitted to see.

• Prevents data leakage in generated answers
• Combines JWT claims with document-level security metadata
• Critical for enterprise deployment of answer engines