Chat Template

Explicitly separates participants in a dialogue using reserved tokens. Common roles include system (high-priority instructions), user (the human), and assistant (the model). This prevents the model from confusing its own output with user input or developer directives, establishing a clear prompt injection boundary.

Defines the sequential boundaries of a multi-turn dialogue. Each turn is wrapped in specific start and stop tokens, allowing the model to distinguish between a single long message and a history of exchanges. This is critical for maintaining session state and preventing context collapse, where distinct temporal states flatten into noise.

A widely adopted JSON-based schema that uses <|im_start|> and <|im_end|> tokens to encapsulate messages. Example:

• <|im_start|>system\nYou are a helpful assistant.<|im_end|>
• <|im_start|>user\nHello!<|im_end|>
• <|im_start|>assistant\nHi there!<|im_end|> This structure is directly tokenized to create the attention mask.

Provides a dedicated mechanism to inject a high-priority system prompt at the start of the context. The template ensures these instructions are never overwritten by the user. This is often combined with prompt caching, where the static system prompt's computed embeddings are stored to reduce latency on subsequent turns.

Translates the structured template directly into a sequence of token IDs that the model's tokenizer understands. The template dictates how the KV-cache is populated, ensuring that the Key and Value tensors for previous turns are correctly aligned with the new user message during autoregressive generation.

Ends the input sequence with the start token for the assistant role, signaling the model to begin generation. For example, appending <|im_start|>assistant\n tells the model to complete the assistant's turn. This leverages the causal attention mask to predict the next token based on all preceding context.

A high-priority instruction block placed at the beginning of the context window. It defines the model's persona, behavioral constraints, and output formatting rules. Unlike user messages, the system prompt is typically immutable during a session to prevent prompt injection. In a ChatML template, this is the <|im_start|>system role.

The maximum span of tokens a model can attend to. The chat template must fit the entire conversation history—including the system prompt, user turns, and assistant responses—within this limit. Exceeding it triggers context window truncation, where the oldest tokens are discarded, potentially causing the model to forget initial instructions.

The process of converting raw text into integer token IDs before feeding them to the model. A chat template must be applied after tokenization. Inconsistent tokenization between the template's special tokens (e.g., <|im_start|>) and the model's vocabulary will cause parsing errors. Always use the model's native tokenizer.

The logical delimiter that separates untrusted user input from trusted developer instructions. A robust chat template enforces this boundary by wrapping user content in a distinct role block. Without strict delineation, a malicious user can inject system or assistant directives to override the original prompt.

A memory optimization that stores the Key and Value tensors of previously computed tokens. When using a chat template, the static prefix (system prompt) benefits from prompt caching—its KV-cache is reused across turns. This dramatically reduces time-to-first-token latency for multi-turn conversations.