Controlled Generation

A parameter that directly modifies the raw prediction scores (logits) of specific tokens before the softmax sampling step. By adding a positive or negative bias value to targeted token IDs, developers can forcefully increase or decrease the probability of certain words appearing in the generated output. This is a foundational technique for implementing hard constraints, such as banning specific terminology or enforcing the inclusion of required keywords in a summary.

A controlled generation method that forces a language model to output text that strictly conforms to a predefined formal grammar or JSON schema. Unlike probabilistic prompting, this technique masks the logits of any token that would violate the syntactic rules of the target format, guaranteeing structurally valid output. It is essential for applications requiring machine-parseable responses, such as API calls or structured data extraction.

A generation technique that improves text quality by searching for tokens that maximize the probability difference between a strong expert model and a weaker amateur model. The core insight is that desirable behaviors like factuality and coherence are amplified in larger models, while repetitive or generic patterns are common to both. By contrasting their logit distributions, the decoding process filters out bland or hallucinatory outputs.

A decoding strategy that prevents a language model from generating any sequence of n tokens that has already appeared in the preceding context. During autoregressive generation, if a candidate n-gram matches a previously seen sequence, its probability is set to zero. This effectively eliminates repetitive phrasing at a granular level, ensuring more diverse and natural-sounding summaries without requiring model retraining.

A decoding strategy that contrasts the logit outputs from a later, mature transformer layer against an earlier, premature layer within the same model. The mature layers contain more factual knowledge, while early layers rely on superficial linguistic patterns. By amplifying the difference between these internal representations, DoLa surfaces factual knowledge and reduces hallucinations without requiring an external contrastive model.

A parameter in decoding or retrieval algorithms that penalizes repetition and encourages the selection of semantically varied tokens or passages. Often implemented via techniques like Maximum Marginal Relevance (MMR), it balances the relevance of a candidate piece of information against its similarity to content already selected. This produces more comprehensive and non-redundant summaries, especially in multi-document synthesis.