Repetition Penalty

Repetition penalty operates by directly modifying the raw logits before the softmax function. For each token already present in the generated sequence, its logit score is divided by a hyperparameter (typically 1.2). This logit scaling reduces the probability of the model selecting the same word again without completely zeroing it out. Unlike hard bans, this allows for natural repetition of function words like 'the' while heavily discouraging the looping of rare or distinctive tokens.

Modern implementations often split the penalty into two distinct parameters:

• Frequency Penalty: Scales the penalty proportionally to how many times a token has appeared. A word used 10 times is penalized more heavily than one used twice.
• Presence Penalty: Applies a flat, binary penalty if a token has appeared at all, regardless of count. This distinction allows fine-grained control over whether to simply encourage topic diversity or strictly prevent any repetition.

The parameter typically operates on a continuous scale centered around 1.0:

• 1.0: No penalty applied. The model behaves with its default distribution.
• 1.1–1.3: A moderate setting suitable for creative writing, gently discouraging obvious phrase repetition.
• 1.5+: Aggressive penalization. High values risk breaking the model's syntax, causing it to emit nonsensical or ungrammatical output as it desperately avoids any previously used token.
• < 1.0: Inverts the logic, encouraging repetition. Rarely used outside of specific stylistic experiments.

Repetition penalty interacts non-trivially with other decoding parameters:

• Top-k Sampling: The penalty is applied before the top-k filter truncates the vocabulary. A heavily penalized token might be pushed out of the top-k set entirely.
• Temperature: High temperature flattens distributions, potentially counteracting the penalty's effect. Low temperature sharpens distributions, making the penalty's impact more binary and abrupt.
• Nucleus Sampling (Top-p): The penalty reshapes the cumulative probability mass, potentially altering which tokens fall within the nucleus.

A deterministic alternative to the probabilistic repetition penalty is n-gram blocking. This hard constraint prevents the model from generating any sequence of n tokens that already exists in the context window. For example, a 3-gram block stops any identical 3-word phrase from repeating. While effective at stopping verbatim loops, it can be too rigid for poetry or lyrics where refrains are intentional. Many production systems combine both a soft penalty and a hard n-gram block.

The penalty is typically applied against the entire current context window, including both the user prompt and the model's own generated output. This prevents the model from simply echoing the user's input back. Advanced implementations may restrict the penalty scope to only the model's recent generations or apply a decaying penalty where the penalization strength diminishes for tokens generated many steps ago, allowing the model to naturally circle back to a topic after a long digression.

A hyperparameter that controls the randomness of predictions by scaling the logits before applying softmax. A temperature of 0 makes the model deterministic, always choosing the most likely token. Higher values (e.g., 0.8–1.2) flatten the probability distribution, making less likely tokens more probable and increasing output diversity. Unlike repetition penalty, temperature does not specifically target repeated tokens but adjusts overall creativity.

A sampling method that restricts the model's next-token selection to the k most likely candidates from the probability distribution. After filtering, the model samples from this truncated set. This prevents the model from choosing highly improbable tokens that could derail coherence. Top-k is often combined with repetition penalty to ensure that even within the allowed set, already-used tokens are suppressed.

An alternative to top-k that selects the smallest set of tokens whose cumulative probability exceeds a threshold p (e.g., 0.9). This dynamically adjusts the number of candidate tokens based on the confidence of the model's predictions. When the model is very confident, few tokens are considered; when uncertain, more are included. Works synergistically with repetition penalty to maintain diversity without sacrificing coherence.

A closely related parameter that reduces the probability of a token based on its cumulative frequency in the generated text so far. Unlike repetition penalty, which applies a flat penalty to any prior occurrence, frequency penalty scales the penalty proportionally to how many times the token has appeared. A token appearing 10 times receives a much stronger penalty than one appearing twice, making it effective at curbing excessive repetition of common words.

A parameter that applies a flat, one-time penalty to any token that has appeared at least once in the generated sequence. Unlike frequency penalty, it does not scale with the number of occurrences. A token is penalized equally whether it appeared once or fifty times. This encourages the model to introduce entirely new topics and vocabulary rather than reusing any previously mentioned concepts.

A deterministic decoding constraint that absolutely forbids the generation of any n-gram (sequence of n tokens) that has already appeared in the output. For example, with 3-gram blocking, the model cannot generate any three-word sequence that exists earlier in the text. This is a hard constraint, not a probabilistic penalty, making it the most aggressive anti-repetition mechanism. Often used in summarization tasks to prevent phrase duplication.