Self-Refine

The core mechanism is a cyclical process executed by a single LLM. The model first generates an initial output for a given task. It then enters a self-feedback phase, where it critiques its own output based on the task constraints. Finally, it uses this critique to produce a refined output. This loop can repeat multiple times, with each iteration using the previous output and feedback as context, leading to progressively higher-quality results.

Self-Refine is not a fine-tuning method but a prompting architecture. The generator, feedback, and refiner roles are instantiated within the model's context window using carefully designed few-shot examples. A typical prompt includes:

• Task description: The initial instruction.
• Few-shot examples: Demonstrations of a draft output, a critique of that draft, and the resulting refined version.
• Inference instruction: A command to apply the same draft-critique-refine process to a new input.

The quality of the critique is the linchpin of the framework. The model is prompted to generate actionable, specific feedback rather than vague judgments. Effective feedback identifies concrete flaws, such as logical gaps, missing constraints, or stylistic inconsistencies. The framework can be adapted to solicit feedback on multiple dimensions (e.g., factual accuracy, code efficiency, tone) by structuring the feedback prompt to address each dimension sequentially.

A key insight is that the model jointly optimizes both the generation and the critique. The act of generating a critique forces the model to re-evaluate its own output distribution, effectively performing a form of internal self-consistency check. This process allows the model to catch errors that are often invisible during a single forward pass, such as subtle contradictions or arithmetic mistakes, by explicitly verbalizing a critical review.

The framework is universally applicable across task modalities. It has demonstrated significant performance gains in:

• Dialogue Response Generation: Improving specificity and empathy.
• Code Optimization: Refining for time complexity and readability.
• Mathematical Reasoning: Correcting calculation steps.
• Acronym Generation: Ensuring relevance and memorability.
• Constrained Text Generation: Adhering to strict lexical rules.

While performance generally improves with iterations, the framework is subject to diminishing returns and potential over-correction. A stopping condition is often implemented by comparing the current output to the previous one. If the model's feedback indicates 'no further improvements needed' or if the semantic similarity between consecutive outputs exceeds a threshold, the loop terminates to prevent unnecessary computation or degradation.

An agentic framework that uses verbal reinforcement learning to improve reasoning. Unlike Self-Refine's immediate feedback loop, Reflexion stores failure signals and self-reflections in episodic memory, allowing the agent to learn from mistakes across multiple episodes and tasks.

• Uses a long-term memory module to persist critiques
• Applies lessons learned to future, unseen tasks
• Combines trial-and-error with linguistic self-analysis

A fact-checking framework where a model systematically reduces hallucination by generating a series of independent verification questions after its initial response. It then answers each question and cross-references the results to produce a corrected final output.

• 4-step process: Draft, Plan Verifications, Execute, Finalize
• Explicitly separates generation from fact-checking
• Reduces hallucination rates without external retrieval

A bootstrapping technique that trains a model on its own correctly reasoned answers. The model generates rationales for problems, and those that lead to correct final answers are used as training data for fine-tuning. Incorrect rationales are discarded or corrected with hints.

• Iteratively improves intrinsic reasoning capabilities
• No external supervision required beyond final answer verification
• Enables models to self-improve on complex reasoning tasks

A training methodology that provides feedback on each intermediate step of a reasoning chain, not just the final outcome. This contrasts with outcome supervision, which only evaluates the final answer. Process supervision is critical for training models to produce faithful, logically sound reasoning traces.

• Rewards correct logical progression
• Reduces post-hoc rationalization
• Used to train Process Reward Models (PRMs)

A decoding strategy that samples multiple diverse reasoning paths for a single problem and selects the most consistent final answer via majority voting. While Self-Refine improves a single chain, Self-Consistency exploits the wisdom of the crowd across multiple independent chains.

• Complements Self-Refine for complex reasoning tasks
• Effective when multiple valid paths lead to the same answer
• Increases reliability without additional training

A quantitative score measuring the degree to which a generated reasoning trace causally represents the model's true computational process. Self-Refine outputs must be evaluated for faithfulness to ensure the critique and refinement steps reflect genuine error correction rather than post-hoc rationalization.

• Distinguishes genuine reasoning from confabulation
• Critical for auditing Self-Refine pipelines
• Often measured via counterfactual interventions