ReAct

ReAct prompts generate sequences that alternate between thoughts (internal reasoning traces) and actions (executable commands). This interleaving allows the model to perform dynamic, high-level planning while also interacting with an external environment to ground its reasoning in real-world observations. Unlike standard Chain-of-Thought, which is a static internal monologue, ReAct's action steps enable information retrieval and state modification.

• Thought: 'I need to find the distance from San Francisco to Los Angeles.'
• Action: search[San Francisco to Los Angeles distance]
• Observation: 'The driving distance is approximately 383 miles.'
• Thought: 'Now I can calculate the travel time at 60 mph.'

A defining feature of ReAct is its ability to process observations from an external environment, such as a search engine, database, or API. After an action is executed, the resulting text is fed back into the model's context as an observation. This creates a closed-loop system where the model can adapt its plan based on new information, correct its own errors, and synthesize data from multiple sources to arrive at a final, grounded answer.

ReAct excels at tasks that require both internal knowledge and external information. The reasoning trace helps the model decompose a complex question into a series of answerable sub-questions, while the action steps retrieve the necessary facts. This synergy reduces hallucination by grounding the model's internal reasoning in verifiable external data and improves interpretability by making the entire problem-solving process transparent and auditable.

The ReAct framework requires a pre-defined action space—a set of allowable commands the model can generate. A typical action space for a knowledge task includes:

• search[query]: Queries an external knowledge base.
• lookup[string]: Finds an exact string match on a page.
• finish[answer]: Signals task completion and returns the final answer.

This constrained grammar ensures the model's outputs are parseable by an external executor, bridging the gap between free-form text generation and structured tool use.

ReAct is implemented through carefully constructed few-shot prompts that demonstrate the thought-action-observation loop. Each example in the prompt shows a complete trajectory of interleaved reasoning and actions. This in-context learning teaches the model not just what format to use, but the strategic logic of when to reason internally and when to seek external information, enabling it to generalize the strategy to novel questions.

The observation loop provides a natural mechanism for error recovery. If a search action returns irrelevant information, the model can reason about the failure in a subsequent thought and reformulate its query. This self-correcting behavior is a key advantage over standard prompting, where a model has no way to recognize or fix a factual error once it's generated. The trace of failed actions and corrections also provides a rich log for debugging model behavior.

A paradigm where a language model is granted the ability to call external tools (calculators, search engines, code interpreters). This is the core 'Action' component of the ReAct loop.

• Key Tools: Search APIs for factual grounding, Python interpreters for precise math.
• ReAct Synergy: ReAct provides the reasoning framework to decide when and which tool to call based on the current reasoning trace.

A decoding strategy that samples multiple diverse reasoning paths and selects the most consistent final answer. This can be applied to the reasoning traces within a ReAct loop to improve reliability.

• Process: Generates k independent ReAct trajectories, then marginalizes over the final outcomes.
• Benefit: Mitigates the risk of a single faulty reasoning step derailing the entire task.

An agentic framework using verbal reinforcement learning. An agent reflects on task failure signals stored in episodic memory to improve its reasoning on subsequent attempts.

• Core Loop: Actor (ReAct) -> Evaluator -> Self-Reflection model.
• ReAct Integration: ReAct serves as the 'Actor' module, generating actions and reasoning, while Reflexion provides the long-term memory of past mistakes to refine future ReAct traces.

A reasoning trace that accurately reflects the true causal process by which the model arrived at its answer. This is a critical evaluation criterion for ReAct traces.

• Problem: Models can exhibit post-hoc rationalization, generating plausible but causally inaccurate justifications.
• ReAct Advantage: The interleaving of actions and observations in ReAct naturally constrains the reasoning, making it more likely to be faithful than a static CoT trace.

A training methodology that provides feedback on each intermediate step of a reasoning chain. This directly rewards the logical progression in a ReAct loop.

• Implementation: Uses a Process Reward Model (PRM) to score each Thought -> Action -> Observation cycle.
• Outcome vs. Process: Contrasts with outcome supervision, which only rewards the final answer. Process supervision is crucial for training robust ReAct agents that can recover from intermediate errors.