Chain-of-Thought Prompting

CoT instructs the model to decompose a complex problem into a sequence of logical sub-steps. Instead of directly mapping an input to an output, the model generates a reasoning trace that mirrors human analytical thought. This is critical for high-stakes clinical tasks like differential diagnosis, where the path to the answer is as important for auditability as the answer itself. The model might first identify key symptoms, then link them to possible pathologies, and finally rank them by likelihood.

CoT prompting is most effective on tasks requiring multi-step logic, arithmetic, or symbolic reasoning. Standard prompting often fails on these. By verbalizing the reasoning process, the model allocates more computational effort to the problem. In a clinical context, this translates to higher accuracy in tasks like drug dosage calculation based on weight and renal function, or resolving conflicting medication lists where a simple extraction model would fail.

Chain-of-Thought reasoning is an emergent ability primarily observed in sufficiently large language models (typically >100B parameters). Smaller models do not benefit from CoT and may even produce nonsensical reasoning traces that lead to incorrect answers. For clinical deployment, this necessitates the use of massive, compute-intensive models like Med-PaLM 2 or GPT-4, where the scale enables the model to internalize complex clinical pathways and generate coherent, medically sound reasoning chains.

The most powerful form is Few-Shot CoT, where the prompt includes a small number of complete examples demonstrating the desired reasoning format. Each exemplar pairs a complex clinical question with a detailed, step-by-step rationale ending in the correct answer. The design of these exemplars is a form of context engineering; a well-crafted prompt for prior authorization might show the model how to map a patient's history to specific policy criteria through a series of logical deductions.

Reasoning can be elicited without hand-crafted examples by appending a simple trigger phrase like "Let's think step by step" to the end of a prompt. This Zero-Shot CoT technique forces the model to generate a reasoning path before its final answer. While generally less reliable than a well-optimized few-shot prompt, it provides a surprisingly effective baseline for clinical summarization tasks, nudging the model to structure a discharge summary logically before writing it.

A critical limitation is that a model's generated reasoning trace may not faithfully represent its actual decision process. The model can produce a plausible-sounding but factually incorrect rationale that leads to the right answer, or a correct rationale that leads to the wrong answer. In clinical settings, an unfaithful CoT can be dangerous, creating a false sense of security. Mitigation requires coupling CoT with retrieval-augmented generation (RAG) to ground each reasoning step in retrieved medical evidence.

A foundational technique that provides the model with a small number of input-output examples within the prompt context before asking it to perform a new task.

• Mechanism: Leverages in-context learning to condition the model's behavior without updating its weights.
• Clinical Use: Providing 2-3 examples of a correctly formatted radiology report summary before asking the model to summarize a new one.
• Synergy with CoT: Few-shot examples can themselves contain reasoning chains, demonstrating the desired step-by-step logic for complex tasks like differential diagnosis generation.

A simple variant that elicits reasoning without any worked examples by appending a generic trigger phrase like 'Let's think step by step' to the prompt.

• Mechanism: The trigger phrase acts as a heuristic that activates the model's latent reasoning pathways learned during pre-training.
• Clinical Use: Quickly generating a structured clinical assessment plan from a brief patient summary when no annotated examples are available.
• Limitation: Generally less reliable than Few-Shot CoT for highly specialized medical tasks requiring specific diagnostic schemas.

A decoding strategy that samples multiple diverse reasoning paths for the same problem and selects the most consistent final answer via marginalization over the generated rationales.

• Mechanism: Replaces greedy decoding with temperature-based sampling to generate a set of candidate chains, then aggregates the final answers by majority voting.
• Clinical Use: Improving the reliability of automated ICD-10-CM coding by generating multiple coding rationales from a complex discharge summary and selecting the most frequently predicted code set.
• Benefit: Significantly reduces the variance of a single sampled reasoning path, which is critical for safety in clinical decision support.

An extension of CoT that generalizes linear reasoning to exploratory tree search, allowing the model to consider multiple reasoning steps simultaneously and evaluate their promise before committing.

• Mechanism: The model generates several candidate 'thoughts' at each step, self-evaluates their viability using a value prompt, and uses search algorithms like BFS or DFS to explore the most promising paths.
• Clinical Use: Complex treatment planning where the model must explore different medication pathways, evaluate potential drug-drug interactions, and backtrack from contraindicated options.
• Distinction: Unlike CoT's single linear path, ToT enables deliberate lookahead and backtracking, mimicking a clinician's diagnostic reasoning process.

A strategy that decomposes a complex problem into simpler sub-problems and solves them sequentially, feeding the answer of one sub-problem into the prompt for the next.

• Mechanism: First prompts the model to identify the decomposition, then iteratively solves each sub-task, building context cumulatively.
• Clinical Use: Automating a complex prior authorization by first extracting the requested procedure, then the patient's diagnosis history, and finally mapping the clinical evidence to payer-specific medical necessity criteria.
• Advantage: Reduces the cognitive load on the model for tasks with long reasoning horizons that exceed the effective context window of standard CoT.

A prompting paradigm that interleaves verbal reasoning traces with executable actions, enabling the model to interact with external tools like a clinical knowledge base API or a drug interaction checker.

• Mechanism: The model generates a thought, then an action command, observes the result from the environment, and continues reasoning. This loop grounds the reasoning in retrieved facts.
• Clinical Use: A model tasked with medication reconciliation might reason about a discrepancy, query a RxNorm API to resolve a brand name to a generic ingredient, and then update its final medication list based on the retrieved data.
• Synergy: Directly addresses CoT's limitation of hallucinating factual knowledge by dynamically retrieving grounding information mid-reasoning.