DSPy

DSPy abstracts the LLM call into a Signature, a declarative specification of input fields and output fields. Instead of writing a prompt, you define the semantic role of the transformation.

• Input/Output Schema: Define fields like question, context, and answer with optional descriptions.
• Semantic Roles: The compiler infers the task type (e.g., classification, summarization) from the signature.
• Prompt Agnosticism: You never write a raw prompt string; the compiler generates and optimizes it for you.

Example: qa = dspy.ChainOfThought('context, question -> answer') defines a reasoning step without specifying how to reason.

A Teleprompter is a compiler that tunes the parameters of a DSPy program to maximize a metric. It treats prompt instructions and few-shot examples as trainable parameters.

• BootstrapFewShot: Automatically generates and selects effective few-shot examples from a training set.
• BootstrapFinetune: Distills a large program into a smaller, fine-tuned model for production.
• MIPROv2: A Bayesian optimizer that jointly tunes instructions and few-shot examples across all modules.

Mechanism: The teleprompter runs the program on training data, evaluates outputs against a metric, and iteratively updates the program's internal state.

DSPy programs are built by composing reusable Modules, which are parameterized layers that abstract prompting techniques. Modules can be nested and chained like neural network layers.

• dspy.ChainOfThought: Implements step-by-step reasoning with an explicit rationale before the answer.
• dspy.ReAct: An agentic module that interleaves thought, action, and observation steps for tool use.
• dspy.MultiChainComparison: Generates multiple reasoning chains and compares them to synthesize a final answer.
• Custom Modules: Subclass dspy.Module to encapsulate complex multi-step logic with internal state.

The DSPy compiler requires a metric function that scores the quality of a program's output against a ground-truth label. This metric is the objective function for all optimization.

• Semantic Metrics: Use dspy.evaluate.answer_exact_match for exact string matching or dspy.evaluate.SemanticF1 for meaning-level comparison.
• Custom Metrics: Define any Python function that takes (example, pred, trace) and returns a float score.
• Program-Aware: The metric can inspect the full execution trace, not just the final output.

Impact: The metric replaces manual prompt tuning with a principled, reproducible optimization loop.

DSPy decouples program logic from the underlying model provider through a unified LM interface. Swap between local and remote models without changing program code.

• Provider Agnostic: Supports OpenAI, Anthropic, Cohere, local Hugging Face models, and custom endpoints.
• Configuration: Set dspy.settings.configure(lm=model, rm=retriever) globally or per-module.
• Retrieval Models: A parallel RM abstraction integrates vector databases and search APIs as retrievable modules.

Benefit: Benchmark the same compiled program across GPT-4, Claude, and Llama 3 to find the optimal cost-performance ratio.

The core innovation of DSPy is treating prompt engineering as a discrete optimization problem. The compiler searches over the space of possible instructions and demonstrations.

• Instruction Tuning: The optimizer generates and refines natural language instructions for each signature field.
• Few-Shot Selection: It selects the subset of training examples that maximizes validation performance when included as demonstrations.
• Self-Improvement: Programs can be re-compiled as new data arrives, adapting to distribution shifts automatically.

Result: A DSPy program compiled for GPT-3.5 can outperform hand-crafted prompts designed for GPT-4.