Response Shaping

A constrained decoding technique that restricts a model's token-by-token generation to follow a formal grammar, ensuring syntactically valid output. The grammar, often defined in Extended Backus-Naur Form (EBNF), acts as a real-time filter during inference.

• Key Mechanism: The decoder checks each candidate token against the grammar's allowable next tokens.
• Primary Use: Guaranteeing outputs are valid JSON, SQL, or code without relying on the model's latent knowledge of syntax.
• Example: Using the guidance or outlines library to force generation of a valid JSON object matching a specific schema.

A technique for guaranteeing a model's output strictly adheres to a predefined JSON Schema, including data types, required fields, and value constraints. This is often implemented via API parameters (e.g., OpenAI's response_format).

• Key Mechanism: The model is explicitly instructed, often at a system level, to output only JSON that validates against the provided schema.
• Primary Use: Creating reliable data contracts for downstream API consumption, ensuring fields like user_id are integers and email is a string.
• Implementation: In the OpenAI API, setting response_format={ "type": "json_object" } forces JSON output.

A prompt engineering pattern where a pre-formatted text skeleton with placeholders is provided within the prompt, guiding the model to fill in specific information.

• Key Mechanism: The prompt includes the exact output structure with clear delimiters (e.g., {{PLACEHOLDER}}) where content should be inserted.
• Primary Use: Enforcing consistent formatting for lists, reports, or standardized responses without complex decoding logic.
• Example Prompt: "Summarize the article. Use this exact format:\nTitle: {{TITLE}}\nKey Points:\n- {{POINT_1}}\n- {{POINT_2}}\nConclusion: {{CONCLUSION}}"

An advanced form of constrained decoding where the generation process is dynamically guided by a live, in-memory representation of the target output schema.

• Key Mechanism: The decoder maintains state about which part of the schema (e.g., which object property or array element) is currently being generated to inform valid next tokens.
• Primary Use: Handling complex, nested schemas more efficiently than static grammars, improving generation speed and accuracy for deep JSON structures.
• Contrast: Goes beyond simple grammar checking by understanding the semantic context within the schema, such as required vs. optional fields.

A design pattern where instructions and context are organized using non-natural language formatting tags (like XML or markdown code blocks) to implicitly guide structure.

• Key Mechanism: The model learns from the prompt's own structure that responses should mirror a similar formal organization.
• Primary Use: Improving adherence for complex outputs by separating instructions, context, and examples into distinct, labeled sections.
• Example Prompt: <summary_request>\n<article>\n[Article text here]\n</article>\n<instruction>Output in JSON with 'title' and 'sentiment' keys.</instruction>\n</summary_request>

The application of rule-based scripts to clean, parse, validate, and normalize a model's raw text output into a canonical format. This is a safety net for other shaping techniques.

• Key Components:
  • Validation: Checking output against a JSON Schema using a library like jsonschema.
  • Sanitization: Escaping special characters or removing markdown artifacts.
  • Normalization: Converting dates, numbers, or booleans into standard formats (e.g., ISO 8601, Python bool).
• Primary Use: Ensuring robustness in production pipelines, catching and correcting minor formatting errors before data is passed to downstream systems.

Generating syntactically correct artifacts where format is non-negotiable. This goes beyond simple JSON to complex, nested structures.

• Code Generation: Using grammar-based decoding to ensure generated Python, SQL, or YAML code is always syntactically valid and follows style guides.
• Standardized Reporting: Automating the creation of reports in specific XML or JSON formats required by regulatory bodies or internal systems.
• Example: A model generates a Kubernetes manifest; the output is constrained to the exact YAML structure and API version (apiVersion: v1, kind: Pod) required by kubectl.

Enabling deterministic communication between autonomous AI agents. Shaped responses act as the inter-agent protocol, ensuring messages are reliably parsed and acted upon.

• Action-Oriented Outputs: An agent specializing in analysis outputs a shaped result like {"task": "data_analysis_complete", "findings": [...], "next_agent": "report_generator"}.
• Error Handling: Structured error objects ({"error": true, "code": "INSUFFICIENT_DATA"}) allow other agents to programmatically handle failures.
• Foundation: Critical for frameworks implementing the ReAct (Reasoning + Acting) pattern or agentic workflows.

Driving personalized user interfaces by generating structured data for front-end components. This separates content generation from presentation logic.

• Catalog & Recommendation Feeds: A model analyzes user queries and outputs a shaped list of product attributes ([{ "id": "prod_123", "title": "...", "price": 49.99 }]) for immediate rendering in a UI grid.
• Dynamic Forms: Generating the schema for a next-step form based on a conversation, output as JSON Schema for a front-end form builder.
• Benefit: Enables Answer Engine Architecture where the LLM provides the structured data, and a separate system handles the display.

Enabling automated, scalable evaluation of model performance. By forcing model outputs into a consistent grading schema, evaluation becomes a programmatic check.

• Automated Scoring: A model's answer to a question is shaped to always output: {"final_answer": "...", "confidence": 0.8, "step_count": 5}. An evaluator script compares final_answer to a gold standard.
• Consistency in Testing: Ensures every model response in a benchmark test suite has the same fields, enabling apples-to-apples comparison and metric calculation (accuracy, latency).
• Core to Eval-Driven Development: Provides the deterministic output required for unit testing prompts and model versions.