Schema-Constrained Decoding

The core mechanism replaces the model's unconstrained sampling with a finite-state automaton that masks invalid tokens at each step. The system dynamically constructs a mask over the model's logits, setting the probability of any token that would violate the target schema to zero. This is typically implemented using a pushdown automaton for context-free grammars like JSON, ensuring that every generated brace, bracket, and key is properly nested and closed. The result is a mathematically guaranteed structural output, eliminating the possibility of malformed JSON, invalid XML, or broken API calls.

A critical distinction exists between basic JSON mode and true schema-constrained decoding. JSON mode simply instructs the model to output valid JSON, but it remains a statistical process prone to structural errors. Schema-constrained decoding forces the output to conform to a specific, user-provided JSON Schema definition.

• JSON Mode: Guarantees valid JSON syntax, but not the presence of required fields or correct data types
• Schema-Constrained: Guarantees exact compliance with a schema, including required keys, enum values, and nested object structures
• Implementation: Libraries like outlines, guidance, and lm-format-enforcer build the constraining grammar from the schema itself

A foundational technique within schema-constrained decoding is the use of regular expressions to define valid token sequences. The regex is compiled into a deterministic finite automaton (DFA) that tracks the valid state at each generation step. This is particularly powerful for structured data extraction tasks:

• Date formats: Enforcing \d{4}-\d{2}-\d{2} prevents hallucinated dates like "2024-13-40"
• Enum fields: Restricting output to a predefined set like (approved|denied|pending)
• Numeric ranges: Constraining integers to a specific min/max boundary
• Identifiers: Enforcing patterns like [A-Z]{3}-\d{5} for legal docket numbers

In legal AI, schema-constrained decoding serves as a critical guardrail for machine-to-machine communication between services. When a language model extracts structured data from a contract, the output must be a valid object for downstream processing.

• Contract Clause Extraction: The model is forced to output a {clause_type, text, parties, obligations} object that strictly conforms to the integration schema
• Citation Parsing: Legal citations are decoded into a structured {volume, reporter, page, court, year} format, preventing malformed references
• Docket Entry Classification: Outputs are constrained to a controlled vocabulary of motion types, eliminating the risk of a hallucinated category breaking a filing system

Schema-constrained decoding introduces computational overhead from the grammar compilation and mask construction steps. The finite-state automaton must be advanced with each generated token, and the valid token mask must be applied to the model's full vocabulary logits before sampling.

• Compilation: The schema or regex is compiled once into an efficient index, typically adding negligible latency (< 1ms)
• Per-Token Masking: Applying the mask requires a vocabulary-wide operation, but optimized implementations use bitwise operations on pre-computed token-to-state maps
• Throughput Impact: Benchmarks show a 5-15% reduction in tokens-per-second compared to unconstrained generation, a trade-off accepted for guaranteed structural validity

For complex nested structures like legal document formats, a context-free grammar (CFG) provides the formal specification. The CFG defines production rules that the decoder must follow, enabling the generation of arbitrarily nested structures with guaranteed balance.

• Production Rules: Define how non-terminal symbols expand into terminals and other non-terminals
• Earley Parser Integration: An incremental parser tracks the valid next tokens based on the partial generation, building the constraining mask
• Legal Document Example: A CFG can enforce the hierarchical structure of a legal brief, ensuring sections like Statement of Facts and Argument appear in the correct order with proper sub-headings

A prompting technique where a language model drafts an initial response, then systematically generates a series of fact-checking questions about its own output. The model independently answers these verification questions and revises the original response to correct any identified inconsistencies.

• Process: (1) Draft baseline response, (2) Plan verification questions, (3) Execute verifications independently, (4) Generate final revised output.
• Legal Use Case: Validates that a synthesized legal argument does not misattribute a holding to the wrong court or conflate distinct statutory provisions.
• Efficacy: Reduces hallucination rates by forcing explicit self-critique without requiring an external verifier model.

A quantitative evaluation framework that measures the factual consistency of a generated summary or answer relative to the provided source material. It identifies contradictions and unsupported fabrications by decomposing the output into atomic claims and checking each against the source.

• Methodology: Often uses Natural Language Inference (NLI) entailment models to classify each atomic claim as entailed, contradicted, or neutral.
• Legal Relevance: A critical KPI for legal AI systems, where a single fabricated precedent can undermine the entire analysis.
• Common Implementations: Includes metrics like FactCC, DAE, and SummaC, which provide fine-grained hallucination scores.

A pair of complementary metrics specifically designed for evaluating the citation integrity of legal AI outputs.

• Citation Recall: The proportion of factual claims in a generated text that are correctly supported by a citation. High recall means the model provides authority for its assertions.
• Citation Precision: The proportion of provided citations that genuinely support the associated claim. High precision means the model is not fabricating or misattributing references.
• Combined Signal: A system with high recall but low precision is citing prolifically but incorrectly; high precision but low recall is making unsupported assertions. Both must be optimized for legal trustworthiness.

The automated process of verifying that every factual claim in a generated text is explicitly supported by the provided source document. It serves as a critical guardrail against hallucination in legal AI.

• Implementation: Typically uses a fine-tuned NLI model to check if a generated statement is entailed by the source text.
• Operational Role: Acts as a post-generation filter, flagging or blocking outputs that contain unsupported claims before they reach the user.
• Contrast with RAG: While RAG provides the context, groundedness detection verifies that the model actually adhered to it, catching instances where the model ignored or contradicted the retrieved evidence.

A set of statistical techniques that enable a model to estimate the confidence of its own predictions. This allows a legal AI system to flag high-risk outputs for human review or trigger an abstention mechanism.

• Methods: Includes conformal prediction (generating prediction sets with formal coverage guarantees) and calibration error measurement (aligning confidence scores with empirical accuracy).
• Legal Application: A contract clause classifier might output 'Limitation of Liability' with 99.9% confidence but 'Indemnification' with 60% confidence, routing the latter to a human reviewer.
• Strategic Value: Shifts the system from a binary correct/incorrect paradigm to a risk-aware framework, essential for high-stakes legal workflows.