Inferensys

Prompt

ADR Validation Checklist Prompt

A practical prompt playbook for using the ADR Validation Checklist Prompt to enforce quality standards, rationale depth, and traceability in architecture decision records before merge.
QA engineer performing AI quality assurance on laptop, test results visible, casual technical debugging session.
PROMPT PLAYBOOK

When to Use This Prompt

Identify the right conditions for automating ADR quality checks before human review.

Engineering teams that maintain an Architecture Decision Record (ADR) repository need consistent quality before a decision is accepted. Manual review catches some issues but is slow, inconsistent, and hard to enforce at scale. This prompt produces a structured, pass/fail validation checklist with specific criteria for each ADR section, rationale depth, and traceability. It is designed to run as a CI/CD gate on pull requests that add or modify ADRs, giving authors immediate, actionable feedback and giving reviewers a pre-validated document.

Use this prompt when you have a draft ADR and need an automated quality assessment before human approval. The ideal user is an engineering lead, architect, or platform engineer who owns the ADR quality bar and wants to shift validation left into the PR workflow. The prompt requires a complete ADR text as input—it evaluates what exists rather than generating missing sections. It works best with ADRs that follow a standard template (Title, Status, Context, Decision, Options Considered, Consequences) so the validation criteria can map to known sections. The output is a structured checklist with pass/fail status per criterion, specific citations to the ADR text that triggered failures, and an overall compliance score that can be used as a merge gate threshold.

Do not use this prompt for generating ADRs from scratch, for comparing architectural options, or for extracting context from meeting notes. It is not a drafting assistant—it assumes the ADR text is complete enough to evaluate and will not fill in missing sections. Avoid using it on ADRs that intentionally deviate from your standard template, as the validation criteria will produce false negatives. For teams just starting their ADR practice, pair this prompt with the ADR Generation Prompt Template to produce a compliant first draft, then run validation as a quality check before the review cycle begins.

PRACTICAL GUARDRAILS

Use Case Fit

Where the ADR Validation Checklist Prompt works well and where it introduces risk. Use these cards to decide if this prompt fits your workflow before integrating it into a CI/CD gate.

01

Good Fit: Structured ADR Templates

Use when: your team already uses a standard ADR template with predictable sections (Context, Decision, Options, Consequences). The prompt performs best when it can map checklist criteria to known structural elements. Guardrail: define a canonical template first; the prompt should validate against it, not infer structure from freeform text.

02

Bad Fit: Freeform Decision Docs

Avoid when: architectural decisions are captured in unstructured meeting notes, chat threads, or narrative design docs without consistent headings. The prompt will hallucinate section boundaries and produce unreliable pass/fail scores. Guardrail: run a Context Extraction Prompt first to normalize the input before validation.

03

Required Inputs

Risk: incomplete or missing ADR sections cause false negatives or skipped criteria. Guardrail: the harness must verify that required fields (Context, Decision, Options, Consequences) are present before invoking the checklist prompt. Return an explicit INPUT_INCOMPLETE status rather than attempting partial validation.

04

Operational Risk: Gate Automation

Risk: wiring the prompt directly into a merge-blocking CI/CD gate without human override can stall urgent hotfix ADRs or intentionally lightweight decisions. Guardrail: implement a bypass mechanism with audit logging. Require human approval for any ADR that fails the checklist but must proceed.

05

Score Drift Over Time

Risk: as your team's ADR standards evolve, the prompt's checklist criteria become stale and either too strict or too lenient. Guardrail: version the checklist criteria alongside your ADR template. Run regression tests comparing prompt scores against human reviewer scores quarterly to detect drift.

06

Rationale Depth False Positives

Risk: the prompt may mark rationale as sufficient when it contains fluent but hollow text—long paragraphs that restate the decision without explaining trade-offs. Guardrail: add a specific criterion requiring at least one explicit trade-off statement per option. Use a secondary LLM-as-judge check on rationale substance for high-risk ADRs.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A ready-to-use prompt that validates an ADR against quality criteria and returns a structured pass/fail checklist.

This prompt template is the core of the ADR validation workflow. It takes a complete ADR document and a set of quality criteria, then produces a structured checklist with pass/fail status, specific evidence from the ADR text, and actionable remediation guidance for each failing criterion. The prompt is designed to be strict and evidence-based—it should not pass a criterion unless the ADR explicitly contains the required content.

text
You are an ADR quality reviewer. Your task is to validate an architecture decision record against a defined quality standard and produce a structured pass/fail checklist.

## INPUT

### ADR Document
[ADR_CONTENT]

### Quality Criteria
[QUALITY_CRITERIA]

### Output Schema
Return a JSON object with this exact structure:
{
  "adr_title": "string",
  "overall_score": number (0-100),
  "overall_pass": boolean,
  "criteria": [
    {
      "criterion_id": "string",
      "criterion_name": "string",
      "status": "pass" | "fail" | "not_applicable",
      "evidence": "string (quote from ADR or explain absence)",
      "remediation": "string (specific fix if failed, else empty)"
    }
  ],
  "summary": "string (2-3 sentence overall assessment)"
}

## CONSTRAINTS
[CONSTRAINTS]

## INSTRUCTIONS
1. Read the entire ADR document carefully.
2. For each quality criterion, determine if the ADR satisfies it.
3. A criterion passes ONLY if the ADR contains explicit, specific content that addresses it. Do not infer, assume, or fill gaps.
4. For each failing criterion, quote the specific section or absence that caused the failure.
5. For each failing criterion, provide a concrete, actionable remediation that tells the author exactly what to add or fix.
6. Calculate overall_score as the percentage of applicable criteria that pass.
7. Set overall_pass to true only if overall_score meets the minimum threshold specified in constraints.
8. If a criterion is genuinely not applicable to this ADR type, mark it as not_applicable and explain why briefly.

To adapt this template for your team, replace the square-bracket placeholders with your actual content. [ADR_CONTENT] should contain the full ADR text, including all sections like context, decision, options considered, consequences, and metadata. [QUALITY_CRITERIA] should be a list of specific, testable criteria—for example, "ADR includes at least two alternative options with trade-offs" or "Decision rationale references specific constraints from the context section." [CONSTRAINTS] should define the minimum pass threshold, any domain-specific requirements, and whether certain sections are mandatory. After running the prompt, always validate the output JSON against the schema before accepting results. For high-stakes ADRs, route failing checklists to a human reviewer rather than auto-blocking a merge.

IMPLEMENTATION TABLE

Prompt Variables

Inputs the ADR Validation Checklist Prompt needs to work reliably. Validate each before running the prompt to prevent false positives or missed quality issues.

PlaceholderPurposeExampleValidation Notes

[ADR_TEXT]

The full body of the Architecture Decision Record to validate

Context

We need to choose a message broker...

Decision

We will use Kafka...

Must be non-empty string. Check minimum length of 200 characters to ensure substantive content. Reject if only contains headers without body text.

[QUALITY_STANDARD]

Defines the quality bar for pass/fail criteria

strict

Must be one of: 'strict', 'standard', 'minimum'. 'strict' requires all sections present and rationale depth >= 3 sentences. 'standard' allows minor omissions. 'minimum' only checks for decision and context sections.

[REQUIRED_SECTIONS]

List of ADR sections that must be present for a pass

['Context', 'Decision', 'Options Considered', 'Trade-offs', 'Consequences', 'Status']

Must be a valid JSON array of strings. Each string must match a known ADR section name. Empty array is allowed only if [QUALITY_STANDARD] is 'minimum'. Validate array is parseable before prompt execution.

[RUBRIC_DIMENSIONS]

Specific quality dimensions to score individually

['Clarity', 'Completeness', 'Rationale Depth', 'Traceability', 'Option Coverage']

Must be a valid JSON array of strings. Each dimension must be from the approved list: Clarity, Completeness, Rationale Depth, Traceability, Option Coverage, Assumption Explicitness, Risk Documentation. Reject unknown dimensions.

[PASS_THRESHOLD]

Minimum score or percentage of checks that must pass

0.8

Must be a float between 0.0 and 1.0. Represents the proportion of checks that must pass. For [QUALITY_STANDARD]='strict', recommend >= 0.9. Validate numeric type before prompt execution.

[OUTPUT_FORMAT]

Desired output structure for the validation results

json

Must be one of: 'json', 'markdown', 'ci_json'. 'ci_json' returns a compact format with only pass/fail and failed checks for CI/CD integration. Validate against allowed enum values.

[FAILURE_MODE]

Controls whether the prompt returns all checks or stops at first failure

complete

Must be one of: 'complete', 'fail_fast'. 'fail_fast' returns immediately on first failed check. 'complete' evaluates all checks regardless. Use 'fail_fast' for CI/CD gates to reduce latency.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the ADR validation prompt into a CI/CD pipeline, review workflow, or automated quality gate.

The ADR Validation Checklist Prompt is designed to be a deterministic quality gate, not an advisory chatbot. To use it in production, wrap the prompt in a thin application harness that controls the input assembly, model call, output parsing, and post-validation logic. The harness should treat the model's output as a structured compliance report—never as a final authority without programmatic checks. The core integration points are: a pre-commit or PR-check hook that triggers validation when ADR files change, a model invocation layer that injects the ADR content and the checklist criteria, and a post-processing step that parses the pass/fail result and attaches it as a review comment or CI status check.

Start by defining the trigger. In a GitHub Actions or GitLab CI pipeline, configure the workflow to run on pull requests that modify files in your ADR directory (e.g., docs/adr/**). The harness should extract the changed ADR file content, inject it into the [ADR_CONTENT] placeholder, and supply the [CHECKLIST_CRITERIA] from a version-controlled criteria file in the repository. This keeps the quality standard auditable and updatable without changing the prompt. For the model call, prefer a model with strong structured-output capabilities and low latency—GPT-4o or Claude 3.5 Sonnet are good defaults. Set temperature=0 and request a JSON output mode to enforce the checklist schema. Implement a retry wrapper with up to 2 retries on parse failures, using the validation error message as context for the retry. If the output still fails to parse after retries, fail the check open with a warning rather than blocking the pipeline silently.

Post-processing is where the harness earns its keep. Parse the model's JSON output and validate it against an expected schema: a top-level pass boolean, a score integer, and a checks array where each item has an id, criterion, status (pass/fail/warn), and detail string. If the pass field is false, the harness should post a structured comment on the PR with the failing checks and their details. For teams using GitHub Checks API or GitLab Merge Request notes, map the checklist results to annotations on the specific ADR file. Store the full validation result as a CI artifact for auditability. For high-risk architectural decisions (e.g., security boundaries, data storage choices, compliance-affecting changes), add a human approval gate: if the automated check fails or scores below a configurable threshold, require a designated architect's review before merge. This prevents the automation from becoming a rubber stamp while still catching the majority of incomplete or weak ADRs before human reviewers spend time on them.

Avoid wiring this prompt into a blocking gate without an override path. ADR quality is a practice, not a binary property, and edge cases exist—emergency decisions, intentionally lightweight records, or decisions where full trade-off analysis is deferred. The harness should support a skip-validation label on the PR or an adr-validation: bypass footer in the commit message that allows authorized users to override the gate while logging the bypass event. Monitor the bypass rate and the pass/fail distribution over time. If the bypass rate climbs above 10%, the criteria are probably too strict or the prompt is flagging false positives. If the pass rate is near 100%, the criteria are probably too lenient to catch real quality gaps. Treat the harness as a feedback loop: the prompt, the criteria, and the thresholds should all be tuned based on what the team actually accepts or rejects in review.

IMPLEMENTATION TABLE

Expected Output Contract

Define the exact structure, types, and validation rules for the ADR validation checklist response. Use this contract to build a parser that gates CI/CD pipelines on valid, machine-readable output.

Field or ElementType or FormatRequiredValidation Rule

overall_score

number (0.0-100.0)

Must be a float between 0 and 100 inclusive. Parse check: JSON number type, range enforcement.

overall_pass

boolean

Must be true if overall_score >= [PASS_THRESHOLD], else false. Schema check: strict boolean type.

sections

array of objects

Must contain exactly one object per required ADR section: Context, Decision, Options, Consequences, Status. Schema check: array length and key matching.

sections[].name

string

Must exactly match one of the canonical section names. Enum check: ['Context', 'Decision', 'Options Considered', 'Consequences', 'Status'].

sections[].score

number (0.0-100.0)

Must be a float between 0 and 100 inclusive. Parse check: JSON number type, range enforcement.

sections[].pass

boolean

Must be true if section score >= [SECTION_THRESHOLD], else false. Schema check: strict boolean type.

sections[].issues

array of strings

Must contain 0 or more non-empty strings describing specific failures. Null check: array must exist, can be empty.

sections[].rationale

string

Must be a non-empty string summarizing the scoring justification. Null check: string length > 0.

PRACTICAL GUARDRAILS

Common Failure Modes

ADR validation prompts fail in predictable ways. These are the most common failure modes observed in production CI/CD gates and review workflows, along with concrete mitigations.

01

False Pass on Shallow Rationale

What to watch: The prompt accepts vague rationale like 'we chose X because it's better' without detecting missing trade-off analysis. The checklist marks sections complete when they contain filler text rather than substantive reasoning. Guardrail: Include explicit depth criteria in the prompt schema—require at least one rejected alternative with a specific disadvantage, and use a secondary LLM-as-judge pass to score rationale density before final pass/fail.

02

Context Window Truncation Silently Drops Sections

What to watch: Long ADRs with extensive option catalogs or appendices exceed the model's context window mid-evaluation. The model returns a passing score but has only evaluated the first half of the document. Guardrail: Pre-process the ADR to count tokens and split evaluation into section-level checks with independent context windows. Add a coverage assertion that every required section header appears in the evaluation output.

03

Schema Drift Between Prompt and CI/CD Harness

What to watch: The prompt expects fields like rationale_depth but the harness parses rationale_score. The model returns valid JSON that the pipeline silently ignores, producing false passes. Guardrail: Version the output schema alongside the prompt template. Add a pre-merge schema validation step that rejects outputs with unrecognized or missing fields before scoring is recorded.

04

Over-Penalization of Concise ADRs

What to watch: The checklist prompt equates length with quality, flagging short but complete ADRs as missing detail while passing verbose documents that restate the same point three ways. Guardrail: Calibrate scoring rubrics against a golden set of ADRs that includes both concise and expanded examples. Include explicit instruction that brevity with completeness is not a defect.

05

Hallucinated Compliance Mappings

What to watch: When the prompt includes regulatory or compliance criteria, the model invents mappings between ADR sections and standards it doesn't actually reference. The checklist claims traceability that doesn't exist in the source document. Guardrail: Require the model to quote the exact sentence or paragraph from the ADR that satisfies each compliance criterion. If no quote exists, the criterion must fail. Never accept inferred compliance.

06

Checklist Blindness to Missing Options

What to watch: The prompt validates the structure of documented options but never asks whether obvious alternatives were omitted. An ADR that considers only two straw-man options passes validation while missing the industry-standard approach. Guardrail: Add a pre-check that enumerates expected option categories for the decision type and flags ADRs that don't address at least one from each category. Maintain a living catalog of common alternatives per architectural concern.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the ADR Validation Checklist prompt before integrating it into a CI/CD gate. Each criterion maps to a specific quality dimension that the prompt output must satisfy. Run these checks against a golden set of known-good and known-bad ADRs to calibrate pass/fail thresholds.

CriterionPass StandardFailure SignalTest Method

Section Completeness

Checklist identifies all required ADR sections (Title, Status, Context, Decision, Options, Consequences) as present or missing

Output marks a section as present when it is empty or contains only placeholder text

Run against 5 ADRs with known missing sections; verify 100% detection rate

Rationale Depth

Checklist flags decisions with fewer than 2 distinct trade-off dimensions or missing justification for the chosen option

Output passes an ADR that states a decision without explaining why alternatives were rejected

Test with 3 shallow-rationale ADRs and 3 well-reasoned ADRs; confirm correct classification

Traceability

Checklist verifies that each ADR references at least one source (issue, incident, requirement, or stakeholder input)

Output passes an ADR with no links to external context or prior decisions

Feed ADRs with 0, 1, and 3+ references; confirm traceability flag triggers at zero

Option Enumeration

Checklist confirms at least 2 alternatives were considered, including the status-quo option where applicable

Output passes an ADR that presents only the chosen approach with no alternatives documented

Test with single-option ADRs; verify checklist marks them as non-compliant

Consequence Coverage

Checklist validates that positive, negative, and neutral consequences are addressed for the chosen decision

Output passes an ADR that lists only benefits and omits known downsides or migration costs

Use ADRs with one-sided consequence sections; confirm failure flag

Assumption Explicitness

Checklist detects unstated assumptions by flagging decisions that lack an explicit assumptions section or contain vague claims

Output passes an ADR with implicit assumptions like 'the team is familiar with this technology' stated as fact

Run against ADRs with hidden assumptions; verify checklist surfaces the gap

Output Schema Validity

Checklist output is valid JSON matching the expected schema with all required fields present and correctly typed

Output is malformed JSON, missing required fields, or contains extra fields that break downstream parsing

Validate output against JSON Schema in CI; reject any response that fails structural validation

False Positive Rate

Checklist does not flag compliant ADRs as failing when all quality criteria are met

Output marks a fully compliant ADR as non-compliant due to overly strict or misapplied rules

Run against 10 known-compliant ADRs; require false positive rate below 10% before shipping

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single ADR as [INPUT]. Remove the scoring rubric and CI/CD harness references. Focus on the checklist items only. Accept plain-text output without strict JSON schema enforcement.

Watch for

  • The model skipping sections it considers 'obvious'
  • Overly generous pass ratings on weak rationale
  • No distinction between missing sections and empty sections
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.