Inferensys

Prompt

Structured Review Feedback Generation Prompt

A practical prompt playbook for using Structured Review Feedback Generation Prompt in production AI workflows.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job-to-be-done, ideal user, and boundaries for the Structured Review Feedback Generation Prompt.

This prompt is designed for engineering teams integrating automated code review into a CI/CD pipeline. Its job is to ingest a pull request diff and produce a machine-readable JSON payload of review findings. Each finding maps to a specific line, carries a severity level and category, and includes a suggested fix. The output is built for direct ingestion by code review tools, ticketing systems, or dashboards. Use this prompt when you need consistent, structured, and actionable feedback that can be processed programmatically.

The ideal user is a platform engineer or developer setting up an automated review step that runs alongside linters and tests. The prompt requires a raw unified diff as input, along with a strict JSON output schema. It works best for statically analyzable concerns—code style violations, potential null dereferences, resource leaks, and clear anti-patterns. It is not a replacement for human review on critical paths, security-sensitive changes, or architectural decisions. For those, route the output to a human approval queue rather than auto-applying findings.

Do not use this prompt for natural-language code summaries, conversational feedback, or subjective design critique. It is optimized for deterministic, schema-conformant output that downstream systems can parse without ambiguity. Before deploying, validate the JSON output against your schema in a staging pipeline, and run a regression suite of known diffs to measure finding accuracy and false positive rates. If the prompt misses a finding class you care about, add a [CONSTRAINTS] block with explicit rules and counterexamples before considering fine-tuning.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Structured Review Feedback Generation Prompt works well and where it introduces risk. Use these cards to decide whether this prompt fits your workflow before wiring it into a CI/CD pipeline.

01

Good Fit: Automated CI/CD Review

Use when: you need a machine-readable review payload (JSON) for direct ingestion by code review tools, GitHub Checks, or GitLab merge request widgets. Guardrail: validate the output schema before posting results; a single malformed JSON object should block the check run.

02

Bad Fit: Sole Approver for Critical Paths

Avoid when: the PR touches authentication, authorization, cryptographic material, or financial logic. Risk: the model may miss context-dependent security flaws or produce confident but incorrect severity ratings. Guardrail: require human approval for any finding above severity: critical before merge.

03

Required Inputs

Requires: a unified diff, optional full-file context, and a defined output schema (finding categories, severity scale, fix suggestion format). Risk: without full-file context, the model hallucinates line numbers or misidentifies the scope of a change. Guardrail: always provide at least 50 lines of surrounding context per hunk.

04

Operational Risk: Schema Drift

What to watch: the model gradually changes field names, enum values, or nesting structure across runs, breaking downstream ingestion. Guardrail: run a JSON Schema validator in the harness and reject any payload that fails validation. Log schema violations to detect drift patterns.

05

Operational Risk: Severity Inflation

What to watch: the model assigns severity: critical to style nits or low-impact findings, desensitizing the team to real issues. Guardrail: include severity definitions with concrete examples in the prompt. Post-process findings to flag PRs where critical findings exceed 10% of total findings for human review.

06

Scale Limit: Large Diffs

What to watch: diffs exceeding the model's context window cause truncated reviews, missed files, or dropped findings. Guardrail: split diffs over 50k tokens into per-file or per-module chunks. Run the prompt on each chunk independently and merge results with deduplication logic.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt that generates a machine-readable JSON review payload from a pull request diff, ready for integration into CI/CD pipelines.

This prompt template is designed to be pasted directly into your AI harness. It instructs the model to act as a structured code reviewer, consuming a raw diff and producing a strict JSON payload that maps findings to specific lines, severity levels, and actionable fix suggestions. The output is intended for direct ingestion by code review tools, dashboards, or automated comment posters, not for free-text human reading. Before using this template, ensure you have a reliable method for extracting the full file diff and any relevant context from your repository, as the model cannot access your codebase directly.

code
SYSTEM:
You are a strict code review agent. Your only output is a valid JSON object conforming to the [OUTPUT_SCHEMA] below. Do not include any text outside the JSON object. Do not wrap the JSON in markdown fences.

USER:
Analyze the following pull request diff for bugs, logic errors, security vulnerabilities, and code quality issues.

[CONTEXT]
- Repository: [REPO_NAME]
- Base Branch: [BASE_BRANCH]
- Target Branch: [TARGET_BRANCH]
- Review Focus: [REVIEW_FOCUS]

[INPUT]
```diff
[RAW_DIFF]

[OUTPUT_SCHEMA] { "summary": "string (a concise, one-sentence summary of the change)", "findings": [ { "id": "string (unique finding identifier, e.g., FIND-001)", "file": "string (full file path)", "line_range": ["number (start line in the new file)", "number (end line in the new file)"], "category": "string (enum: 'bug', 'security', 'performance', 'maintainability', 'style')", "severity": "string (enum: 'critical', 'high', 'medium', 'low', 'info')", "title": "string (a short, descriptive title for the finding)", "description": "string (a clear explanation of the issue and why it is a problem)", "suggestion": "string (a concrete, code-level suggestion for fixing the issue)", "confidence": "number (a score from 0.0 to 1.0 indicating how certain the model is about this finding)" } ], "risk_assessment": { "overall_risk": "string (enum: 'low', 'medium', 'high', 'critical')", "breaking_change_detected": "boolean", "test_coverage_concern": "boolean" } }

[CONSTRAINTS]

  • Only report findings present in the [RAW_DIFF]. Do not hallucinate issues.
  • Map findings to the correct line numbers in the new version of the file.
  • If no issues are found, return an empty findings array and a risk_assessment with overall_risk set to low.
  • For severity, use critical only for security vulnerabilities or data loss bugs. Use high for functional bugs. Use medium for maintainability issues. Use low for style nits.
  • The suggestion field must be a specific code change, not a vague recommendation.

To adapt this template, replace the square-bracket placeholders with data from your CI/CD environment. [RAW_DIFF] should be the complete unified diff of the pull request. [REVIEW_FOCUS] can be used to narrow the model's attention, for example, to 'security and authentication logic' or 'database migration safety'. The [OUTPUT_SCHEMA] is the contract your application must validate against. After receiving the model's response, always run a JSON schema validator before processing the findings. If validation fails, trigger a retry with a repair prompt that includes the raw output and the validation error message. For critical or high severity findings, route the payload to a human reviewer queue instead of automatically posting comments.

IMPLEMENTATION TABLE

Prompt Variables

Inputs required by the Structured Review Feedback Generation Prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to verify the input is well-formed before execution.

PlaceholderPurposeExampleValidation Notes

[DIFF_CONTENT]

The raw unified diff or git diff output to review. This is the primary subject of the review.

diff --git a/src/auth.py b/src/auth.py @@ -45,7 +45,9 @@ def login(user):

  • token = generate_token(user)
  • token = generate_token(user, expiry=3600)
  • log_access(user.id)

Must be non-empty. Validate that the string contains at least one diff header line (diff --git or Index:). Reject if the input is a plain code snippet without diff markers. Max 200KB to avoid context overflow.

[REVIEW_CATEGORIES]

A list of review categories the model should check for. Controls which types of findings are generated.

["security", "performance", "error_handling", "null_safety"]

Must be a valid JSON array of strings. Allowed values: security, performance, error_handling, null_safety, concurrency, api_contract, code_style, test_coverage, breaking_change, resource_management. Empty array defaults to all categories. Reject unknown values.

[SEVERITY_LEVELS]

The severity taxonomy to use when classifying findings. Defines the labels and their priority order.

["critical", "high", "medium", "low", "info"]

Must be a valid JSON array with at least 2 unique string values. Order implies priority (first is highest). Validate that no duplicate labels exist. Default if omitted: ["high", "medium", "low"].

[FILE_FILTER]

Optional glob or regex pattern to restrict review to specific files in the diff. Limits scope for large PRs.

"src/**/*.py"

If provided, must be a valid glob pattern string or null. Validate by testing against at least one filename in the diff. If the pattern matches zero files, emit a warning but proceed. Null means review all files.

[MAX_FINDINGS]

Upper limit on the number of findings returned. Prevents overwhelming output on large diffs.

15

Must be a positive integer between 1 and 50. Defaults to 20 if omitted. Validate type and range before prompt assembly. Values above 50 risk truncation in downstream tooling.

[CONTEXT_SNIPPET_LENGTH]

Number of surrounding lines to include with each finding for code reference. Controls output verbosity.

5

Must be a positive integer between 0 and 20. Defaults to 3. A value of 0 omits code snippets and returns only line references. Validate that the value does not exceed the available context window budget.

[OUTPUT_SCHEMA_VERSION]

Schema version identifier for the JSON output format. Enables forward compatibility as the schema evolves.

"v2"

Must be a string matching the pattern v[0-9]+. Currently supported: v1, v2. Reject unknown versions. The prompt template should include the corresponding schema definition for the requested version.

[REPOSITORY_LANGUAGE]

Primary programming language of the codebase. Helps the model apply language-specific rules and idioms.

"python"

Must be a non-empty string matching a known language identifier (python, javascript, typescript, java, go, rust, ruby, csharp, php, kotlin, swift). Used to select language-specific validation patterns. Null allowed if language is unknown.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the structured review feedback prompt into a CI/CD pipeline or code review tool.

The structured review feedback prompt is designed to be a deterministic component in an automated review pipeline, not a conversational assistant. Its primary contract is to accept a raw diff and a strict JSON schema, and to return a machine-readable payload that can be directly ingested by code review platforms like GitHub, GitLab, or Gerrit. The implementation harness must treat the LLM call as a function with a well-defined input and output, wrapping it in validation, retry logic, and human approval gates before any finding is posted as a review comment.

To wire this into an application, start by constructing the [INPUT] from the actual git diff output, truncated to fit the model's context window. The [OUTPUT_SCHEMA] placeholder should be replaced with a complete JSON Schema definition that includes required fields (id, file, line, severity, category, finding, suggestion), enums for severity (critical, high, medium, low, info) and category (bug, security, performance, maintainability, style, documentation), and a maxItems constraint on the top-level array. After receiving the model's response, the harness must validate it against this schema. If validation fails, implement a single retry by feeding the raw response and the validation error back into the prompt's [CONSTRAINTS] section, instructing the model to repair the output. If the retry also fails, log the failure and alert the on-call channel; never post invalid data to the review tool.

For high-risk repositories, the harness must enforce a human approval step. All findings with a severity of critical or high should be routed to a review queue where a senior engineer can approve, edit, or dismiss each finding before it is posted. This can be implemented as a blocking step in the CI pipeline or as a draft review that requires manual publish. Additionally, implement observability by logging every prompt request, the validated response, the retry count, and the final action (posted, queued for approval, or failed). This trace data is essential for debugging prompt drift, model misbehavior, and schema evolution over time. Avoid using this prompt on diffs larger than the model's effective context window without first chunking the diff by file and running the prompt per file, then merging the results.

IMPLEMENTATION TABLE

Expected Output Contract

The structured review feedback prompt must produce a JSON payload conforming to this contract. Use these fields, types, and validation rules to build your post-processing harness, schema validator, and CI/CD integration checks before accepting the model output.

Field or ElementType or FormatRequiredValidation Rule

review_id

string (UUID v4)

Must parse as valid UUID v4. Reject on format mismatch.

findings

array of objects

Must be a non-null array. Reject if missing or null. Allow empty array only if diff is empty.

findings[].id

string (kebab-case)

Must match pattern ^finding-[a-z0-9-]+$. Must be unique within the array. Reject on duplicate or invalid format.

findings[].severity

enum: critical, high, medium, low, info

Must be exactly one of the allowed enum values. Reject on any other value or case variation.

findings[].category

enum: bug, security, performance, style, maintainability, documentation, testing, dependency

Must be exactly one of the allowed enum values. Reject on unknown category.

findings[].file_path

string (relative path)

Must be a non-empty string. Must match a file path present in the input diff. Reject if path not found in diff context.

findings[].line_range

object with start and end integers

start and end must be positive integers with start <= end. Must fall within the line range of the referenced file in the diff. Reject on out-of-bounds or inverted range.

findings[].title

string (max 120 chars)

Must be non-empty and <= 120 characters. Reject on empty or over-length string.

PRACTICAL GUARDRAILS

Common Failure Modes

When generating structured review feedback from PR diffs, these failure modes surface most often in production. Each card identifies a specific risk and a concrete guardrail to prevent it before it reaches a developer's inbox.

01

Hallucinated Line References

What to watch: The model invents line numbers, function names, or code snippets that do not exist in the actual diff. This destroys trust and wastes developer time on ghost hunts. Guardrail: Post-process the output with a strict validator that checks every line or location field against the original diff. Discard or flag any finding that cannot be mapped to a real hunk.

02

Severity Inflation

What to watch: The model classifies every style nit as critical or blocker, causing alert fatigue and causing developers to ignore real issues. Guardrail: Add explicit severity definitions with examples in the prompt (e.g., critical = data loss or security breach). Implement a post-processing cap: if >30% of findings are critical, route the entire batch for human triage.

03

Context Window Truncation

What to watch: Large diffs exceed the context window, causing the model to silently ignore the tail end of the change set. Missing files mean missing review coverage. Guardrail: Pre-process the diff to count tokens. If it exceeds a safe threshold (e.g., 70% of the model's limit), chunk the diff by file or module and run the prompt independently on each chunk, then merge the results.

04

Schema Drift in Output

What to watch: The model returns valid-looking JSON that violates the expected schema—missing required fields, wrong enum values, or extra keys that break downstream parsers. Guardrail: Never trust the raw output. Run it through a JSON Schema validator immediately. On failure, feed the validation error back to the model in a retry loop (max 2 attempts) with the original prompt and the specific error message.

05

Over-Reliance on Style Linters

What to watch: The model duplicates the work of deterministic tools (ESLint, RuboCop, Black) by flagging formatting and trivial style issues, wasting tokens and review attention. Guardrail: Explicitly instruct the prompt to ignore issues already covered by the project's automated linter configuration. Pre-filter the diff to exclude whitespace-only changes before sending it to the model.

06

Missing Business Logic Context

What to watch: The model flags a valid pattern as a bug because it lacks domain knowledge (e.g., a deliberate deviation from a standard library to meet a compliance requirement). Guardrail: Include a [PROJECT_CONTEXT] variable in the prompt that accepts architecture decision records (ADRs) or coding standards. If the model's confidence score for a finding is below a threshold, append a disclaimer: "Verify against project-specific requirements."

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the quality and reliability of the structured review feedback payload before integrating it into a CI/CD pipeline. Each criterion targets a specific failure mode common in machine-readable code review generation.

CriterionPass StandardFailure SignalTest Method

Schema Validity

Output parses as valid JSON matching the [OUTPUT_SCHEMA] exactly

JSON.parse fails or required fields are missing

Automated schema validator run against output

Line Number Accuracy

All line_start and line_end values exist in the provided [DIFF] and point to the correct finding

Line numbers reference deleted lines, are out of bounds, or point to unrelated code

Parse diff hunks; assert each finding's line range falls within a valid hunk

Severity Consistency

severity field matches the [SEVERITY_RUBRIC] definition for the described impact

A cosmetic whitespace issue is marked critical or a security vulnerability is marked info

LLM-as-Judge pairwise comparison against rubric definitions for a sample of findings

Finding Relevance

Every finding describes a real issue in the diff context; no hallucinated problems

Finding describes code or logic not present in the [DIFF] or misinterprets standard patterns as bugs

Human review of a random sample; automated check for hallucinated identifiers

Fix Suggestion Actionability

suggested_fix contains a concrete, minimal code change that addresses the finding

Suggestion is a vague comment like 'fix this' or proposes a change that breaks other functionality

Apply suggestion as a patch in a sandbox; assert tests still pass and the finding is resolved

Category Classification

category field matches one of the allowed enum values in [OUTPUT_SCHEMA] and fits the finding

Finding is misclassified (e.g., a security issue labeled as style) or uses an undefined category

Enum membership check; spot-check 10% of findings for category fit manually

Noise Floor

Zero findings reported for trivial or non-issue diffs (e.g., whitespace-only changes)

Output contains findings for a diff that has no functional changes

Run prompt against a known-clean diff; assert findings array is empty

Output Truncation

All findings in the diff are captured; output is not cut off mid-JSON

JSON is incomplete or the last finding is truncated due to max token limits

Validate JSON completeness; assert the number of findings matches a manual count for complex diffs

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt\nAdd strict JSON schema validation with retries. Include a `[CONSTRAINTS]` block that enforces line number grounding, severity definitions, and a maximum finding count. Wire the output into a CI/CD harness that validates schema, checks line references against the actual diff, and logs every review for audit.\n\n```markdown\n[CONSTRAINTS]\n- Every finding MUST reference a line number present in the diff.\n- Severity levels: CRITICAL (security, data loss), HIGH (bug, regression), MEDIUM (maintainability), LOW (style, nit).\n- Maximum 15 findings. If more exist, prioritize by severity then impact.\n- If no issues found, return {"findings": []}. Do not invent problems.\n```\n\n### Watch for\n- Silent format drift when model versions change\n- Missing human review gate for CRITICAL findings\n- Schema validation passing but line numbers still wrong\n- Review fatigue when the model flags too many LOW findings

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.