Inferensys

Prompt

Pull Request Smell Summary Prompt

A practical prompt playbook for using Pull Request Smell Summary Prompt in production AI workflows.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Understand the job-to-be-done, ideal user, required context, and when not to use this prompt.

This prompt is designed for CI/CD pipelines and platform engineering teams that need to generate a reviewer-friendly summary of code smells from a full pull request diff. It takes the raw diff as input and produces a structured summary grouped by smell category, a risk heatmap, the top three action items, and a go/no-go recommendation suitable for automated merge gates. The primary job-to-be-done is triage and decision support, not detailed code review. Use this prompt when you need a single, scannable output that distills multiple smell detectors into a decision-ready format for a release manager or on-call engineer who needs to decide whether a PR should proceed, pause, or escalate.

The ideal user is a platform engineer embedding this prompt into an automated merge queue or a tech lead reviewing a daily digest of high-risk PRs. The prompt assumes that detailed smell detection has already occurred—either via upstream static analysis tools, separate detection prompts, or that the model is capable of performing that detection within the same call. You must provide the full PR diff as [INPUT] and, optionally, a [CONTEXT] block containing team-specific severity thresholds, known false positive patterns, or domain-specific anti-pattern catalogs. The output schema should be specified in [OUTPUT_SCHEMA] to enforce a consistent JSON structure with fields for smell_summary, risk_heatmap, top_action_items, and go_no_go_recommendation. Without a strict output schema, the model may produce narrative prose that is difficult to parse in an automated pipeline.

Do not use this prompt for line-by-line code review, for generating fix suggestions, or as a replacement for detailed analysis. It is a summarization and triage layer, not a diagnostic tool. If you need specific file-and-line mappings with remediation guidance, use the sibling prompt 'PR Diff Code Smell Detection Prompt Template' instead. Do not use this prompt when the diff is too large for a single context window—chunk the diff by file or module and run detection prompts first, then feed the aggregated findings into this summary prompt. For high-risk domains such as security vulnerabilities or data integrity changes, always require human review of the go/no-go recommendation before merging. The prompt is a decision aid, not a decision maker.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Pull Request Smell Summary Prompt delivers value and where it introduces risk. Use these cards to decide if this prompt fits your pipeline before wiring it into a merge gate.

01

Good Fit: High-Volume CI Pipelines

Use when: your team reviews 10+ PRs daily and needs a consistent first-pass summary to reduce reviewer fatigue. The prompt scales judgment across many diffs without replacing human review. Guardrail: always position the output as a reviewer aid, not a gate. Require human approval for merge decisions.

02

Bad Fit: Trivial or Single-Line Changes

Avoid when: the diff is a typo fix, version bump, or config value change. The prompt will over-generate findings, creating noise that erodes trust. Guardrail: add a diff-size threshold (e.g., skip if changed lines < 10) before invoking the prompt. Route small changes directly to human review.

03

Required Input: Full Diff with File Context

What to watch: the prompt needs the complete unified diff plus surrounding file context to produce accurate smell categorization. Truncated diffs cause false negatives on cross-file smells like duplicated logic. Guardrail: validate that the diff input includes at least 5 lines of context per hunk before calling the model.

04

Required Input: Team Severity Rubric

What to watch: without a team-specific severity definition, the model defaults to generic heuristics that may not match your risk tolerance. A data-access anti-pattern might be critical for one team and acceptable for another. Guardrail: pass a short severity rubric as part of [CONSTRAINTS] that defines what constitutes Critical, High, Medium, and Low for your codebase.

05

Operational Risk: Merge Gate Over-Reliance

What to watch: teams may wire the go/no-go recommendation directly into an automated merge gate, blocking deployments on model judgment alone. The model can miss context-dependent risks or flag acceptable patterns. Guardrail: never use the recommendation as the sole merge gate. Require at least one human reviewer to acknowledge the summary before merge.

06

Operational Risk: Summary Accuracy Drift

What to watch: as your codebase evolves, the prompt's smell detection may drift—flagging new patterns incorrectly or missing emerging anti-patterns. Guardrail: run periodic eval checks comparing the summary's top-3 action items against detailed findings from a human reviewer or a more expensive model. Trigger a prompt review if agreement drops below 80%.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A ready-to-use prompt template for generating structured code smell summaries from pull request diffs, including risk heatmaps and merge recommendations.

This prompt template is designed to be dropped directly into your CI/CD pipeline or code review automation. It takes a raw pull request diff as input and produces a structured JSON summary that your merge gates, review dashboards, or notification systems can consume. The template enforces a strict output schema so downstream tooling can rely on consistent field names, types, and structure—no parsing guesswork required. Before copying, review the placeholders in square brackets and replace them with your team's specific constraints, severity definitions, and risk thresholds.

text
You are a senior code reviewer analyzing a pull request diff for code smells, anti-patterns, and maintainability risks.

## INPUT
[PR_DIFF]

## CONTEXT
- Repository: [REPO_NAME]
- Base branch: [BASE_BRANCH]
- Target branch: [TARGET_BRANCH]
- Team standards: [TEAM_STANDARDS_REFERENCE]
- Files changed: [FILE_LIST]

## CONSTRAINTS
- Only report findings you can map to specific file paths and line ranges in the diff.
- Do not flag style issues already covered by the team's linter configuration.
- If the diff is empty or contains no detectable smells, return an empty findings array with a 'go' recommendation.
- Severity levels: 'critical' (merge blocker), 'high' (must fix before deploy), 'medium' (should fix in this PR), 'low' (note for future cleanup).
- For each finding, include a confidence score from 0.0 to 1.0. Findings below [MIN_CONFIDENCE_THRESHOLD] should be omitted.
- Group findings by category using the categories defined in [CATEGORY_TAXONOMY].

## OUTPUT SCHEMA
Return valid JSON matching this structure exactly:
{
  "summary": {
    "total_findings": <integer>,
    "categories_affected": [<string>],
    "risk_heatmap": {
      "critical": <integer>,
      "high": <integer>,
      "medium": <integer>,
      "low": <integer>
    },
    "top_action_items": [
      {
        "rank": <integer>,
        "finding_id": <string>,
        "rationale": <string>
      }
    ],
    "merge_recommendation": "go" | "no-go" | "conditional-go",
    "recommendation_reason": <string>
  },
  "findings": [
    {
      "id": <string>,
      "category": <string>,
      "severity": "critical" | "high" | "medium" | "low",
      "confidence": <float>,
      "file": <string>,
      "line_range": {
        "start": <integer>,
        "end": <integer>
      },
      "title": <string>,
      "description": <string>,
      "suggestion": <string>,
      "references": [<string>]
    }
  ],
  "diff_metadata": {
    "files_analyzed": <integer>,
    "lines_added": <integer>,
    "lines_removed": <integer>,
    "analysis_timestamp": <string>
  }
}

## INSTRUCTIONS
1. Parse the diff to identify all changed files and line ranges.
2. For each changed file, scan for code smells and anti-patterns defined in [CATEGORY_TAXONOMY].
3. Assign severity based on [SEVERITY_RUBRIC].
4. Group findings by category.
5. Build the risk heatmap by counting findings per severity level.
6. Select the top 3 action items by severity (highest first), then by confidence (highest first).
7. Determine the merge recommendation:
   - 'no-go' if any critical finding exists.
   - 'conditional-go' if high-severity findings exist but have clear, low-risk fix paths.
   - 'go' otherwise.
8. Return the complete JSON object following the output schema exactly.

To adapt this template for your team, start by defining your [CATEGORY_TAXONOMY]—the specific smell categories you care about, such as error handling, concurrency, security, naming, duplication, or test quality. Replace [SEVERITY_RUBRIC] with your team's agreed-upon definitions for each severity level, including examples of what qualifies as critical versus low. Set [MIN_CONFIDENCE_THRESHOLD] to filter out low-confidence noise; 0.6 is a reasonable starting point for most teams. The [TEAM_STANDARDS_REFERENCE] placeholder should point to your style guide, architectural decision records, or coding conventions document so the model can distinguish intentional patterns from actual smells. If your CI system already provides file lists and branch metadata, wire those into [FILE_LIST], [BASE_BRANCH], and [TARGET_BRANCH] automatically rather than asking the model to extract them from the diff.

Before deploying this prompt into an automated merge gate, test it against a golden dataset of known PRs with labeled smells. Validate that the JSON output parses correctly, that finding IDs are unique and stable across runs, and that the merge recommendation aligns with your team's manual review decisions. Pay special attention to false positives on intentional patterns—if your team commonly uses a pattern that the model flags as a smell, add an explicit exclusion to the [CONSTRAINTS] section. For high-risk repositories, keep a human-in-the-loop step: use 'conditional-go' as a signal to request manual review rather than blocking the merge outright. The next section covers how to wire this template into an application harness with validation, retries, and logging.

IMPLEMENTATION TABLE

Prompt Variables

Define these variables before calling the prompt. Validation notes indicate how to check each input at runtime to prevent malformed requests.

PlaceholderPurposeExampleValidation Notes

[PR_DIFF]

Full unified diff of the pull request to analyze

diff --git a/src/auth.py b/src/auth.py @@ -12,7 +12,12 @@ def login(user, pass):

  • return db.query(user)
  • return db.query(user, pass)

Non-empty string. Must contain valid diff headers. Reject if only whitespace or no file changes detected.

[PR_TITLE]

Title of the pull request for context

fix: prevent SQL injection in login flow

String, max 200 chars. If empty, prompt should still function but may produce less contextual summary.

[PR_DESCRIPTION]

Body of the pull request describing intent

Addresses CVE-2024-1234 by parameterizing user input in the auth module.

String, nullable. If null, pass empty string. No validation required beyond null check.

[REPO_LANGUAGE]

Primary programming language of the repository

Python

Must match a supported language identifier. Validate against allowlist: Python, JavaScript, TypeScript, Java, Go, Rust, Ruby, C#, PHP, C++. Reject unknown values.

[TEAM_STANDARDS]

Team-specific code quality rules or style guide excerpts

No mutable default arguments. Use type hints on all public functions.

String, nullable. If provided, must be under 2000 tokens. Truncate with warning if exceeded.

[SEVERITY_RUBRIC]

Custom severity definitions for smell classification

CRITICAL: security vulnerability or data loss. HIGH: production bug risk. MEDIUM: maintainability degradation. LOW: style preference.

String, nullable. If null, use default rubric. If provided, must contain at least CRITICAL and HIGH definitions. Reject if only LOW defined.

[MAX_FINDINGS]

Upper limit on number of smells to return

15

Integer between 5 and 50. Clamp to range if out of bounds. Default to 20 if not provided.

[INCLUDE_SUGGESTIONS]

Whether to include refactoring suggestions for each smell

Boolean. Accept true, false, 1, 0. Default to true if unparseable.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Pull Request Smell Summary Prompt into a CI/CD pipeline or code review application.

The Pull Request Smell Summary Prompt is designed to be called as a single step in an automated pipeline, not as a conversational chat. The primary integration point is a CI/CD workflow (GitHub Actions, GitLab CI, Jenkins) that triggers on pull request creation or update. The application layer is responsible for gathering the full PR diff, assembling the prompt with the correct placeholders, calling the model, validating the structured output, and posting the summary as a review comment or status check. Do not expose this prompt directly to end users; the diff input is large, the output schema is strict, and the risk of hallucinated findings requires automated validation before the summary reaches a human reviewer.

Integration flow: 1) The CI job fetches the PR diff using the provider's API (e.g., git diff origin/main...HEAD). 2) The diff is truncated to a maximum token budget (reserve ~60% of the model's context window for the diff, leaving room for the system prompt and output). If the diff exceeds the budget, split by file and run multiple passes, then merge and deduplicate findings. 3) The prompt template is populated with [PR_DIFF], [REPOSITORY_CONTEXT] (language, framework, team conventions), and [SEVERITY_RUBRIC] (team-specific definitions of Critical, High, Medium, Low). 4) The model is called with response_format set to JSON schema mode (OpenAI) or with a structured output constraint (Anthropic, Gemini). 5) The raw JSON output is validated against the expected schema: required fields (summary, categories, heatmap, top_actions, go_no_go), enum values for severity and category types, and line-range references that fall within the diff boundaries. 6) If validation fails, retry once with the validation errors appended to the prompt as [PREVIOUS_ERRORS]. If the retry also fails, fail the check open (post a warning, not a block) and log the raw output for manual review.

Model selection: Use a model with strong structured output adherence and a context window large enough for typical PR diffs. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are good defaults. For cost-sensitive pipelines, a smaller model (GPT-4o-mini, Claude 3 Haiku) can work if you tighten the output schema and increase validation strictness. Avoid models known to drift on long-context structured generation without explicit JSON mode. Logging and observability: Log the prompt version, model, diff size in tokens, validation pass/fail, retry count, and the final structured output. Attach the smell summary to the PR as a comment or status check with a link to the raw log. This creates an audit trail for false positive disputes and prompt drift detection over time. Human review gate: The go_no_go recommendation is advisory only. For repositories with compliance requirements (SOC 2, HIPAA, PCI), configure the CI check to post a non-blocking summary and require a human reviewer to acknowledge Critical or High findings before merge. Never auto-merge on a go signal alone without human approval for regulated codebases.

Eval and calibration: Before enabling the prompt in production, run it against a golden dataset of 20-30 PRs with known smell findings (both true positives and intentional clean diffs). Measure precision (do flagged smells match known issues?), recall (are known smells missed?), and hallucination rate (are findings referencing lines that don't exist in the diff?). Calibrate the severity rubric by comparing model-assigned severities against team-assigned severities on the golden set. Re-run this eval suite on any prompt version change and set a regression threshold (e.g., recall must not drop below 85% on known Critical smells). Store eval results alongside prompt versions in your prompt registry. Failure modes to monitor: The most common production failure is the model inventing file paths or line numbers that don't exist in the actual diff—always validate line ranges against the diff. The second is category misclassification (e.g., flagging a naming issue as a security smell). The third is severity inflation on large diffs, where the model over-flags to appear thorough. Mitigate these with strict output validation, category-enum constraints, and periodic human spot-checks on a sample of production summaries.

IMPLEMENTATION TABLE

Expected Output Contract

Define the exact shape, types, and validation rules for the Pull Request Smell Summary output. Use this contract to build a parser, validator, or merge-gate decision function.

Field or ElementType or FormatRequiredValidation Rule

smell_summary

object

Top-level object must contain exactly the keys: categories, heatmap, top_actions, recommendation, meta.

categories

array of objects

Each object must have: name (string), count (integer >= 0), findings (array). Array must not be empty if count > 0.

categories[].findings

array of objects

Each object must have: id (string), file (string), line_range (string matching 'L\d+-L\d+'), severity (enum: low|medium|high|critical), smell_type (string), summary (string <= 280 chars).

heatmap

object

Must contain: risk_score (integer 0-100), files_affected (array of strings), hotspots (array of objects with file and risk_contribution_pct). Sum of risk_contribution_pct must be <= 100.

top_actions

array of strings

Must contain exactly 3 items. Each string must be <= 160 chars and start with an imperative verb.

recommendation

string

Must be one of: go, no-go, caution. If no-go, top_actions must not be empty and heatmap.risk_score must be >= 70.

meta

object

Must contain: generated_at (ISO 8601 string), diff_hash (string), model_version (string). No extra keys allowed.

findings[].id

string

Must match pattern 'SMELL-\d{4}' and be unique within the response.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when summarizing PR smells and how to guard against it.

01

Summary Drift from Detailed Findings

What to watch: The summary omits critical smells, invents findings not in the detailed list, or changes severity levels during summarization. Guardrail: Implement a grounding check that verifies every claim in the summary maps back to at least one detailed finding. Flag summary-only claims for removal.

02

Go/No-Go Decision Without Evidence

What to watch: The model recommends blocking a merge based on a vague risk heatmap without citing specific, high-severity findings. Guardrail: Require the go/no-go recommendation to explicitly reference the top-3 action items. If no high-severity items exist, default to a 'go' recommendation with an explicit evidence gap statement.

03

Risk Heatmap Inflation or Deflation

What to watch: The model consistently scores all PRs as high-risk (alarm fatigue) or low-risk (missed critical issues) due to poor calibration. Guardrail: Calibrate the heatmap against a golden dataset of PRs with known risk levels. Track severity distribution over time and alert if the ratio of high-risk PRs deviates beyond a set threshold.

04

Category Grouping Errors

What to watch: Smells are placed in the wrong category (e.g., a security vulnerability grouped under 'readability'), causing reviewers to miss critical context. Guardrail: Use a structured output schema with strict enum categories. Add a post-processing validation step that checks if the smell's description semantically matches its assigned category.

05

Action Item Vagueness

What to watch: The top-3 action items are generic ('fix the bug', 'improve code quality') and lack specific file paths, line numbers, or concrete steps. Guardrail: Enforce a schema for action items that requires a file reference, a line range, and a one-sentence actionable directive. Reject summaries with placeholder action items.

06

Context Window Truncation

What to watch: A large PR diff exceeds the context window, causing the model to summarize only the first part of the diff and silently miss smells in the truncated portion. Guardrail: Implement a diff chunking strategy with overlapping windows. Require the model to output a chunks_processed field and compare it against the total number of chunks sent. Flag mismatches for human review.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the quality of the Pull Request Smell Summary Prompt's output before integrating it into a CI/CD pipeline or merge gate. Each criterion targets a specific failure mode observed in production smell-summary systems.

CriterionPass StandardFailure SignalTest Method

Category Grouping Accuracy

Every smell is assigned to exactly one correct category from the defined taxonomy (e.g., Security, Performance, Maintainability).

A smell appears in multiple categories or is placed in an obviously wrong category (e.g., an N+1 query under 'Naming').

Spot-check 10 random smells from the output against a manually labeled golden set. Pass if 9/10 are correctly categorized.

Risk Heatmap Calibration

The go/no-go recommendation aligns with the heatmap: a 'no-go' must have at least one Critical or High severity smell. A 'go' must have no Critical smells.

A 'no-go' recommendation is issued with only Low severity smells, or a 'go' is issued with a Critical security vulnerability present.

Run 5 diffs with known severity profiles (1 Critical-only, 1 High-only, 1 Low-only, 1 mixed, 1 clean). Check recommendation alignment for all 5.

Top-3 Action Item Relevance

The top-3 action items are the highest-severity, highest-impact smells from the detailed findings. They must be deduplicated.

An action item repeats a finding from the detailed list verbatim without deduplication, or a Low severity smell appears in the top-3 while a Critical smell is omitted.

For 3 diffs with mixed severity, manually rank the top-3 by severity and impact. Compare to the prompt's top-3. Pass if 2/3 items match the manual ranking.

Finding Groundedness

Every smell in the detailed findings references a specific file path and line range present in the input diff.

A smell cites a file not in the diff, a line range that doesn't exist, or describes a pattern with no code evidence (hallucination).

Parse the output and cross-reference each file path and line range against the input diff. Fail if any reference is invalid.

Summary Conciseness

The summary section is under 150 words and contains no code blocks or line references.

The summary exceeds 150 words, includes raw code snippets, or duplicates the detailed findings section.

Automated word-count check on the summary field. Manual check for code block presence. Fail on either condition.

Output Schema Compliance

The output is valid JSON matching the [OUTPUT_SCHEMA] exactly. All required fields are present and non-null.

The output is missing a required field (e.g., 'heatmap'), contains extra top-level keys, or a required field is null.

Validate the raw model response against the [OUTPUT_SCHEMA] using a JSON schema validator. Fail on any validation error.

False Positive Rate

Fewer than 15% of reported smells are false positives (flagged as a smell but not a genuine issue).

More than 15% of smells are false positives, indicating the prompt is over-flagging and will erode reviewer trust.

Run on 3 diffs with pre-labeled smells. Calculate false positive rate = (false positives / total reported smells). Fail if >15% on any diff.

Go/No-Go Decision Consistency

The same diff produces the same go/no-go recommendation across 3 separate runs with temperature=0.

The recommendation flips between 'go' and 'no-go' across runs, indicating instability in severity assessment.

Run the same diff 3 times with temperature=0. Pass if the go/no-go field is identical across all 3 runs.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single model call and minimal post-processing. Remove the structured output schema requirement and accept a markdown summary. Replace the go/no-go recommendation with a simple risk flag.

code
Analyze this PR diff and produce a code smell summary grouped by category. Include a risk heatmap (High/Medium/Low) and the top 3 action items.

PR Diff:
[PR_DIFF]

Watch for

  • Missing schema checks leading to inconsistent formatting
  • Overly broad smell categories without file/line references
  • Go/no-go decisions made without severity thresholds
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.