Error Handling Coverage Assessment Prompt Template

Reliability engineers and test infrastructure teams use this prompt to audit a service's error handling robustness. The prompt instructs an AI coding agent to trace error propagation paths through source code, identify error conditions that lack test coverage, flag assertions with inadequate depth, and surface missing recovery validation. The output is a structured coverage gap report organized by error type, suitable for prioritization in a test remediation backlog.

Use this prompt when you have access to source code, existing test files, and optionally coverage reports or production error logs. The prompt works best for services where error handling is explicit—try/catch blocks, error return values, status code branches, retry logic, circuit breakers, and fallback paths. It is less effective for codebases that rely on implicit error propagation through unchecked exceptions or panic-style error handling without structured recovery. Before running the prompt, ensure the AI coding agent has read access to the relevant source directories, test directories, and any coverage report files. If production error logs are available, include them to ground the analysis in real failure patterns rather than theoretical error paths alone.

Do not use this prompt for real-time incident response or for services where the codebase is unavailable for static analysis. This prompt performs a static audit of error handling coverage, not a runtime diagnosis of an active incident. It will not catch error conditions that arise from infrastructure failures, network partitions, or resource exhaustion unless those conditions are explicitly modeled in the code. For production incidents, use the Error Diagnosis and Runtime Investigation Prompts instead. For services where only compiled artifacts are available without source access, this prompt cannot perform meaningful analysis. Always review the output before committing to a remediation plan—the AI may flag error paths that are intentionally left untested due to platform guarantees or operational controls that are not visible in the codebase.

This template is designed to be pasted into your AI coding agent or LLM interface after replacing the square-bracket placeholders with concrete values. It instructs the model to act as a reliability engineer auditing a service's error handling surface. The prompt forces the model to reason about error propagation paths—how errors flow from their origin through callers, middleware, and recovery logic—and then cross-reference those paths against the existing test suite to find coverage gaps. Use this when you need a structured, evidence-backed assessment of where your error handling is brittle, not just a list of untested functions.

textCopy
You are a senior reliability engineer auditing the error handling robustness of a service. Your task is to map error propagation paths through the codebase and identify error conditions that lack adequate test coverage, have insufficient assertion depth, or are missing recovery validation.

## INPUT
- Source code files: [CODEBASE_FILES]
- Test files: [TEST_FILES]
- Test coverage report (optional): [COVERAGE_REPORT]
- Service architecture description: [ARCHITECTURE_DESCRIPTION]
- Known error types and their expected handling: [ERROR_CATALOG]

## OUTPUT_SCHEMA
Produce a JSON object with the following structure:
{
  "error_propagation_map": [
    {
      "error_type": "string (e.g., TimeoutError, DatabaseConnectionError)",
      "origin_location": "string (file:line where error is raised or thrown)",
      "propagation_path": ["string (ordered list of callers, middleware, and handlers the error passes through)"],
      "recovery_points": ["string (locations where error is caught, retried, or transformed)"],
      "expected_behavior": "string (documented or inferred handling strategy)"
    }
  ],
  "coverage_gaps": [
    {
      "error_type": "string",
      "gap_category": "UNTESTED_PATH | WEAK_ASSERTION | MISSING_RECOVERY_VALIDATION | UNVERIFIED_SIDE_EFFECT",
      "location": "string (file:line or function name)",
      "description": "string (what is missing and why it matters)",
      "severity": "CRITICAL | HIGH | MEDIUM | LOW",
      "suggested_test_scenario": "string (concrete test case description)"
    }
  ],
  "summary": {
    "total_error_paths_identified": "number",
    "total_coverage_gaps": "number",
    "critical_gaps": "number",
    "high_gaps": "number",
    "assessment_confidence": "HIGH | MEDIUM | LOW (based on codebase coverage and documentation completeness)"
  }
}

## CONSTRAINTS
- Only report gaps where you have concrete evidence from the provided code and test files. Do not speculate about hypothetical errors not present in the code.
- For each gap, you MUST cite the specific source code location and the specific test file (or absence thereof) that supports your finding.
- If a coverage report is not provided, state your confidence as MEDIUM or LOW and note that static analysis alone may miss runtime error paths.
- Classify a gap as CRITICAL only if the error path involves data loss, corruption, security exposure, or unrecoverable service outage.
- Do not flag error paths that are handled by framework-level middleware unless the middleware itself is untested or misconfigured.

## TOOLS
You may use the following tools if available:
- search_codebase: to find error raising, throwing, and catching patterns
- read_file: to inspect specific source and test files
- grep: to search for error type references across the codebase

## RISK_LEVEL: HIGH
This assessment informs production reliability decisions. If you are uncertain about a gap, mark it with lower confidence rather than omitting it. Human review of all CRITICAL and HIGH severity gaps is required before remediation.

After pasting this template, replace each square-bracket placeholder with concrete data from your repository. The [CODEBASE_FILES] and [TEST_FILES] placeholders should include the relevant source and test file paths or contents. If you lack a formal coverage report, omit [COVERAGE_REPORT] but expect the model to flag lower confidence. The [ERROR_CATALOG] is optional but strongly recommended—it can be a list of known error types your service handles, drawn from documentation or prior incidents. For high-risk services, always pair this prompt's output with a human review step before acting on any CRITICAL or HIGH severity gap. Validate the output against the schema before integrating it into your reliability dashboard or remediation backlog.

This prompt is designed to be integrated into a code analysis pipeline, not run as a one-off chat interaction. The primary integration point is a CI/CD or scheduled audit job that triggers when coverage reports, static analysis results, or new code diffs are available. The harness must supply the prompt with structured inputs—source code, existing test files, and error propagation metadata—and then parse, validate, and route the structured output for action.

Input Assembly: Before calling the model, assemble the [SOURCE_CODE] and [TEST_CODE] blocks by walking the repository's abstract syntax tree (AST) to extract functions with throw, raise, reject, or error-return patterns. For each error site, trace the call graph to identify upstream callers and downstream recovery points. Package this as a structured [ERROR_PROPAGATION_MAP] with fields: error_type, origin_file, propagation_path, and recovery_point. The [EXISTING_TEST_COVERAGE] input should be a machine-readable coverage report (e.g., lcov or JaCoCo XML) filtered to the files in scope. Model Selection: Use a model with strong code reasoning and large context windows (e.g., Claude 3.5 Sonnet, GPT-4o, or Gemini 1.5 Pro). The prompt's structured output requirement and multi-file reasoning demand a capable frontier model; smaller or local models may produce malformed JSON or miss cross-file error propagation paths.

Output Validation and Routing: The prompt requests a JSON array of gap objects. Implement a post-processing validator that checks: (1) every error_type references a real exception class or error code in the codebase, (2) every file_path exists in the repository, (3) severity is one of the allowed enum values, and (4) suggested_test_scenario is non-empty. Reject and retry malformed outputs once; if the retry also fails, log the failure and alert the reliability team. Valid outputs should be written to a gap tracking system (e.g., a GitHub Issue with the error-handling-coverage label, a Jira ticket, or a dedicated dashboard database). Each gap should include a unique fingerprint (hash of error_type + file_path + function_name) to enable deduplication across runs and track remediation over time.

Human Review and Escalation: Gaps with severity: critical—especially those in authentication, data persistence, or network boundary code—should automatically trigger a notification to the service owner and create a high-priority ticket. Do not auto-generate patches from this prompt's output; the suggested test scenarios are starting points for a human engineer to write precise, context-aware tests. Run this assessment on every major release branch and on a weekly schedule for the main branch. Track gap counts over time as a reliability metric: a rising trend in uncovered error paths signals that error handling discipline is slipping and warrants a team-level discussion.