CI Pipeline Failure Signal Isolation Prompt

Use this prompt when a CI pipeline run contains mixed failures and you need to separate infrastructure-level problems from genuine test or build failures before the on-call engineer starts investigating. The ideal user is a DevOps engineer, SRE, or platform engineer who has access to raw pipeline logs, runner telemetry, and environment metadata. The prompt expects a structured log dump, a list of failed jobs, and any available infrastructure state (CPU/memory pressure, network partition indicators, artifact checksums, service health endpoints). Without this context, the model cannot reliably distinguish a container OOM kill from a test assertion failure.

This prompt is designed for post-run triage, not real-time pipeline monitoring. It works best when you feed it the complete output of a failed pipeline stage—stdout, stderr, exit codes, and timing data—rather than filtered or truncated logs. The output is a categorized failure report that assigns each failure to one of four buckets: runner resource exhaustion, network partition or service unavailability, artifact corruption or missing dependencies, or genuine test/code failure. Each bucket includes evidence extraction, remediation steps, and a rerun criteria recommendation. The prompt also flags ambiguous cases where multiple signals overlap, which is common when a network blip causes a test timeout that looks like a code defect.

Do not use this prompt for real-time pipeline gating decisions, for diagnosing flaky tests in isolation (use the Flaky Test Log Pattern Extraction Prompt instead), or for pipelines where you cannot share full log output due to security constraints. If the failure involves a third-party service whose internals you cannot inspect, pair this prompt with the Third-Party API Flakiness Attribution Prompt. For high-severity production deployment pipelines, always require human review of the categorized output before acting on remediation steps—the model can misclassify a novel failure mode that spans multiple categories. After classification, route genuine test failures to the Flaky Test vs Genuine Defect Classification Prompt for deeper investigation.

This prompt template is designed to be copied directly into your AI harness, prompt library, or orchestration layer. It expects raw or structured CI pipeline logs as input and produces a categorized failure report that separates infrastructure issues (runner exhaustion, network partitions, artifact corruption, service unavailability) from genuine test failures. The square-bracket placeholders let you inject your specific log format, output schema, constraints, and risk tolerances without rewriting the core instruction set. Use this template as the foundation for a pipeline triage agent, a Slackbot command, or an automated incident enrichment step.

textCopy
You are a CI pipeline failure isolation analyst. Your job is to analyze pipeline execution logs and separate infrastructure failures from genuine test failures. Do not diagnose the root cause of test failures themselves—only classify whether a failure is infrastructure-caused or test-caused.

## INPUT
[PIPELINE_LOGS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "pipeline_run_id": "string",
  "analysis_timestamp": "ISO8601",
  "summary": {
    "total_failures": int,
    "infrastructure_failures": int,
    "test_failures": int,
    "unclassified_failures": int
  },
  "infrastructure_failures": [
    {
      "failure_id": "string",
      "category": "RUNNER_RESOURCE_EXHAUSTION | NETWORK_PARTITION | ARTIFACT_CORRUPTION | SERVICE_UNAVAILABILITY | UNKNOWN_INFRA",
      "evidence": ["log line or timestamped excerpt"],
      "remediation_steps": ["actionable step"],
      "confidence": "HIGH | MEDIUM | LOW"
    }
  ],
  "test_failures": [
    {
      "failure_id": "string",
      "test_name": "string",
      "evidence_of_test_failure": ["log line or assertion failure"],
      "infrastructure_indicators_checked": ["what was ruled out"]
    }
  ],
  "unclassified_failures": [
    {
      "failure_id": "string",
      "reason_unclassified": "string",
      "recommended_investigation": "string"
    }
  ],
  "rerun_criteria": {
    "safe_to_rerun": ["failure_id"],
    "do_not_rerun_without_fix": ["failure_id"],
    "rerun_conditions": "string describing preconditions for rerun"
  }
}

## CLASSIFICATION RULES
- RUNNER_RESOURCE_EXHAUSTION: OOM kills, disk full errors, CPU throttling messages, container eviction events.
- NETWORK_PARTITION: Connection refused, DNS resolution failures, timeout patterns consistent with network splits, proxy errors.
- ARTIFACT_CORRUPTION: Checksum mismatches, incomplete downloads, corrupted container images, missing dependencies.
- SERVICE_UNAVAILABILITY: External service 503/502 errors, internal service health check failures, database connection pool exhaustion.
- If a failure has mixed signals, classify by the dominant signal and note the ambiguity in evidence.
- If you cannot determine the category with at least LOW confidence, place it in unclassified_failures.

## CONSTRAINTS
[CONSTRAINTS]

## EXAMPLES
[EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, replace each placeholder with your pipeline's specifics. For [PIPELINE_LOGS], inject the raw log output from your CI system—this can be a concatenated string, a structured JSON array of log lines, or a reference to a log file that your harness reads before prompting. For [CONSTRAINTS], add rules like "ignore known flaky tests listed in [FLAKY_TEST_REGISTRY]" or "treat any failure in the staging environment as high severity." For [EXAMPLES], provide 2-3 few-shot examples of correctly classified failures from your own pipeline history to improve accuracy. For [RISK_LEVEL], specify whether this is a pre-merge check (lower risk, auto-rerun acceptable) or a release-blocking pipeline (higher risk, require human review before rerunning). After adapting, validate the output against your pipeline's known failure history before deploying to production triage workflows.

This prompt is designed to be the first stage in a CI failure triage pipeline, not a standalone chatbot. The primary integration point is a post-build or post-stage hook in your CI/CD system (e.g., a Jenkins post block, a GitHub Actions if: failure() step, or a GitLab CI after_script). The application harness must collect the raw, unredacted pipeline logs, the pipeline definition file, and the current infrastructure-as-code configuration, then assemble them into the [PIPELINE_LOGS], [PIPELINE_CONFIG], and [INFRA_CONFIG] input variables. Because these logs can be massive, implement a pre-processing step that truncates each log to the last N lines (e.g., 5000) and filters for well-known failure markers (ERROR, FATAL, exit code 1, connection refused) to keep the prompt within the model's context window. The harness should also inject the [OUTPUT_SCHEMA] as a strict JSON schema that your application will validate before acting on any recommendation.

After the model returns a response, the harness must run a multi-step validation and safety layer before the output is surfaced to an on-call engineer. First, validate the JSON structure against your expected schema; if parsing fails, retry the prompt once with a simplified schema and a specific error message. Second, implement a rule-based safety check: if the model classifies a failure as infrastructure and recommends a remediation like 'restart the production database', the harness must automatically escalate to a human without executing the action. Third, log every model input, output, and the final validated classification to your observability platform (e.g., Datadog, Grafana, or a custom audit table) for post-incident review. For model choice, start with a fast, cost-effective model like claude-3-haiku or gpt-4o-mini for initial triage, and only escalate to a larger model if the first response has a confidence_score below 0.85 or the JSON validation fails.

Avoid wiring this prompt directly to an auto-remediation playbook. The output infrastructure_remediation_steps should be treated as a suggestion for a human operator, not an executable script. A safe implementation will post the structured failure report to your team's incident channel (e.g., Slack, PagerDuty) with a clear call to action: 'Review the categorized failure and approve the recommended rerun criteria.' The test_rerun_criteria field should be parsed by your harness to automatically trigger a selective retry of only the failed test suite, but only after the identified infrastructure issue is resolved or explicitly waived by an engineer. The next step is to build an evaluation set from 20-30 historical pipeline failures where you know the ground-truth root cause, and run this prompt against them to calibrate your confidence_score threshold for fully automated vs. human-review workflows.