Stale Code Sample Identification Prompt

This prompt is designed for DevRel engineers, documentation maintainers, and technical writers who need to systematically audit a collection of code samples for staleness. The core job-to-be-done is identifying code snippets that no longer compile, execute, or follow current best practices due to deprecated APIs, breaking language changes, or outdated library versions. The ideal user is someone responsible for maintaining a high-quality developer documentation surface where broken examples directly erode trust and increase support load. Required context includes the code sample itself, the target language and version, and ideally the specific framework or library versions the sample claims to support.

Use this prompt when you have a batch of code samples from documentation, READMEs, or tutorials and you need a structured, sample-by-sample audit report. It is most effective when paired with a downstream compilation or linting harness that can verify the model's claims. Do not use this prompt for real-time code execution, for generating new code examples from scratch, or for reviewing application logic correctness beyond API compatibility. It is not a substitute for a full CI/CD pipeline but rather a pre-filtering step that surfaces likely failures before you invest in automated build verification. The prompt assumes you can provide explicit version context; without it, the model will guess and may produce unreliable results.

Before running this prompt, ensure you have a clear inventory of the code samples to audit, including their source locations and any documented version requirements. After receiving the audit report, you should route flagged samples to a language-specific build harness for confirmation. Do not automatically delete or rewrite samples based solely on the model's output—treat the audit as a prioritized investigation queue. The next step is to integrate this prompt into a scheduled documentation health check, running it against your entire sample corpus on a regular cadence aligned with your dependency release cycles.

The prompt below is designed to be copied directly into your AI harness. It accepts a list of code samples extracted from your documentation, along with the target language runtime context, and returns a structured audit of each sample's health. Use square-bracket placeholders to inject your specific inputs before sending the request to the model. The template is language-agnostic but expects you to provide the compilation or linting output as part of the context so the model can reason about errors rather than guess at syntax.

textCopy
You are a documentation QA engineer auditing code samples for staleness.

## INPUT
[CODE_SAMPLES]

## CONTEXT
- Target language and version: [LANGUAGE_RUNTIME]
- Compilation or linting output per sample: [VALIDATION_OUTPUT]
- API deprecation schedule: [DEPRECATION_SCHEDULE]
- Known breaking changes in recent releases: [BREAKING_CHANGES]

## OUTPUT_SCHEMA
Return a JSON array of objects with the following fields for each sample:
- sample_id: string (from input)
- status: "current" | "deprecated_api" | "syntax_error" | "version_mismatch" | "removed"
- error_details: string | null (compiler/linter error message or deprecation notice)
- deprecated_apis_used: string[] (list of deprecated functions, methods, or classes)
- suggested_replacement: string | null (corrected code snippet or migration guidance)
- minimum_runtime_version: string | null (earliest version where sample compiles)
- breaking_change_impact: "none" | "minor" | "major"
- human_review_required: boolean

## CONSTRAINTS
- Do not invent compiler errors. Only reference errors present in [VALIDATION_OUTPUT].
- If a sample compiles but uses deprecated APIs, set status to "deprecated_api" and include the deprecation notice from [DEPRECATION_SCHEDULE].
- If no validation output is available for a sample, set status to "version_mismatch" and flag human_review_required as true.
- For syntax errors, include the exact error message and line number if available.
- Suggested replacements must be compilable in the target [LANGUAGE_RUNTIME].
- If a breaking change in [BREAKING_CHANGES] affects the sample, note it even if the sample currently compiles.

## RISK_LEVEL
High. Incorrect code samples in documentation cause developer trust erosion and support burden. Flag any sample you are uncertain about for human review.

To adapt this template, replace each square-bracket placeholder with real data from your documentation pipeline. [CODE_SAMPLES] should be a JSON array of objects with at minimum a sample_id and code field. [VALIDATION_OUTPUT] is the raw output from your CI step that compiles or lints each sample against the target runtime. If you don't have automated validation wired up yet, the prompt will still flag samples for human review, but the accuracy of error_details and suggested_replacement will degrade. Wire the validation harness first, then use this prompt to turn raw compiler output into a prioritized audit report. For high-risk SDKs or security-sensitive APIs, always require human review on samples flagged with breaking_change_impact: "major" before publishing updates.

This prompt is designed to be embedded in a documentation pipeline, not used as a one-off chat interaction. The core workflow involves extracting code blocks from documentation sources, submitting each block alongside its declared language and target version context, and then processing the structured audit output. The prompt expects a specific input shape—a code block, a language identifier, and a version constraint—so the harness must parse documentation files (Markdown, MDX, reStructuredText) and isolate fenced code blocks before calling the model. A pre-processing step should strip irrelevant markup and attach metadata such as the source file path, the section heading, and the last-modified date of the document. This metadata is critical for downstream reporting and for routing fixes to the right owners.

The implementation harness should wrap the model call in a validation and retry layer. The prompt's output schema is a JSON object with a status field (current, stale, error, unknown) and an array of issues. After receiving the model response, validate the JSON structure against a strict schema. If the model returns status: "error" but the issues array is empty, or if the replacement_suggestion field is missing when status is stale, flag the response for retry. Use a retry budget of 2–3 attempts with a backoff, and on each retry, include the validation error message in the prompt context so the model can self-correct. For high-stakes documentation (e.g., security libraries, payment SDKs), route all stale and error findings to a human review queue before publication. The harness should log every model call, the raw output, the validation result, and the final disposition to an observability platform for audit and prompt drift analysis.

The most critical integration point is the language-specific compilation or linting validation step. The prompt alone cannot guarantee that a code sample compiles; it can only flag likely issues based on its training data. After the model identifies a sample as potentially stale, the harness must execute a real linter or compiler for the declared language and version. For example, for a Python sample targeting version 3.9+, run python3.9 -m py_compile on the extracted block. For JavaScript, use ESLint with the appropriate preset. The harness should compare the model's findings with the actual compiler output and reconcile discrepancies. If the model flags an issue that the compiler does not reproduce, mark it as a false positive and feed that back into your evaluation dataset. If the compiler catches an issue the model missed, log it as a false negative and consider adding it as a few-shot example in the next prompt iteration. This human-in-the-loop compiler integration is what elevates the prompt from a heuristic guess to a reliable documentation gate.