Inferensys

Prompt

Extract and Validate Python Code Blocks Prompt Template

A practical prompt playbook for extracting Python code blocks from model responses and validating syntax, imports, and style before file insertion.
ML engineer running AI model benchmarks, performance charts on multiple screens, late night home office setup.
PROMPT PLAYBOOK

When to Use This Prompt

Understand the ideal use case, required context, and boundaries for the Extract and Validate Python Code Blocks prompt.

This prompt is designed for Python-specific tooling that needs to extract fenced code blocks from AI model responses and validate them before writing to disk. Use it when you are building coding agents, documentation generators, or IDE integrations that must guarantee Python code is syntactically valid, imports are available, and style is compliant before file insertion. The core job-to-be-done is transforming a raw, untrusted model response containing one or more Python code blocks into a set of validated, linted, and file-ready code segments that won't break downstream tooling.

The ideal user is a developer or platform engineer wiring an LLM into a code generation pipeline where output correctness is non-negotiable. You should have access to a Python runtime for AST parsing and import resolution, and you should be prepared to run flake8, pylint, or your team's linter as a post-extraction gate. This prompt assumes the input is a complete model response—not a streaming chunk—and that the response uses standard markdown fenced code blocks with a python language tag. It is not suitable for extracting non-Python code, handling streaming responses where fences may be incomplete, or recovering from severely malformed markdown where the model has lost track of backtick pairing.

Do not use this prompt when you need to extract shell commands, configuration files, or multi-language projects from a single response. For those cases, use the Multi-Language Code Block Extraction or Shell Command Extraction playbooks instead. Also avoid this prompt if your pipeline cannot tolerate AST parse failures—if the model generates pseudocode or incomplete snippets, the validator will reject them, and you'll need a fallback repair loop. For high-risk production systems, always pair this extraction step with human review gates when the code will be executed automatically, and log every extraction failure for prompt debugging and model performance tracking.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Extract and Validate Python Code Blocks prompt works well, where it breaks, and what you must provide before using it in a production pipeline.

01

Good Fit: Automated Code Review Pipelines

Use when: You need to extract Python code from AI-generated responses, validate syntax via AST parse, and check import availability before writing to disk. Guardrail: Always run the extracted code through a sandboxed linter and AST checker before merging into any branch.

02

Bad Fit: Pseudocode or Incomplete Snippets

Avoid when: The model response contains explanatory pseudocode, partial examples, or prose-heavy code fragments that are not intended to be executable. Guardrail: Pre-filter responses with a classifier that detects runnable vs. illustrative code blocks before extraction.

03

Required Inputs

Must provide: A model response containing fenced Python code blocks with explicit language tags. Guardrail: If language tags are missing or ambiguous, use a language-detection fallback with a confidence threshold before attempting AST validation.

04

Operational Risk: Unsafe Imports and System Calls

What to watch: Extracted code may contain os.system, subprocess, eval, or network calls that pose security risks when executed. Guardrail: Scan the AST for dangerous call patterns and block or flag blocks containing them before any automated execution.

05

Operational Risk: Stale or Unavailable Imports

What to watch: The code references packages not installed in the target environment, causing runtime failures. Guardrail: Resolve imports against the target environment's package list during validation and report missing dependencies before file insertion.

06

Operational Risk: Style Drift and Lint Noise

What to watch: Valid code that violates project style guides creates lint noise and review friction. Guardrail: Run extracted code through the project's configured linter and auto-formatter, and include lint results alongside the validated block.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A ready-to-use prompt for extracting Python code blocks from model responses and validating them against syntax, import availability, and style rules before file insertion.

This prompt template is designed to be placed in your system instructions or sent as a user message when you need to extract Python code blocks from a model's response and validate them before writing to disk. It handles the full pipeline: identifying fenced Python blocks, running AST parse checks, resolving import availability, applying style linting, and returning a structured report with fix suggestions. Replace the square-bracket placeholders with your specific constraints, style configuration, and allowed import whitelist.

text
You are a Python code extraction and validation engine. Your job is to extract all Python code blocks from the provided [INPUT] and validate each one before it can be written to a file.

## Extraction Rules
1. Identify all fenced code blocks with language tag `python`, `py`, `python3`, or untagged blocks that contain unambiguous Python syntax.
2. Ignore inline code spans, pseudocode, and interactive shell sessions (lines starting with `>>>` or `...`).
3. For each extracted block, record the block index, language tag, and raw content.

## Validation Pipeline
For each extracted block, run the following checks in order. Stop at the first failure and report it.

### 1. Syntax Validation
- Parse the code with Python's `ast.parse()` at the target version [TARGET_PYTHON_VERSION].
- If parsing fails, report the error line, column, and message. Do not proceed to further checks.

### 2. Import Availability Check
- Extract all top-level import statements (`import X` and `from X import Y`).
- Compare each imported module against the allowed import whitelist: [ALLOWED_IMPORTS].
- If an import is not in the whitelist, flag it as UNAVAILABLE and suggest alternatives from the whitelist if possible.
- If the whitelist is empty, skip this check and mark imports as UNVERIFIED.

### 3. Style Compliance
- Run the code through the style checker specified in [STYLE_CONFIG].
- Report violations by line number, rule code, and message.
- Classify violations as ERROR (must fix) or WARNING (should fix) based on [STYLE_SEVERITY_THRESHOLD].

## Output Schema
Return a JSON object with this exact structure:
{
  "blocks": [
    {
      "index": 0,
      "language_tag": "python",
      "content": "<raw code>",
      "validation": {
        "syntax": {
          "valid": true,
          "error": null
        },
        "imports": {
          "detected": ["os", "json"],
          "unavailable": [],
          "suggestions": {}
        },
        "style": {
          "violations": [
            {
              "line": 3,
              "rule": "E501",
              "message": "Line too long (89 > 79 characters)",
              "severity": "ERROR"
            }
          ],
          "error_count": 1,
          "warning_count": 0
        },
        "overall_pass": false
      },
      "fix_suggestions": [
        "Line 3: Break long string into multiple lines or use parentheses for implicit continuation"
      ]
    }
  ],
  "summary": {
    "total_blocks": 1,
    "passed": 0,
    "failed": 1,
    "failure_reasons": ["Block 0: 1 style error"]
  }
}

## Constraints
- Do not modify the code content. Report violations and suggest fixes, but never rewrite the code.
- If no Python blocks are found, return an empty `blocks` array with `summary.total_blocks: 0`.
- If [RISK_LEVEL] is "high", flag any use of `eval()`, `exec()`, `__import__()`, or `subprocess` as a CRITICAL finding and set `overall_pass` to false regardless of other checks.
- If [REQUIRE_HUMAN_REVIEW] is true, add a `human_review_required` boolean to each block's validation object and set it to true if any check fails.

To adapt this template, replace the placeholders with your project-specific values. [TARGET_PYTHON_VERSION] should be a string like "3.11". [ALLOWED_IMPORTS] should be a JSON array of module names, or an empty array [] to skip import checking. [STYLE_CONFIG] should specify the linter and ruleset, such as "flake8 with max-line-length=88" or "ruff with pyproject.toml defaults". [STYLE_SEVERITY_THRESHOLD] controls which violations block the overall_pass flag—set it to "ERROR" to only block on errors, or "WARNING" to block on warnings too. [RISK_LEVEL] accepts "low", "medium", or "high" and gates the dangerous-call detection. [REQUIRE_HUMAN_REVIEW] is a boolean that adds a review gate to the output schema. Always test this prompt against known-good and known-bad Python blocks before deploying it in a file-writing pipeline.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Extract and Validate Python Code Blocks prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to check the input before execution.

PlaceholderPurposeExampleValidation Notes

[RAW_RESPONSE]

The full model response text containing Python code blocks to extract and validate

Here is the script:

python
print('hello')

Must be a non-empty string. Check for null or zero-length input before processing. If the response is streaming, buffer until fence closure is confirmed.

[TARGET_PYTHON_VERSION]

The Python version to validate syntax against

3.11

Must match a valid Python version string (e.g., 3.9, 3.10, 3.11, 3.12). Reject unrecognized or unsupported versions. Use sys.version_info for runtime comparison.

[REQUIRED_IMPORTS]

List of import names that must be available in the target environment

["requests", "pandas", "numpy"]

Each entry must be a valid Python package name. Validate against a known allowlist or pip index check. Empty list is allowed but must be explicit.

[STYLE_GUIDE]

The linting standard to apply (e.g., pep8, black, ruff config)

pep8

Must be one of the supported style guide identifiers. Reject unknown values. If null, skip style validation and note in output metadata.

[MAX_BLOCK_COUNT]

Maximum number of Python code blocks to extract from the response

5

Must be a positive integer. Use to prevent resource exhaustion on responses with excessive code blocks. Default to 10 if not specified.

[ALLOW_UNTAGGED]

Whether to extract code blocks that lack a python language tag

Must be true or false. When true, apply content-based language detection as a fallback and flag confidence in output. When false, skip untagged blocks entirely.

[INSERTION_CONTEXT]

Optional file path or module name where the extracted code will be inserted

src/utils/helpers.py

If provided, validate path does not contain traversal sequences (../). If null, skip insertion marker validation. Used to check import consistency with target location.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Extract and Validate Python Code Blocks prompt into an application with validation, retries, and safe file operations.

This prompt template is designed to be called as a post-processing step after a primary model response that contains Python code blocks. The typical integration pattern is a two-stage pipeline: the first stage generates or retrieves a response containing code, and the second stage passes that response through this extraction and validation prompt. The harness should treat the extraction prompt as a structured output call—request JSON mode or a tool-call response with a defined schema containing blocks, validation_results, and lint_results arrays. This ensures the output is machine-readable without additional parsing.

The implementation must include pre-validation of the input before calling the prompt. Check that the source response is a non-empty string and that it contains at least one fenced code block marker (```). If no code blocks are detected, skip the extraction call and log a NO_CODE_BLOCKS event. After receiving the structured output, run AST parse validation on each extracted block using Python's ast.parse() in a subprocess or sandboxed environment. This catches syntax errors the model may have missed. For import validation, maintain an allowlist of permitted standard library and project-specific modules; any import outside this list should flag the block for human review. The harness should also execute a configured linter (such as ruff or flake8) on each block and compare the model's reported lint results against the actual linter output to detect hallucinated diagnostics.

Retry logic should be conservative. If the extraction prompt returns malformed JSON or fails schema validation, retry once with a simplified instruction that asks only for the code block array without validation commentary. If AST parsing fails on a block, do not retry the extraction prompt—instead, pass the block to a separate repair prompt or surface it in a review queue. For file insertion workflows, always write extracted blocks to a temporary staging directory first, run all validations, and only move to the target path after all checks pass. Never write directly to the target file system from model output. Log every extraction attempt with the source response hash, extracted block count, validation pass/fail status, and any human-review flags. This audit trail is essential for debugging extraction failures in production and for demonstrating due diligence in regulated environments.

IMPLEMENTATION TABLE

Expected Output Contract

Fields, format, and validation rules for the JSON object returned by the Extract and Validate Python Code Blocks prompt. Use this contract to build a parser, validator, or test harness.

Field or ElementType or FormatRequiredValidation Rule

extracted_blocks

Array of objects

Must be an array. Can be empty if no valid Python blocks are found. Schema check: validate each element against the block object contract.

extracted_blocks[].block_id

String

Must be a unique identifier within the response, formatted as 'block-<index>' where index is a zero-padded integer matching the extraction order. Regex: ^block-\d{3}$.

extracted_blocks[].language_tag

String

Must be 'python' or a recognized alias like 'py', 'python3'. Normalize to lowercase 'python' before validation. Enum check against a controlled list of Python aliases.

extracted_blocks[].raw_content

String

The exact code string extracted from the fenced block. Must not be null or empty. Leading/trailing whitespace should be preserved but checked for excessive blank lines.

extracted_blocks[].ast_valid

Boolean

Must be true if the raw_content was successfully parsed by Python's ast.parse() without a SyntaxError. False otherwise. No null allowed.

extracted_blocks[].imports

Array of strings

A list of top-level module names parsed from import statements. Must be an array, can be empty. Each string must match the pattern for a valid Python module name (e.g., 'os', 'numpy').

extracted_blocks[].lint_results

Array of objects or null

Must be an array of lint violation objects if a linter was run, or null if linting was skipped or unavailable. If an array, each object must have 'code' (string) and 'message' (string) fields.

extraction_metadata

Object

Must be a non-null object containing summary information about the extraction process. Schema check: validate presence of total_blocks_found and valid_python_blocks fields.

extraction_metadata.total_blocks_found

Integer

The total number of fenced code blocks found in the input, regardless of language. Must be >= 0 and >= the length of extracted_blocks. Parse check: integer comparison.

extraction_metadata.valid_python_blocks

Integer

The count of blocks in extracted_blocks where ast_valid is true. Must be <= the length of extracted_blocks. Parse check: integer comparison.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when extracting and validating Python code blocks, and how to guard against it before the code reaches a file or executor.

01

Missing or Incorrect Language Tag

What to watch: The model outputs a fenced block without a language tag, uses py instead of python, or includes extra characters like python3. Downstream parsers that filter by exact tag miss valid Python code. Guardrail: Normalize language tags with a mapping of aliases (py, python3, python) to a canonical form before filtering. Log any block that fails tag resolution for manual review.

02

Broken or Asymmetric Fences

What to watch: The model opens a fenced block with ```python but closes with ``` or omits the closing fence entirely. This corrupts extraction, causing content from subsequent blocks or prose to be captured as code. Guardrail: Implement a fence-pairing state machine that tracks open fence depth and reports unclosed blocks. Reject any extracted block where the closing fence is missing or the fence count is odd.

03

Inline Code Misclassified as Fenced Block

What to watch: Single-backtick inline code like print('hello') is incorrectly extracted as a fenced block, producing tiny, non-runnable fragments that pollute the output. Guardrail: Enforce a minimum content length and newline presence for fenced block extraction. Classify single-line, single-backtick spans as inline code and exclude them from file-bound outputs.

04

AST Parse Failure on Valid-Looking Code

What to watch: The extracted code looks correct but fails ast.parse() due to syntax errors (missing colons, indentation mismatches, or Python 2/3 incompatibilities). The block passes fence extraction but fails validation. Guardrail: Run ast.parse() on every extracted block and capture the specific SyntaxError message and line number. Return the error alongside the block so the caller can surface it for repair or human review.

05

Unresolvable Imports in Extracted Code

What to watch: The code imports pandas, requests, or a custom internal module that is not available in the target environment. The block passes syntax checks but will fail at runtime. Guardrail: Use ast to walk the tree and collect all top-level import names. Cross-reference against an allowlist of available packages. Flag unknown imports with a severity level and suggest a pip install command or manual review.

06

Style Violations Masking Logic Errors

What to watch: The code passes syntax and import checks but contains style issues (unused variables, bare excepts, or line-length violations) that obscure real bugs. Teams that skip linting ship messy, hard-to-maintain code. Guardrail: Run a configurable linter (e.g., ruff or flake8) on the extracted block and attach the lint results as structured metadata. Set a severity threshold above which the block is flagged for human review before insertion.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the Extract and Validate Python Code Blocks prompt before shipping. Each criterion targets a specific failure mode observed in production code extraction pipelines.

CriterionPass StandardFailure SignalTest Method

Syntax Validity

All extracted blocks parse successfully with ast.parse()

SyntaxError raised on any extracted block

Run ast.parse(block) on every extracted code block in the test set

Import Availability

All top-level imports resolve in the target Python environment

ModuleNotFoundError for any import statement

Execute python -c "import <module>" for each unique import across extracted blocks

Fence Completeness

Every opening fence has a matching closing fence; no orphaned blocks

Unclosed fence detected; content after final fence is misclassified as code

Count opening and closing fences per response; verify parity and ordering

Language Tag Accuracy

All fenced blocks tagged as python, py, or python3 are extracted; non-Python blocks are excluded

JavaScript or shell block extracted as Python; Python block missed due to missing or malformed tag

Run extraction on mixed-language responses; measure precision and recall against ground-truth Python blocks

Style Compliance

Extracted code passes flake8 or ruff with project-configured rules

Linter violations exceed threshold; style drift from project conventions

Run configured linter on each extracted block; count violations per block

Inline Code Exclusion

Inline backtick spans are not extracted as code blocks

Inline print('hello') extracted as a fenced block

Include prose with inline code in test inputs; verify zero inline spans in extraction output

AST Completeness

Extracted blocks produce complete AST nodes (not partial statements)

ast.parse succeeds but produces incomplete or empty Module body

Parse each block and assert len(ast.parse(block).body) > 0

Fix Suggestion Quality

Lint fix suggestions are syntactically valid and address the specific violation

Fix suggestion introduces new syntax error or does not resolve the flagged violation

Apply fix suggestion to original block; re-run linter and verify violation count decreases

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single [PYTHON_CODE_INPUT] placeholder. Focus on AST parse success and import availability. Skip style linting and complexity checks. Accept a simple JSON array of validated blocks with status, block, and errors fields.

Prompt snippet

code
Extract all Python code blocks from [PYTHON_CODE_INPUT]. For each block, validate syntax using ast.parse and check that all imports are available in a standard Python 3.11 environment. Return a JSON array with fields: block (string), status (valid|invalid), errors (string[]).

Watch for

  • Missing language tags causing false negatives
  • ast.parse failing on incomplete snippets without recovery
  • Import checks flagging standard library modules as unavailable
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.