Inferensys

Prompt

Dynamic Format for Agent Tool Response Prompt Template

A practical prompt playbook for AI agent builders whose tools must return results in formats consumable by different downstream agents or systems, with eval for tool-contract compliance per format.
Developer demonstrating multi-agent tool use, agent tool selection interface on laptop, casual tech demo moment.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for dynamic agent tool response formatting.

This prompt is for AI agent builders whose tools must return results consumable by different downstream systems or audiences. The core job-to-be-done is producing a single tool response that adapts its format based on a response_target parameter: structured JSON for tool chaining and API consumption, Markdown for human-in-the-loop review dashboards, or raw text for logging and audit trails. The ideal user is a platform engineer or agent developer wiring tool outputs into a multi-agent architecture where the consumer of a tool's result is not known at design time. You need this prompt when a single tool serves multiple callers with incompatible format expectations, and you want the model to enforce the format contract rather than relying on post-hoc transformation layers that can drop fields or misinterpret structure.

Do not use this prompt when the output format is fixed and known at build time. If every consumer expects the same JSON schema, use a schema-first prompt with strict field definitions instead. Do not use it when the tool response is always consumed by a human—skip the format switch and optimize for readability. Avoid this pattern when the format decision depends on complex business logic better handled in application code; the prompt works best when the format selector is a simple enum passed as a parameter. For high-stakes financial, healthcare, or compliance workflows, always add a human review step before the formatted output is consumed by downstream systems, regardless of the selected format.

Before implementing, confirm you have a clear response_target taxonomy with exactly the values your system supports—typically json, markdown, and text. Map each target to a concrete output contract: JSON requires a specific schema with typed fields, Markdown requires section headings and readable structure, and text requires a flat string suitable for log ingestion. Wire the prompt into your agent harness so the response_target value is injected from the calling agent's context or configuration, never from untrusted user input. After deployment, evaluate format compliance per target: JSON must parse without errors and match the declared schema, Markdown must render correctly in your review UI, and text must contain no unescaped control characters that could break log pipelines.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works and where it does not. Use these cards to decide if the Dynamic Format for Agent Tool Response template fits your architecture before you integrate it.

01

Good Fit: Multi-Agent Pipelines

Use when: A tool's output must be consumed by another agent for function chaining. Guardrail: The prompt enforces strict JSON contracts for machine consumption, preventing parsing failures downstream. Validate the JSON schema against the consuming agent's expected input contract before deployment.

02

Good Fit: Human-in-the-Loop Review

Use when: A tool response must be surfaced to a human operator for approval or inspection before the next step. Guardrail: The prompt switches to Markdown for readability. Ensure the human review interface renders Markdown safely and that critical data isn't lost in the format translation.

03

Bad Fit: Single-Consumer Systems

Avoid when: The tool response always goes to the same downstream system with a fixed format. Guardrail: Dynamic format selection adds unnecessary complexity and a new failure mode. Use a static, schema-locked prompt instead to reduce variability and simplify testing.

04

Required Inputs

What to watch: The prompt requires a reliable response_target parameter to select the output format. Guardrail: Validate the parameter against an allowed list of targets before injecting it into the prompt. Reject unknown targets early with a clear error to prevent the model from guessing a format.

05

Operational Risk: Format Drift

What to watch: The model may produce valid Markdown that is missing structured data fields required by a downstream system if the target is misconfigured. Guardrail: Implement post-generation validation that checks for the presence of required fields regardless of the output format. Log any missing fields as a format-contract violation.

06

Operational Risk: Tool Contract Violation

What to watch: An agent expecting a JSON object may receive a Markdown string if the response_target is set incorrectly, causing a runtime error. Guardrail: The agent calling the tool should validate the Content-Type of the response before parsing. If the type is unexpected, the agent should log the mismatch and execute a fallback or escalation path.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt that selects and enforces the correct output format based on a response target parameter.

This prompt template is the core instruction set for an agent tool that must return results in a format consumable by different downstream systems. It accepts a [RESPONSE_TARGET] parameter that dictates whether the output should be structured JSON for tool chaining, Markdown for human-in-the-loop review, or raw text for logging. The template enforces strict format contracts per target, ensuring that a JSON response is always a valid, parseable object and a Markdown response always includes clear section headers. Copy the block below and replace the square-bracket placeholders with your specific tool's input data, context, and constraints.

text
You are a tool response formatter for an AI agent system. Your job is to return the result of a tool execution in the exact format specified by the [RESPONSE_TARGET] parameter. Do not add commentary, explanations, or apologies outside the specified format.

## INPUT DATA
Tool Name: [TOOL_NAME]
Tool Result: [TOOL_RESULT]
Execution Status: [STATUS]
Error Details (if any): [ERROR_DETAILS]

## RESPONSE TARGET
[RESPONSE_TARGET]

## FORMAT RULES
You must follow the format rules for the selected [RESPONSE_TARGET] exactly.

### If [RESPONSE_TARGET] is "json"
Return a single valid JSON object with no markdown fences, no trailing commas, and no comments. The object must conform to this schema:
{
  "tool_name": "string",
  "status": "success" | "error" | "partial",
  "data": <the complete tool result, preserving its original structure>,
  "error": null | {"code": "string", "message": "string"},
  "metadata": {
    "format": "json",
    "generated_at": "ISO-8601 timestamp"
  }
}

### If [RESPONSE_TARGET] is "markdown"
Return a Markdown-formatted response suitable for human review. Use the following structure:
# Tool Result: [TOOL_NAME]
**Status:** [STATUS]
**Generated:** [current ISO-8601 timestamp]

## Result
[TOOL_RESULT formatted as readable markdown with appropriate headers, lists, and emphasis]

## Errors
[ERROR_DETAILS if status is error or partial, otherwise "None"]

### If [RESPONSE_TARGET] is "text"
Return a plain text log entry with no markdown formatting. Use this exact structure:
[ISO-8601 timestamp] [TOOL_NAME] [STATUS]
Result: [TOOL_RESULT as compact text]
Error: [ERROR_DETAILS or "None"]

## CONSTRAINTS
- Do not wrap the output in markdown code fences unless the target format itself requires them.
- If [RESPONSE_TARGET] is not one of "json", "markdown", or "text", default to "json" and set status to "error" with an appropriate error message.
- Never invent or modify the tool result data; preserve it exactly.
- If the tool result contains sensitive-looking data, do not redact it; the caller is responsible for data hygiene.

To adapt this template, replace the placeholders with your actual tool context. The [TOOL_RESULT] placeholder should receive the raw output from your tool execution step. If your agent framework passes structured objects, serialize them to a string representation before insertion. The [RESPONSE_TARGET] parameter should be set by the calling agent or orchestrator based on the downstream consumer: use "json" when the next step is another tool or API, "markdown" when a human needs to review the output in a dashboard or chat interface, and "text" when writing to a log aggregation system. For production use, add a post-processing validation step that parses the model output and confirms it matches the requested format before forwarding it to the consumer.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Dynamic Format for Agent Tool Response Prompt Template. Each placeholder must be populated before the prompt is sent to the model. Missing or malformed inputs are the most common cause of format-switching failures in production.

PlaceholderPurposeExampleValidation Notes

[TOOL_OUTPUT]

Raw unstructured or semi-structured data returned by the agent tool before formatting

{"status": "ok", "rows_affected": 42, "warnings": ["truncated"]}

Must be non-null. Accepts JSON object, plain text, or key-value pairs. Reject empty string. If tool returned error object, route to error-formatting path instead.

[RESPONSE_TARGET]

Enum flag selecting the output format based on downstream consumer

json_api | markdown_review | raw_log

Must match one of the allowed enum values exactly. Reject unknown targets with a clear error message listing valid options. Case-sensitive.

[TOOL_NAME]

Identifier for the tool that produced the output, used for traceability and log context

database_query | file_search | api_gateway_fetch

Must be a non-empty string matching the tool registry name. Used in log format and error attribution. Validate against registered tool list if available.

[CALLER_CONTEXT]

Information about the agent or system requesting the formatted response, used for consumer-type routing

{"agent_id": "planner-v2", "capabilities": ["json_parser"]}

Optional object. If provided, must contain agent_id string. Used to override format selection when consumer capabilities are known. Null allowed.

[HUMAN_REVIEW_FLAG]

Boolean indicating whether a human will review the output, influencing verbosity and annotation level

Must be true or false. When true, Markdown format adds explanatory annotations and confidence indicators. When false, JSON format omits review-only fields. Defaults to false if null.

[OUTPUT_SCHEMA]

JSON Schema or format-specific schema definition the output must satisfy

{"type": "object", "required": ["tool_name", "result", "format"]}

Required for json_api target. Must be a valid JSON Schema object. For markdown_review, optional structure hints. For raw_log, ignored. Validate schema parseability before prompt assembly.

[MAX_RESPONSE_SIZE]

Character or token limit for the formatted output, used to trigger truncation or summary mode

4096

Must be a positive integer. When output exceeds limit, prompt instructs model to truncate with a truncation notice field. Null means no limit. Validate as integer > 0 if provided.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Dynamic Format for Agent Tool Response prompt into a production agent tool execution loop with validation, retries, and format-specific contract checks.

This prompt is designed to sit inside an agent's tool-response handler—the code that runs after a tool executes and before the result is passed to the next agent, a human reviewer, or a logging sink. The harness must inject three runtime values into the prompt's placeholders: [TOOL_OUTPUT] (the raw result from the tool call), [RESPONSE_TARGET] (one of agent, human, or log), and [TOOL_NAME] for traceability. The model's job is to reformat the raw output into the contract specified by the target, not to modify the underlying data. Treat the model as a deterministic serializer, not a reasoning step. If the tool output already matches the target format, the model should pass it through with minimal changes—add a guard in your harness to skip the LLM call entirely when a simple format check passes, saving latency and cost.

Wire the prompt into a tool execution wrapper function. After the tool returns, call the LLM with this prompt and parse the response. The critical implementation detail is format-specific post-validation. For agent target, validate the output against your tool-chaining JSON schema—reject responses missing required fields like status, data, or error. For human target, check that the output contains no raw JSON blobs and that Markdown tables render correctly in your UI. For log target, confirm the output is a single flat string with no markdown formatting. Implement a retry loop: if validation fails, re-invoke the prompt with the validation error appended to [CONSTRAINTS]. Cap retries at 2 to avoid infinite loops on fundamentally malformed tool outputs. Log every format switch and validation failure with the [TOOL_NAME] and [RESPONSE_TARGET] for debugging.

Model choice matters here more than for most prompts. This task requires strict instruction following, not creative generation. Prefer models with strong JSON mode and format-following benchmarks. Avoid models known to inject conversational filler into structured outputs. Set temperature=0 and disable sampling parameters. If your agent framework supports it, use structured output APIs (e.g., OpenAI's response_format with a JSON Schema for the agent target) rather than relying solely on prompt instructions. For the human and log targets, plain text completion is sufficient. The biggest production risk is format bleed: the model produces a hybrid output that mixes JSON fields with Markdown explanations. Catch this in validation by checking that the response is parseable as exactly one format. If your tool outputs contain PII or sensitive data, add a redaction step before the LLM call—this prompt should never be the place where data minimization happens.

IMPLEMENTATION TABLE

Expected Output Contract

Field-level contract for the agent tool response. Each response target enforces a distinct shape. Validate against the active target before returning control to the orchestrator.

Field or ElementType or FormatRequiredValidation Rule

response_target

enum: tool | human | log

Must match one of the three allowed values. Reject unknown targets before generating the body.

tool_payload

object (JSON)

true if response_target=tool

Must parse as valid JSON. Must include 'status', 'data', and 'errors' fields. 'data' must match the tool's declared output schema.

tool_payload.status

enum: success | partial | failure

true if response_target=tool

Must be one of the three enum values. 'partial' requires at least one entry in 'errors'.

human_summary

string (Markdown)

true if response_target=human

Must contain a top-level heading, a one-sentence outcome, and a bulleted key-findings section. No raw JSON blocks unless explicitly requested.

human_approval_flag

boolean

true if response_target=human

Set to true when the tool result requires human decision before the next agent step. Must be accompanied by a clear decision prompt in the summary.

log_entry

string (plain text)

true if response_target=log

Must be a single line or structured logfmt string. Must include timestamp, tool_name, outcome, and duration_ms fields. No markdown formatting.

metadata

object

Must include 'tool_name', 'call_id', 'duration_ms', and 'format_generated'. 'format_generated' must match the active response_target.

errors

array of strings

If present, each entry must be a non-empty string describing a specific failure. Required when tool_payload.status is 'partial' or 'failure'.

PRACTICAL GUARDRAILS

Common Failure Modes

When a prompt dynamically selects an output format based on a parameter, the most dangerous failures are silent ones: the model picks the wrong format, mixes formats, or produces valid-looking output that breaks the downstream contract. These cards cover the failures that hit first in production and how to guard against them.

01

Format Parameter Ignored

What to watch: The model ignores the response_target or format parameter and defaults to its own preferred structure. A request for XML returns JSON, or a request for CSV returns Markdown. Guardrail: Place the format selector instruction at both the start and end of the system prompt. Validate the top-level structure type before parsing content, and reject with a specific error code if the format envelope doesn't match.

02

Mixed-Format Output

What to watch: The model starts in the requested format but switches mid-response. A JSON object suddenly contains Markdown blocks, or a CSV output includes a JSON error object inline. Guardrail: Post-process with a format-purity check. For JSON, confirm the entire response parses as a single valid document. For CSV, confirm every line has the same column count. Fail closed if any foreign format markers appear.

03

Tool Contract Violation Under Format Switch

What to watch: When the format changes, required fields for tool chaining disappear or get renamed. A JSON response for agent consumption drops the tool_call_id or status_code field that the orchestrator expects. Guardrail: Maintain a per-format schema registry. Validate every output against the schema for its declared format, not just the format envelope. Reject responses that pass format checks but fail field-level contracts.

04

Human-Readable Format Leaks Machine Fields

What to watch: Markdown or plain-text responses intended for human review accidentally include raw JSON metadata, internal IDs, or confidence scores that confuse the reader and clutter the UI. Guardrail: Define a strict field allowlist per consumer type. For human-targeted formats, strip or redact internal fields before rendering. Add an eval that checks for the presence of forbidden field names in human-format outputs.

05

Fallback Chain Produces Wrong Format Silently

What to watch: When the preferred format fails, the fallback logic produces a different format but doesn't signal the change. The downstream system receives JSON when it expected XML, with no indication that a fallback occurred. Guardrail: Always include a format_used and fallback_reason field in the response envelope. Require consumers to check this field before processing. Log every fallback event for monitoring.

06

Streaming Format Corruption

What to watch: For streaming responses, the format selection works for the first chunk but subsequent chunks drift. A JSON Lines stream starts correctly but later lines are malformed, or an SSE stream drops the data: prefix mid-stream. Guardrail: Validate each chunk independently for format compliance. Buffer incomplete chunks until they form a valid unit. Set a maximum chunk age and force-close the stream with an error event if validation fails repeatedly.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test whether the Dynamic Format for Agent Tool Response prompt produces valid, contract-compliant outputs before shipping. Each criterion targets a specific failure mode in multi-format agent tool responses.

CriterionPass StandardFailure SignalTest Method

Format Selection Accuracy

Output format matches the [RESPONSE_TARGET] parameter exactly: json, markdown, or text

Output is in a format not requested by [RESPONSE_TARGET] or defaults to JSON when Markdown was requested

Run 20 test cases with varied [RESPONSE_TARGET] values and verify Content-Type or format markers match the parameter

JSON Tool Contract Compliance

JSON output contains all required fields from [TOOL_OUTPUT_SCHEMA] with correct types and no extra top-level keys

Missing required field, wrong type for a field, or unexpected top-level key present in the JSON object

Validate output against [TOOL_OUTPUT_SCHEMA] using a JSON Schema validator; flag any schema violations

Markdown Human-Readability

Markdown output uses headers, lists, or tables appropriately and contains no raw JSON or machine-only tokens

Markdown output is a raw JSON dump, contains unrendered escape sequences, or is a single unformatted paragraph

Human review by 2 evaluators on 10 samples; both must rate output as readable and well-structured

Text Log Completeness

Text output includes timestamp, tool name, status, and key result fields in a parseable log format

Text output omits tool name, lacks a status indicator, or is truncated without a termination marker

Regex check for required log fields: timestamp pattern, tool_name, status, and result_summary present in every output

Cross-Format Content Equivalence

All three formats contain the same core data: result status, key findings, and any error information

JSON contains data fields that are absent from Markdown or text versions of the same tool response

Extract key fields from each format output for the same input; assert field values are identical across formats

Error Handling in All Formats

When tool returns an error, all formats include error code, message, and recovery hint in format-appropriate structure

Error output is missing in one format, or error is returned as a success with no error fields populated

Inject 10 tool error scenarios; verify error_code, error_message, and recovery_hint fields exist in JSON, Markdown, and text outputs

Empty Result Handling

When tool returns empty or null results, output explicitly signals empty state without hallucinating data

Empty result is represented as a success with fabricated data, or output is an empty string with no explanation

Test with 5 empty-result tool responses; assert output contains explicit empty indicator and no fabricated fields

Format Switch Latency Consistency

Format selection adds less than 200ms overhead compared to a single-format baseline prompt

Format switching adds more than 500ms latency or causes timeout on large payloads

Benchmark 50 requests across all three formats; measure p95 latency and compare to single-format control prompt

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single [RESPONSE_TARGET] parameter and minimal validation. Start with two formats (JSON and Markdown) before adding more. Hardcode the tool output schema inline rather than referencing external contracts.

code
RESPONSE_TARGET: [json | markdown]

Watch for

  • Missing schema checks when switching formats
  • Overly broad instructions that produce hybrid JSON-Markdown
  • No fallback when the target format is unrecognized
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.