Inferensys

Prompt

Async Tool Call Cancellation and Cleanup Prompt Template

A practical prompt playbook for using Async Tool Call Cancellation and Cleanup Prompt Template in production AI workflows.
Operations team reviewing AI workflow automation on laptop, workflow builder visible, casual office setup.
PROMPT PLAYBOOK

When to Use This Prompt

Defines the job-to-be-done, ideal user profile, required context, and explicit boundaries for the Async Tool Call Cancellation and Cleanup prompt template.

This prompt is for agent developers who need to implement graceful cancellation of in-flight asynchronous tool calls. It is designed for systems where a user or upstream process requests cancellation of a long-running operation, and the agent must propagate that signal, handle any partial results, verify resource cleanup, and notify the user of what was completed and what was abandoned. Use this prompt when your agent orchestrates tools that support cancellation tokens, abort signals, or equivalent mechanisms. The prompt assumes the agent has access to tool-specific cancellation functions and a log of in-flight call metadata including correlation IDs, start times, and resource handles.

Do not use this prompt for synchronous tool calls that complete within a single request-response cycle; cancellation there is a transport-layer concern, not an agent reasoning task. This prompt is also inappropriate for fire-and-forget async calls where the agent has no ability to propagate a cancellation signal or verify cleanup. Before using this template, confirm that your tool contracts expose cancellation endpoints, that your agent runtime can track in-flight calls with correlation IDs, and that you have defined what 'partial completion' means for each tool in your ecosystem. Without these prerequisites, the agent will hallucinate cancellation confirmations without actually stopping work.

The ideal user is an agent infrastructure developer or platform engineer building a multi-step agent that invokes tools with variable latency—database migrations, report generation, external API workflows, or model inference jobs. The prompt expects structured input: a list of in-flight tool calls with their correlation IDs, tool names, start times, resource handles, and current status. It produces a structured cancellation plan, per-tool cleanup verification steps, a partial-results summary, and a user-facing status message. Wire this prompt behind a cancellation command handler in your agent loop, not as a standalone chat interaction.

Avoid using this prompt when the cancellation request itself is ambiguous—for example, when a user says 'stop' but might mean 'pause' or 'go back a step.' In those cases, use a clarification prompt before invoking cancellation logic. Also avoid this prompt when the agent lacks authority to release resources; if cleanup requires human approval, the prompt should escalate rather than attempt autonomous resource deallocation. The next section provides the copy-ready template with placeholders you'll adapt to your specific tool contracts and cancellation primitives.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Async Tool Call Cancellation and Cleanup Prompt Template works and where it does not.

01

Good Fit: Long-Running Agent Workflows

Use when: your agent invokes tools that may run for seconds or minutes (e.g., data pipeline triggers, report generation, batch processing) and the user or system may need to cancel mid-flight. Guardrail: ensure the prompt explicitly instructs the agent to propagate cancellation signals to the tool layer, not just stop its own reasoning.

02

Good Fit: User-Interruptible Copilots

Use when: a user-facing agent needs to handle 'stop' or 'cancel' commands gracefully, clean up in-progress work, and report what was partially completed. Guardrail: include a structured output schema for the cancellation summary so the UI can render partial results and cleanup status consistently.

03

Bad Fit: Synchronous-Only Tool Chains

Avoid when: all tool calls are synchronous and complete in under a second. Adding cancellation logic to synchronous calls adds complexity without benefit. Guardrail: profile your tool call latency before introducing async cancellation patterns; if p99 latency is under 500ms, keep the prompt simple.

04

Required Inputs

Must provide: the list of in-flight tool call IDs, their current statuses, and any partial results already received. Guardrail: if your system cannot reliably track in-flight call state, the prompt cannot make accurate cleanup decisions—build that tracking first.

05

Operational Risk: Orphaned Resources

Risk: the agent cancels its own reasoning but fails to send cancellation to the external tool, leaving resources running and accruing cost. Guardrail: include explicit verification steps in the prompt that require the agent to confirm each tool received and acknowledged the cancellation signal.

06

Operational Risk: Incomplete Side-Effect Reversal

Risk: a tool partially completed a multi-step operation (e.g., wrote some records but not all) and cancellation leaves the system in an inconsistent state. Guardrail: the prompt should instruct the agent to identify partial side effects and either request a compensating action or escalate for human review.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable system prompt for agents that must gracefully cancel in-flight async tool calls, verify resource cleanup, and communicate partial results to users.

This prompt template gives your agent explicit instructions for handling cancellation requests while async tool calls are still in progress. It covers three critical phases: propagating the cancellation signal to the external tool or service, assessing whether partial results are salvageable, and verifying that no orphaned resources remain. Paste this into your agent's system prompt or cancellation-handler instruction block, then replace the square-bracket placeholders with your actual tool names, resource types, and output schema.

text
You are an agent that executes tool calls asynchronously. When a user or upstream system requests cancellation of an in-flight operation, follow these rules precisely.

## Cancellation Signal Propagation
1. Immediately send a cancellation request to every in-flight tool call using the appropriate cancellation mechanism for each tool.
2. For tools that accept cancellation tokens: pass [CANCELLATION_TOKEN] in the cancellation request.
3. For tools that require an explicit undo or rollback call: invoke the [ROLLBACK_TOOL] with the original [REQUEST_ID] and [TOOL_CALL_ID].
4. For tools with no native cancellation: log the [TOOL_NAME] and [TOOL_CALL_ID] as UNRECOVERABLE and include them in the user-facing summary.

## In-Progress Result Handling
5. If a tool returns partial results before acknowledging cancellation, evaluate whether those results are safe to present:
   - If the partial result is a complete, valid subset of the expected output, include it in the final response with a PARTIAL_RESULT marker.
   - If the partial result is incomplete or potentially misleading, discard it and mark the tool output as CANCELLED_INCOMPLETE.
   - If the partial result represents a side effect that cannot be reversed, flag it as ORPHANED_SIDE_EFFECT and include the [TOOL_NAME], [TOOL_CALL_ID], and a description of the unreversed action.

## Resource Cleanup Verification
6. After all cancellation signals are sent, verify cleanup by checking the following resource types:
   - [RESOURCE_TYPE_1]: check using [VERIFICATION_TOOL_1] with [VERIFICATION_PARAMETERS].
   - [RESOURCE_TYPE_2]: check using [VERIFICATION_TOOL_2] with [VERIFICATION_PARAMETERS].
7. If any resource remains allocated after cancellation, retry cleanup once using [CLEANUP_TOOL]. If the retry fails, log the resource as ORPHANED and include it in the user-facing summary.

## User Notification
8. Produce a final cancellation summary in the following [OUTPUT_SCHEMA]:
   {
     "status": "CANCELLED" | "PARTIALLY_CANCELLED" | "CANCELLATION_FAILED",
     "cancelled_tool_calls": [
       {
         "tool_call_id": "string",
         "tool_name": "string",
         "cancellation_status": "CANCELLED" | "CANCELLED_INCOMPLETE" | "UNRECOVERABLE",
         "partial_result": { } | null,
         "orphaned_resources": ["string"] | []
       }
     ],
     "orphaned_side_effects": [
       {
         "tool_name": "string",
         "tool_call_id": "string",
         "description": "string"
       }
     ],
     "user_message": "string"
   }
9. The user_message field must clearly state what was cancelled, what partial work was saved, and what could not be reversed. Do not claim successful cleanup if any resource or side effect remains orphaned.

## Constraints
- Never ignore a cancellation request to let a tool call complete.
- Do not fabricate cleanup confirmation. Only report resources as cleaned if verification confirms it.
- If [RISK_LEVEL] is HIGH, escalate orphaned resources to [ESCALATION_CHANNEL] immediately after producing the summary.

To adapt this template, replace the bracketed placeholders with your actual tool names, resource types, verification procedures, and escalation channels. If your system doesn't support native cancellation tokens, remove that branch and expand the rollback or logging path. For low-risk workflows, you can simplify the output schema by removing the orphaned_side_effects array, but always preserve the distinction between confirmed cleanup and unverified claims. Test the adapted prompt against the eval checks described in the full playbook: orphaned resource detection, incomplete side-effect reversal, and misleading user messages that claim cleanup when resources remain allocated.

IMPLEMENTATION TABLE

Prompt Variables

Inputs the prompt needs to work reliably. Provide these at runtime from your agent's orchestration layer.

PlaceholderPurposeExampleValidation Notes

[TOOL_CALL_ID]

Unique identifier for the in-flight async tool call to cancel

call_4a7b9c2d

Must match the ID from the original invocation. Validate as non-empty string. Null not allowed.

[TOOL_NAME]

Human-readable name of the tool being cancelled

browser_navigate

Must match a registered tool name in the agent's tool registry. Validate against active tool manifest.

[CANCELLATION_REASON]

Why cancellation was triggered (user request, timeout, policy violation, etc.)

user_requested_stop

Must be one of the enumerated cancellation reason codes defined in the agent's policy config. Validate against allowed enum values.

[ELAPSED_TIME_MS]

Milliseconds since the tool call was initiated

45200

Must be a non-negative integer. Validate as number type. Used to decide whether partial results are salvageable.

[PARTIAL_RESULT_SNAPSHOT]

Any intermediate output received before cancellation signal was sent

{"status": "in_progress", "pages_loaded": 3}

Can be null if no partial results arrived. If present, validate against the tool's partial-result schema. Null allowed.

[RESOURCE_MANIFEST]

List of resources (handles, sessions, locks) the tool acquired that need cleanup

["browser_session_12", "temp_file_/tmp/agt_7z"]

Must be a JSON array of resource identifiers. Validate array structure. Empty array allowed if no resources were acquired.

[CLEANUP_CONFIRMATION_REQUIRED]

Boolean flag indicating whether the agent must wait for explicit cleanup confirmation before proceeding

Must be a boolean. If true, the prompt must produce a verification step before the agent continues. Validate as strict boolean type.

[USER_NOTIFICATION_CONTEXT]

What the user was told or should be told about the cancellation

Task cancelled. 3 of 10 items were processed.

Must be a non-empty string if the cancellation is user-facing. Validate string length > 0 when user_notification_required is true. Null allowed for silent cancellations.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the cancellation prompt into an agent runtime with validation, retries, and cleanup verification.

This prompt is designed to sit inside a cancellation handler—a code path triggered when a user interrupts a long-running operation, a timeout fires, or a parent workflow aborts. The handler should inject the prompt into a fresh model call, passing the list of in-flight tool calls, their current states, and any partial results already received. The model's job is to produce a structured cancellation plan, not to execute the cancellation itself. Your application code reads that plan and performs the actual tool-call abort, resource cleanup, and user notification.

The implementation loop follows a strict plan-then-execute pattern. First, gather all active tool_call_id values, their tool names, start times, and any partial payloads. Populate the [ACTIVE_TOOL_CALLS] placeholder with a JSON array of objects containing id, tool, status (one of in_flight, partial_result_received, awaiting_callback), started_at, and partial_result. Set [CANCELLATION_REASON] to the trigger cause (user_interrupt, timeout, parent_abort, circuit_breaker). Call the model with the prompt template and parse the output against a strict schema: an array of actions, each with tool_call_id, action (abort, await_partial, rollback, notify, noop), cleanup_steps (array of strings describing resource release actions), and user_message (a short, honest summary for the end user). Validate that every active tool call appears in the output exactly once. If validation fails, retry once with the validation error appended to [CONSTRAINTS]. After two failures, fall back to a hard-coded abort-all strategy and log the incident.

After parsing the model's cancellation plan, execute actions sequentially in dependency order: abort in-flight calls first, then process any partial results marked for salvage, then run cleanup steps, and finally surface user messages. Log every step with the tool_call_id, action taken, timestamp, and success/failure status. This trace is critical for debugging orphaned resources. Implement a cleanup verification gate: after executing all cleanup steps, wait 2–5 seconds and re-query the tool systems to confirm resources (locks, sessions, file handles, database transactions) were actually released. If verification fails, escalate to an on-call channel rather than silently proceeding. For high-risk tools (database writes, payment operations, external API mutations), require a human approval step before executing rollback actions—the model's plan is advisory, not authoritative. Never allow the model to directly invoke cancellation or rollback APIs; all execution must happen in your application code with audit trails intact.

IMPLEMENTATION TABLE

Expected Output Contract

Validation rules for the JSON object the model must return when handling async tool call cancellation and cleanup. Use this contract to parse, validate, and route the model's response before executing any downstream cleanup actions.

Field or ElementType or FormatRequiredValidation Rule

cancellation_signal_propagated

boolean

Must be true if cancellation was successfully communicated to the tool provider. False only if propagation failed after all retries.

in_flight_result_disposition

enum: discard | save_partial | await_completion

Must match one of the three allowed values. save_partial requires non-null partial_result_summary. await_completion requires non-null estimated_remaining_seconds.

partial_result_summary

string or null

Required when in_flight_result_disposition is save_partial. Must be a concise summary of what was completed before cancellation. Max 500 characters.

estimated_remaining_seconds

integer or null

Required when in_flight_result_disposition is await_completion. Must be a positive integer. Null otherwise.

resources_cleaned_up

array of strings

Each string must name a resource that was released (e.g., 'db_connection_pool', 'file_handle', 'api_session'). Array may be empty if no resources were held.

orphaned_resource_detected

boolean

Must be true if the agent identifies any resource that could not be confirmed as released. Triggers a human-review escalation in the harness.

side_effect_reversal_attempted

boolean

Must be true if the agent attempted to reverse any partial side effects (e.g., deleting a draft record, rolling back a transaction). False if no reversible side effects existed.

user_notification_text

string

A human-readable message explaining what was cancelled, what was saved, and what requires attention. Must not exceed 300 characters. Must not contain unresolved placeholders.

PRACTICAL GUARDRAILS

Common Failure Modes

Async tool call cancellation is a distributed systems problem, not just a prompt problem. These failure modes surface when cancellation signals, in-flight state, and cleanup logic interact under real production conditions.

01

Orphaned Side Effects After Cancellation

What to watch: The agent sends a cancellation signal but the external tool has already committed a partial write (database insert, payment authorization, file creation) before receiving the abort. The agent assumes clean cancellation and proceeds, leaving orphaned state that downstream workflows don't expect. Guardrail: Require the prompt to request a cancellation_receipt from the tool that explicitly lists completed vs. rolled-back side effects. If the receipt is missing or ambiguous, the agent must escalate rather than assume cleanup succeeded.

02

Cancellation Signal Lost in Transit

What to watch: The agent emits a cancellation event but the async transport (message queue, webhook, event bus) drops, delays, or misroutes it. The tool continues executing, eventually returning a result that the agent has already moved past, causing stale state injection or duplicate work. Guardrail: Include a cancellation_correlation_id in every async tool call and require the agent to reject any tool result whose correlation ID matches a previously cancelled request. Log mismatches as operational anomalies.

03

Partial Result Salvage Without Integrity Check

What to watch: The agent receives partial streaming results before cancellation and attempts to salvage them for a degraded response. The partial data is internally inconsistent (e.g., summary totals don't match line items) because the tool was mid-computation. The agent presents corrupted data as a valid partial answer. Guardrail: The prompt must instruct the agent to run a lightweight integrity check on any salvaged partial result before surfacing it. If the check fails, the agent must discard the partial data and report only what can be verified.

04

Resource Leak from Unclosed Connections

What to watch: The agent cancels a long-running tool call but the underlying connection (WebSocket, SSE stream, database cursor, file handle) remains open on the tool server side. Over multiple cancellations, resource exhaustion causes the tool to degrade or fail for subsequent requests. Guardrail: The cancellation prompt must include an explicit cleanup_sequence step: close connection handles, drain buffers, and request a resource_released confirmation. Add an eval that checks for connection leaks by monitoring open handles before and after cancellation tests.

05

User-Facing Progress Claims Diverge from Reality

What to watch: The agent reports "cancelling..." to the user based on its own state transition, but the tool hasn't acknowledged the cancellation yet. The user assumes the operation stopped, but the tool continues executing silently. This creates a trust gap and potential data surprises. Guardrail: The prompt must require the agent to report progress based on tool-acknowledged state, not internal intent. Use a status_source field that distinguishes "agent_requested_cancel" from "tool_confirmed_cancel" in user-facing messages.

06

Retry Storm After Cancellation Timeout

What to watch: The agent cancels a tool call, doesn't receive confirmation within the timeout window, and retries the cancellation—repeatedly. Each retry creates a new cancellation request on an already-overloaded tool server, amplifying the problem. Guardrail: Implement exponential backoff with a maximum cancellation retry count (typically 3). After the final retry, the agent must stop attempting cancellation, log the unconfirmed state, and escalate for manual intervention rather than continuing to hammer the tool.

IMPLEMENTATION TABLE

Evaluation Rubric

Run these checks against a golden dataset of cancellation scenarios to verify the prompt handles cleanup, state, and user communication correctly before shipping.

CriterionPass StandardFailure SignalTest Method

Cancellation Signal Propagation

All in-flight [TOOL_CALL_ID] values are included in the cancellation request payload

Cancellation request omits one or more active tool IDs; orphaned calls continue executing

Inject 3 concurrent tool calls, issue cancellation, and assert the cancellation payload contains exactly 3 IDs

In-Progress Result Handling

Partial results from cancelled calls are captured with status cancelled and any intermediate data preserved in [PARTIAL_RESULTS]

Partial results are discarded silently or returned with status success despite cancellation

Provide a tool that streams 50% of results before cancellation; verify output includes partial data and correct status

Resource Cleanup Verification

All acquired resources (locks, sessions, temp storage) are released within [CLEANUP_TIMEOUT_MS] of cancellation

Resource leak detected: lock held beyond timeout, temp file not deleted, or session remains active after 2x cleanup timeout

Monitor resource state before and after cancellation; assert all resources return to pre-call state within timeout window

User Notification of Partial Completion

Output includes a [USER_NOTICE] field explaining what completed, what was cancelled, and whether retry is safe

User receives generic error with no indication of partial progress or safe retry path

Parse output for required notice fields; assert presence of completed-items list, cancelled-items list, and retry-safety boolean

Side-Effect Reversal Completeness

All reversible side effects (writes, state changes) from cancelled calls are rolled back or flagged as reversal-pending with evidence

Side effect remains applied with no rollback attempt and no pending-reversal flag

Execute a tool with a database write, cancel mid-operation, and query the database to confirm rollback or pending status with evidence

Idempotency Key Preservation

Cancellation response retains the original [IDEMPOTENCY_KEY] for each cancelled call to prevent duplicate retry execution

Idempotency key is missing from cancellation response; retry would create duplicate side effects

Verify cancellation output includes idempotency key per cancelled call; attempt retry with same key and assert rejection

Cancellation Reason Attribution

Output includes a [CANCELLATION_REASON] field sourced from the triggering event (user request, timeout, dependency failure)

Cancellation reason is null, generic, or misattributed to the wrong trigger source

Inject cancellation from user request, timeout, and dependency failure in separate test cases; assert reason field matches trigger source in each

Orphaned Call Detection

Any tool call still running beyond [ORPHAN_THRESHOLD_MS] after cancellation is flagged in [ORPHANED_CALLS] array

Orphaned call completes silently with no detection; agent proceeds as if all calls terminated

Delay one tool response beyond orphan threshold after cancellation; assert orphaned call appears in output with tool ID and elapsed time

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base cancellation template but relax strict resource tracking. Use a simplified state model: PENDING | IN_FLIGHT | CANCELLING | CANCELLED | FAILED. Focus on getting the cancellation signal propagation right before adding cleanup verification.

code
[TOOL_CALL_ID]: [ID]
[TOOL_STATE]: [STATE]
[CANCELLATION_REASON]: [REASON]

You are handling a cancellation request for tool call [TOOL_CALL_ID].
Current state: [TOOL_STATE]
Reason: [CANCELLATION_REASON]

Determine the appropriate action:
- If IN_FLIGHT: signal cancellation, wait for acknowledgement
- If PENDING: mark cancelled immediately
- If already terminal: report current state

Return: { "action": "signal_cancel" | "mark_cancelled" | "report_state", "message": "..." }

Watch for

  • Missing correlation between cancellation request and tool call ID
  • No timeout on waiting for in-flight acknowledgement
  • Orphaned resources when prototype moves to production
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.