Stale Tool Registry Detection Prompt

This prompt is designed for AI operations teams and platform engineers who need to detect when an agent's tool registry has become stale. A stale registry occurs when the tools an agent believes are available no longer match the actual capabilities, health, or schemas of the live tool servers. This mismatch causes silent failures, hallucinated tool calls, and broken multi-step workflows. Use this prompt inside a scheduled monitoring job, a pre-flight agent health check, or a CI/CD pipeline that validates tool availability before deployment. The prompt compares registry metadata (timestamps, versions, schemas) against live server signals and tool response patterns. It outputs a structured staleness report with confidence scores and recommended refresh actions.

This is not a prompt for discovering tools for the first time. It assumes a registry already exists and needs validation. Do not use this prompt when you need to bootstrap a new agent's tool set from scratch—use a registry bootstrap prompt instead. Do not use it when you need to generate capability descriptions from raw documentation; that requires a separate extraction prompt. This prompt is specifically for the operational comparison between a known registry state and live server reality. It works best when you have access to both the registry metadata (tool names, versions, last-registered timestamps, input/output schemas) and live server signals (health check responses, introspection endpoints, recent tool call success/failure patterns).

Before deploying this prompt, ensure you have collected the registry snapshot and live server responses into a structured comparison context. The prompt expects concrete evidence, not assumptions. If you lack live server access, the prompt will flag low-confidence results and recommend deferring the staleness check. For high-risk production systems where stale tool calls could cause data corruption or financial impact, always pair this prompt's output with a human review step before triggering an automated registry refresh. The staleness report is a diagnostic signal, not an autonomous remediation trigger.

This prompt template is designed to be wired into an AI operations pipeline that periodically scans a tool registry for staleness. It expects structured input containing registry metadata, server health signals, and recent tool response samples. The model is instructed to compare timestamps, evaluate health check results, and analyze response patterns to produce a structured staleness report with confidence scores and recommended actions. The template uses square-bracket placeholders that must be replaced with live data from your registry store and monitoring systems before each invocation.

textCopy
You are an AI operations analyst responsible for detecting stale tool registrations in a production agent ecosystem. Your task is to analyze the provided registry data, server health signals, and tool response patterns to identify registrations that may be outdated, unreachable, or misconfigured.

## INPUT DATA

### Current Registry Snapshot
[REGISTRY_SNAPSHOT]

### Server Health Check Results
[HEALTH_CHECK_RESULTS]

### Recent Tool Response Samples (last [TIME_WINDOW])
[TOOL_RESPONSE_SAMPLES]

### Expected Refresh Cadence
[EXPECTED_REFRESH_CADENCE]

## ANALYSIS INSTRUCTIONS

1. **Timestamp Comparison**: For each registered tool, compare the `last_registered_at` and `last_verified_at` timestamps against the current time and the expected refresh cadence. Flag any tool where the time since last verification exceeds the cadence threshold.

2. **Health Signal Correlation**: Cross-reference each tool's registration with its corresponding server health check results. Identify tools whose parent server shows degraded, unreachable, or unresponsive status.

3. **Response Pattern Analysis**: Examine recent tool response samples for anomalies including:
   - Schema mismatches between registered input/output schemas and actual responses
   - Increased error rates or timeout patterns
   - Deprecation headers or warning signals in responses
   - Version mismatches between registered and actual tool versions

4. **Confidence Scoring**: For each flagged tool, assign a confidence score (0.0 to 1.0) indicating how likely the registration is truly stale:
   - 0.9-1.0: Confirmed stale (multiple signals align)
   - 0.7-0.89: Likely stale (strong primary signal with supporting indicators)
   - 0.5-0.69: Possibly stale (single signal or ambiguous indicators)
   - Below 0.5: Insufficient evidence (flag for monitoring but no action)

## OUTPUT FORMAT

Produce a JSON object with the following structure:

{
  "analysis_timestamp": "ISO 8601 timestamp",
  "total_tools_evaluated": integer,
  "stale_registrations": [
    {
      "tool_id": "string",
      "tool_name": "string",
      "server_id": "string",
      "staleness_confidence": float,
      "primary_signal": "timestamp_exceeded | health_check_failed | schema_mismatch | version_mismatch | error_rate | deprecation_detected",
      "supporting_signals": ["string"],
      "last_verified_at": "ISO 8601 timestamp or null",
      "hours_since_verification": integer or null,
      "health_status": "healthy | degraded | unreachable | unknown",
      "recommended_action": "immediate_refresh | scheduled_refresh | manual_investigation | monitor_only | deregister",
      "action_rationale": "string",
      "risk_if_unchanged": "string"
    }
  ],
  "healthy_registrations_summary": {
    "count": integer,
    "average_hours_since_verification": float
  },
  "systemic_issues": [
    {
      "issue_type": "string",
      "affected_server_ids": ["string"],
      "description": "string",
      "recommended_action": "string"
    }
  ]
}

## CONSTRAINTS

- Do not flag tools that are within their expected refresh cadence unless health checks or response patterns independently indicate staleness.
- If a server is unreachable, flag all tools registered to that server with appropriate confidence adjustments.
- Distinguish between transient errors (single failed health check) and persistent failures (multiple consecutive failures).
- When recommending deregistration, include a warning about dependent agent workflows that may be impacted.
- If the input data is incomplete or missing critical fields, note this in `systemic_issues` rather than guessing.

Adaptation Notes: Replace [REGISTRY_SNAPSHOT] with a JSON array of tool registration records including tool_id, tool_name, server_id, last_registered_at, last_verified_at, and registered schema versions. Replace [HEALTH_CHECK_RESULTS] with server health data including status codes, response times, and consecutive failure counts. Replace [TOOL_RESPONSE_SAMPLES] with recent invocation logs containing actual response schemas, error codes, and deprecation headers. Replace [TIME_WINDOW] with the sampling period (e.g., '24 hours'). Replace [EXPECTED_REFRESH_CADENCE] with your organization's threshold (e.g., '1 hour' or '24 hours'). Before deploying this prompt, validate that the output JSON strictly conforms to the specified schema using a JSON Schema validator. For high-stakes production environments where deregistration could break active agent workflows, route recommendations through a human approval queue before automated execution.

The Stale Tool Registry Detection Prompt is designed to run as a scheduled evaluation, not a real-time user-facing call. Wire it into a cron job, a monitoring daemon, or a CI/CD pipeline step that executes every N minutes (typically 5–15 minutes for active registries, or hourly for stable ones). The harness must supply the current registry snapshot, live server health signals, and recent tool response patterns as structured inputs. The model's job is to compare these signals and produce a staleness report—not to fetch the data itself. Keep the data-fetching logic in application code where you can control timeouts, authentication, and error handling.

Input assembly: Before calling the model, gather three data sources: (1) the registry manifest with tool names, versions, and last-registered timestamps; (2) health-check results from each MCP server or tool endpoint, including response codes, latency, and capability listing freshness; (3) a sample of recent tool call logs showing invocation timestamps, success/failure rates, and response schema snapshots. Serialize these into the [REGISTRY_SNAPSHOT], [HEALTH_SIGNALS], and [TOOL_RESPONSE_PATTERNS] placeholders. Use JSON or a structured markdown table—the model handles both well, but JSON reduces parsing ambiguity. Validation: After the model returns its staleness report, validate the output against a strict schema: every detected stale tool must have a tool_id, staleness_confidence (0.0–1.0), evidence_summary, and recommended_action. Reject reports with missing fields or confidence scores outside bounds. If validation fails, retry once with the same inputs and a stronger constraint instruction appended to the prompt. If the second attempt also fails, log the raw output and escalate to the on-call channel—do not silently accept a malformed report.

Model choice and latency: This is a classification-and-summary task with moderate reasoning depth. GPT-4o, Claude 3.5 Sonnet, or equivalent models handle it reliably. For cost-sensitive deployments, a smaller model like Claude 3 Haiku or GPT-4o-mini can work if you tighten the output schema and add few-shot examples of stale vs. healthy tool entries. Expect 2–8 seconds of latency depending on registry size. Retry and circuit-breaking: If the model call fails due to a provider error (rate limit, timeout, 5xx), retry with exponential backoff (1s, 4s, 16s) up to three attempts. If the tool endpoints themselves are unreachable during data gathering, mark those tools as health_status: unknown rather than skipping them—the prompt is designed to flag unknown health as a staleness signal. Logging and audit trail: Log every invocation with the input snapshot hash, the model version, the raw output, the validated report, and any actions taken (e.g., registry refresh triggered, alert fired). This audit trail is essential for debugging false positives and proving to governance teams that stale registries are detected systematically.

Alert routing: Parse the validated staleness report and route entries by severity. Tools with staleness_confidence >= 0.8 and recommended_action: refresh should trigger an automated registry refresh if your platform supports it safely. Tools with recommended_action: investigate or deprecate should create a ticket in your operations queue with the evidence summary attached. Tools flagged as health_status: unknown with high confidence should page the on-call engineer if they are critical-path dependencies. What to avoid: Do not use this prompt inside a user-facing chat loop. Do not let the model directly execute registry mutations—always route recommended actions through application-level safeguards. Do not skip validation; a hallucinated staleness flag that triggers an unnecessary registry refresh can cascade into agent failures if the refresh itself introduces breaking changes. Start with a dry-run mode that logs recommendations without acting on them, then graduate to automated refreshes only after a week of clean runs with zero false positives.