Prompt Prefix Caching Design Prompt for Reusable Context

This prompt is a design tool for infrastructure and platform teams who need to optimize prompt assembly for LLM provider caching mechanisms, such as those offered by Anthropic and OpenAI. The primary job-to-be-done is to take an existing, functional prompt template and produce a concrete segmentation plan that identifies which sections can be made static (and therefore cacheable) and which must remain dynamic. The ideal user is an AI engineer or operator who already has a working prompt but is facing rising inference costs or latency that exceeds their application's service level objectives. You should use this prompt when you control the prompt assembly code and can reorder sections, not when you are merely a consumer of a fixed prompt from another system.

The prompt instructs the model to analyze the provided template, classify each block as static or dynamic, estimate the potential cache hit rate based on the ratio of static to dynamic tokens, and project cost savings. Critically, it also requires the model to define cache invalidation rules—the specific events or data changes that should trigger a cache miss and force a new prefix to be computed. This output directly informs your prompt assembly code and caching proxy configuration. For example, if the analysis identifies a large block of tool definitions as static, you would move that block to the very beginning of your assembled prompt in your application code, ensuring it is always part of the cached prefix. The estimated hit rate then becomes a key performance indicator you can monitor after deployment.

Do not use this prompt for general token budgeting, prompt compression, or content summarization. It is specifically for structural re-engineering to exploit prefix caching. It is also not a substitute for measuring actual cache performance with your model provider; the hit rate and cost savings are estimates based on the structure you provide. After using this prompt, your next step is to implement the segmentation plan in your prompt assembly layer, configure your HTTP client to send identical prefixes, and instrument your application to log actual cache hit/miss events to validate the model's projections against real-world behavior.

This prompt template is designed for infrastructure teams who need to restructure existing system prompts to maximize cache hit rates on LLM providers that support prompt prefix caching. The template takes a full system prompt as input and produces a segmentation plan that identifies static, reusable prefix sections and dynamic sections that must be injected per request. Use this when you have a production prompt with stable instructions, tool schemas, or few-shot examples that can be placed at the beginning of the request to benefit from caching, while keeping user-specific context, retrieved evidence, or conversation history in the uncached suffix.

textCopy
You are a prompt infrastructure architect specializing in prefix caching optimization for large language model deployments.

Analyze the provided system prompt and produce a prefix caching segmentation plan. Your goal is to maximize cache hit rates while preserving behavioral fidelity.

## INPUT
[SYSTEM_PROMPT]

## ADDITIONAL CONTEXT
- Model provider and caching behavior: [MODEL_PROVIDER]
- Average requests per cache window: [REQUESTS_PER_WINDOW]
- Cache TTL in seconds: [CACHE_TTL]
- Cost per million input tokens (cached): [CACHED_TOKEN_COST]
- Cost per million input tokens (uncached): [UNCACHED_TOKEN_COST]

## OUTPUT SCHEMA
Return a JSON object with this structure:
{
  "segments": [
    {
      "id": "string",
      "content": "string",
      "token_count": number,
      "cache_strategy": "static_prefix" | "dynamic_suffix",
      "description": "string",
      "stability_rationale": "string"
    }
  ],
  "cache_boundary_index": number,
  "estimated_cache_hit_rate": number,
  "cost_savings_per_request": number,
  "monthly_savings_projection": number,
  "invalidation_rules": [
    {
      "trigger": "string",
      "affected_segments": ["string"],
      "action": "string"
    }
  ],
  "risks": ["string"],
  "recommended_assembly_order": ["string"]
}

## CONSTRAINTS
- Place all stable, request-independent content in the static prefix.
- The static prefix must be a contiguous block at the start of the prompt.
- Tool definitions, system role instructions, and few-shot examples should be in the static prefix when they do not change per request.
- User input, retrieved documents, conversation history, and time-sensitive data belong in the dynamic suffix.
- If the static prefix exceeds 80% of the total prompt tokens, flag this as a risk.
- If any segment in the static prefix requires invalidation more frequently than the cache TTL, move it to the dynamic suffix.
- Provide specific invalidation rules for when model version, tool schemas, or behavioral policies change.
- Calculate cost savings using the provided token costs and estimated cache hit rate.

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with concrete values. For [SYSTEM_PROMPT], provide the full text of your current system message. For [MODEL_PROVIDER], specify the provider name and caching behavior details (e.g., "Anthropic Claude with automatic prefix caching, minimum 1024 token prefix"). The cost fields should use your actual provider pricing. For [RISK_LEVEL], set to "high" if the prompt controls financial, clinical, or safety-critical behavior—this will instruct the model to flag risks more aggressively and recommend human review of the segmentation plan before deployment. After receiving the output, validate that the cache_boundary_index correctly separates static from dynamic content, and run a regression test comparing outputs from the original prompt versus the segmented version before shipping to production.

This prompt is designed to run as a pre-deployment analysis step, not as part of the real-time inference path. You feed it a candidate prompt structure and it returns a segmentation plan. The plan then becomes configuration for your prompt assembly layer—telling it which sections to mark as cacheable static prefixes and which are dynamic. The implementation harness must consume the structured output (the segmentation plan) and enforce it at runtime, not rely on the model to make caching decisions per-request.

Wire the prompt into a CI/CD pipeline or a prompt review workflow. When a team proposes a new prompt or a major version change, run this analysis prompt against the candidate template. Parse the JSON output into a cache configuration object that maps section identifiers to static or dynamic labels, cache keys, and invalidation rules. Before deploying, validate that the static sections are truly static—run a linter that checks for unresolved variables, user input, or timestamps inside any section marked static. A single dynamic token inside a cached prefix will silently invalidate the cache on every request, destroying the cost savings. Log a hard failure if the validation catches this.

At runtime, your prompt assembly service reads the cache configuration. It constructs the static prefix first, sends it with a cache-breaker key (e.g., a hash of the static template content plus model version), and then appends the dynamic sections. Instrument cache hit rate, token usage per section, and cost savings against the baseline of sending the full prompt uncached. Set alerts if the cache hit rate drops below the projected estimate from the analysis prompt—this usually signals that a dynamic value leaked into the static prefix or that the model provider evicted the cache. For high-traffic systems, batch requests that share the same static prefix to maximize cache reuse. If the analysis prompt recommends a cache invalidation rule (e.g., 'invalidate when system instructions change'), encode that as a deployment hook that bumps the cache key version. Avoid manual cache key management; derive keys from content hashes of the static sections.