Tree-of-Thought Performance Optimization Prompt

This prompt is designed for performance engineers and SREs facing a known, profiled bottleneck where the optimal fix is not obvious. The job-to-be-done is generating a structured comparison of multiple optimization strategies—such as caching, query refactoring, infrastructure scaling, and code-level changes—before committing engineering time. The ideal user has already identified a specific bottleneck through profiling, observability data, or load-test results, and needs to evaluate trade-offs across cost, risk, expected impact, and rollback complexity. The prompt produces a branching analysis tree where each path explores a different approach, making it suitable when the optimization surface is large and the decision requires buy-in from multiple stakeholders.

Do not use this prompt for simple 'make it faster' requests where a single fix is obvious, or when you lack concrete profiling data to ground the analysis. The prompt assumes the bottleneck is already isolated—if you are still hunting for the root cause, use a Chain-of-Thought Root Cause Analysis Prompt instead. The output is a decision-support artifact, not an automated execution plan. It requires human review to validate assumptions, verify cost estimates against your actual infrastructure, and confirm that recommended approaches align with your operational constraints. In high-risk production environments, always pair the analysis with a rollback plan and a canary deployment strategy before implementing any branch.

Before using this prompt, gather your profiling data, current architecture context, and any hard constraints such as latency budgets, cost ceilings, or compliance requirements. The quality of the branching analysis depends directly on the specificity of the bottleneck description you provide. Vague inputs produce shallow trees with generic advice; precise inputs produce actionable comparisons. After generating the analysis, use the evaluation criteria in the harness section to assess branch completeness, assumption validity, and risk coverage before presenting the tree to your team for a decision.

This prompt template is the core instruction set for a Tree-of-Thought performance optimization workflow. It forces the model to explore multiple independent solution branches—such as caching, query refactoring, and infrastructure scaling—rather than converging prematurely on a single recommendation. Each branch must be evaluated against cost, risk, and expected impact before a final synthesis is produced. The template is designed to be pasted directly into a model interface with your specific profiling data and system constraints substituted into the square-bracket placeholders.

textCopy
You are a senior performance engineer analyzing a system bottleneck. Your task is to explore multiple distinct optimization strategies using a Tree-of-Thought approach. For each strategy, generate a branch of reasoning that evaluates feasibility, cost, risk, and expected impact. Do not converge on a single answer until all branches have been explored to the specified depth.

## BOTTLENECK DATA
[BOTTLENECK_DATA]

## SYSTEM CONSTRAINTS
[SYSTEM_CONSTRAINTS]

## INSTRUCTIONS
1. Identify at least [NUM_BRANCHES] distinct optimization strategies that could address the bottleneck. Consider categories such as:
   - Caching strategies (application-level, CDN, database query cache)
   - Query or code refactoring (index changes, query rewriting, algorithm improvements)
   - Infrastructure scaling (vertical, horizontal, read replicas, connection pooling)
   - Architecture changes (async processing, data denormalization, service extraction)

2. For each strategy, generate a branch that includes:
   - A clear description of the proposed change
   - The mechanism by which it addresses the bottleneck
   - An estimated impact on latency and throughput (Low/Medium/High with justification)
   - Implementation complexity and risk (Low/Medium/High with justification)
   - A confidence score (0.0 to 1.0) reflecting how certain you are in this assessment given the available data

3. Explore each branch to a depth of [DEPTH] sub-steps. At each sub-step, evaluate whether the branch remains viable or should be pruned due to a constraint violation, excessive risk, or diminishing returns.

4. After all branches are explored, produce a final synthesis that:
   - Ranks the viable strategies by expected net benefit
   - Identifies the recommended first action
   - Flags any strategies that require additional data before a decision can be made
   - Notes any branches that were pruned and why

## OUTPUT FORMAT
Return a JSON object with the following structure:
{
  "branches": [
    {
      "id": "string",
      "strategy_name": "string",
      "category": "caching|query_refactor|infrastructure|architecture",
      "description": "string",
      "mechanism": "string",
      "estimated_impact": {
        "latency": "Low|Medium|High",
        "throughput": "Low|Medium|High",
        "justification": "string"
      },
      "implementation": {
        "complexity": "Low|Medium|High",
        "risk": "Low|Medium|High",
        "justification": "string"
      },
      "confidence": 0.0,
      "sub_steps": [
        {
          "depth": 0,
          "evaluation": "string",
          "viable": true,
          "prune_reason": null
        }
      ]
    }
  ],
  "synthesis": {
    "ranked_strategies": ["branch_id"],
    "recommended_first_action": "string",
    "requires_additional_data": ["branch_id"],
    "pruned_branches": [
      {
        "branch_id": "string",
        "reason": "string"
      }
    ]
  }
}

## CONSTRAINTS
- Do not recommend strategies that violate [SYSTEM_CONSTRAINTS].
- If a branch requires an assumption, state it explicitly in the sub-step evaluation.
- If the bottleneck data is insufficient to evaluate a branch, flag it in `requires_additional_data` rather than guessing.
- Prefer strategies with lower risk when estimated impact is similar.

Adapting the template: Replace [BOTTLENECK_DATA] with your profiling output—query plans, flame graphs, slow-query logs, or APM trace summaries. Replace [SYSTEM_CONSTRAINTS] with hard limits such as budget caps, maintenance windows, compliance requirements, or technology stack restrictions. Set [NUM_BRANCHES] to at least 3 to force divergent thinking; increase to 5-7 for complex bottlenecks where the solution space is wide. Set [DEPTH] to 2-3 for most use cases—deeper trees increase token cost and can introduce speculative drift without adding decision value.

Validation and safety: The output schema enforces structured evaluation at every leaf node, which makes automated validation possible. Before acting on any recommendation, validate that the model's confidence scores are consistent with the justifications provided. For high-risk production changes—such as database schema migrations or infrastructure scaling decisions—require human review of the synthesis before execution. If the model flags branches as requiring additional data, collect that data and re-run the prompt rather than proceeding with low-confidence recommendations.

Next steps: After generating the tree, feed the ranked strategies into a downstream planning prompt that produces an implementation sequence with rollback checkpoints. Store the full tree output as an audit artifact so the reasoning behind the chosen strategy is traceable. If the model prunes a branch you believe is viable, check whether your [SYSTEM_CONSTRAINTS] were too restrictive or whether the bottleneck data was missing evidence for that path.

This prompt is designed for a single-turn analysis, but its output is the starting point for a critical engineering workflow, not the final decision. The implementation harness must treat the model's response as a structured recommendation that requires human validation before any production changes are made. The primary integration pattern is an asynchronous job: an engineer or an automated monitoring system submits a performance bottleneck description, the prompt generates a branching tree of optimization strategies, and the result is written to a review queue where a human can compare branches, verify assumptions, and select a course of action.

To wire this into an application, wrap the prompt in a function that accepts a [BOTTLENECK_DESCRIPTION] string and an optional [SYSTEM_METRICS] JSON object containing CPU, memory, query latency, and throughput data. The function should inject these into the prompt template, call the model with a low temperature (0.2–0.4) to balance creativity with consistency, and parse the response against a strict JSON schema. The expected output schema includes a branches array where each branch has a strategy_name, approach_description, estimated_impact, implementation_cost, risk_level, and rollback_complexity. Validate that every branch contains all required fields and that risk_level is one of low, medium, or high. If validation fails, retry once with a repair prompt that includes the schema violation details; if it fails again, log the failure and alert the requesting engineer rather than silently dropping the analysis.

After validation, the harness must write the full tree-of-thought output to a persistent store—such as a database or a ticketing system—tagged with the bottleneck description, a timestamp, and the model version used. Do not automatically apply any optimization. Instead, surface the branches in a review interface that allows a human to annotate each branch with notes, mark branches as rejected or selected, and record the final decision. Log every human action for auditability. For high-risk environments, consider adding an approval gate that requires a second reviewer sign-off before any infrastructure or query change is implemented. The prompt is a reasoning accelerator, not an auto-fix tool; the harness must enforce that boundary.