Metrics Collection Architecture Design Prompt

This prompt is designed for platform engineers, SREs, and technical architects who need to produce a structured, defensible Architecture Decision Record (ADR) for a metrics collection pipeline. The job-to-be-done is not just generating a design, but forcing explicit reasoning about cardinality budgets, aggregation strategies, and storage trade-offs before any infrastructure is provisioned. Use this when you are selecting between push vs. pull models, deciding on a time-series database, or defining a standard for metric naming and labeling across services. The prompt assumes you have a clear understanding of the system's scale, the services involved, and the consumers of the metrics (e.g., dashboards, alerting systems, auto-scalers).

Do not use this prompt for general application performance monitoring (APM) setup, log aggregation design, or distributed tracing strategy—those are separate architectural concerns with different constraints. This prompt is also inappropriate for simple, single-service deployments where a managed cloud provider's default metrics agent is sufficient; it is intended for complex, multi-service architectures where uncontrolled metric cardinality can cause outages, storage costs can spiral, and aggregation rules must be defined to prevent data loss. The prompt requires you to provide a [SYSTEM_CONTEXT] that includes the service topology, expected request volumes, and existing observability stack components. Without this context, the model will produce a generic design that fails to address your specific cost and performance constraints.

The primary risk addressed by this prompt is the creation of an unmaintainable, expensive, and operationally dangerous metrics pipeline. The prompt forces the model to validate the design against USE (Utilization, Saturation, Errors) and RED (Rate, Errors, Duration) signal coverage, ensuring that the architecture supports both infrastructure and application-level monitoring. Before using the output, you must review the generated cardinality budget for feasibility against your metrics backend's documented limits. The ADR format ensures the decision is traceable and can be challenged by other engineers. Proceed by gathering your system context, defining your non-negotiable constraints (e.g., data residency, retention period), and then running the prompt to generate a draft ADR that you can refine with your team.

The following prompt template is designed to guide a model through the creation of a comprehensive Architecture Decision Record (ADR) for a metrics collection pipeline. It forces the model to consider cardinality budgets, aggregation strategies, storage selection, and alignment with USE and RED signals. The placeholders in square brackets are the only parts you need to customize before use.

textCopy
You are a senior platform architect specializing in observability systems. Your task is to produce a detailed Architecture Decision Record (ADR) for a metrics collection architecture based on the provided context.

## CONTEXT
[SYSTEM_CONTEXT]

## CONSTRAINTS
- Cardinality Budget: [MAX_CARDINALITY]
- Retention Period: [RETENTION_PERIOD]
- Storage Backend: [STORAGE_BACKEND]
- Query Performance Target: [QUERY_SLA]
- Compliance Requirements: [COMPLIANCE_FRAMEWORKS]

## INSTRUCTIONS
1.  **Analyze the System Context:** Identify the key services, their interactions, and the critical user journeys.
2.  **Propose a Metrics Pipeline:** Design the flow from metric generation to storage and querying. Specify the collection agent (e.g., OpenTelemetry Collector, Prometheus), transport, and aggregation layers.
3.  **Define Aggregation Rules:** Specify how metrics should be pre-aggregated at the application, host, or cluster level to stay within the cardinality budget. Detail which labels must be preserved and which can be dropped.
4.  **Select Metric Types:** Choose between counters, gauges, histograms, and summaries for key signals, justifying your choices.
5.  **Map to USE and RED Signals:** Explicitly map your proposed metrics to the Utilization, Saturation, and Errors (USE) method for infrastructure and the Rate, Errors, and Duration (RED) method for services. Identify any gaps in coverage.
6.  **Storage and Retention Strategy:** Justify the choice of [STORAGE_BACKEND] and define a downsampling or expiration policy that aligns with the [RETENTION_PERIOD] and [QUERY_SLA].
7.  **Identify Risks and Trade-offs:** Document the risks of this architecture, such as cost overruns from label explosion or blind spots in the signal coverage.

## OUTPUT_SCHEMA
Produce your response as a valid JSON object with the following structure. Do not include any text outside the JSON block.
{
  "title": "ADR: Metrics Collection Architecture for [SYSTEM_NAME]",
  "status": "Proposed",
  "context": "A concise summary of the system and the problem this architecture solves.",
  "decision": {
    "pipeline_design": "A textual description of the metrics pipeline.",
    "aggregation_rules": [
      {
        "metric_name": "string",
        "type": "counter|gauge|histogram",
        "aggregation": "sum|average|percentile",
        "preserved_labels": ["label1", "label2"],
        "dropped_labels": ["label3"]
      }
    ],
    "storage_policy": {
      "backend": "[STORAGE_BACKEND]",
      "retention": "[RETENTION_PERIOD]",
      "downsampling_rules": "A description of how data resolution is reduced over time."
    }
  },
  "signal_coverage": {
    "use_method": {
      "utilization": ["metric1"],
      "saturation": ["metric2"],
      "errors": ["metric3"]
    },
    "red_method": {
      "rate": ["metric4"],
      "errors": ["metric5"],
      "duration": ["metric6"]
    },
    "gaps": ["A list of signals not captured by this design."]
  },
  "risks_and_tradeoffs": [
    "A list of identified risks, such as cost, complexity, or blind spots."
  ],
  "consequences": "What teams must do to implement this decision."
}

To adapt this template, replace the placeholders in the CONSTRAINTS section with your project's specific non-functional requirements. The [SYSTEM_CONTEXT] placeholder should be filled with a detailed description of your application's architecture, including a service map and key user flows. The output schema is strict JSON, which makes the result directly parseable by downstream automation, such as a CI/CD pipeline that publishes ADRs to a central repository. If your system has a high-risk profile, you should add a [RISK_LEVEL] constraint and a corresponding human-review step in your implementation harness before the ADR is accepted.

This prompt is designed to be integrated into a platform engineering workflow, not used as a one-off chat. The primary integration point is an internal developer portal or an architecture review pipeline where a platform engineer submits a system context document and receives a structured Architecture Decision Record (ADR) as output. The harness should treat the LLM call as a draft generator, not an authoritative decision-maker. The output must pass through a validation layer that checks for the presence of required ADR sections (Context, Decision, Consequences), cardinality budget estimates, and explicit coverage mapping against USE (Utilization, Saturation, Errors) and RED (Rate, Errors, Duration) signals before it is surfaced to the user or stored in a decision log.

Validation and retry logic is critical because missing cardinality budgets or incomplete signal coverage can lead to production incidents. Implement a post-generation validator that parses the JSON output and checks: (1) the cardinality_budget field is a non-null integer, (2) the aggregation_rules array is non-empty, (3) the signal_coverage object contains keys for at least use and red with non-empty arrays, and (4) the storage_selection field includes a retention_policy. If validation fails, construct a retry prompt that includes the original input, the failed output, and a specific error message like VALIDATION_ERROR: cardinality_budget is missing or null. Limit retries to 2 attempts. If both attempts fail, route the request to a human review queue with the partial output and validation errors attached. Log every attempt, validation result, and final disposition for prompt observability and future regression testing.

Model choice and tool use should favor models with strong JSON mode and structured output support (e.g., GPT-4o with response_format set to json_schema or Claude 3.5 Sonnet with tool-use for schema enforcement). Do not rely on free-text completion for this prompt; the output must be machine-readable to feed downstream automation like Terraform module generation or OpenTelemetry Collector configuration. If your platform supports it, provide the model with a tool definition that matches the expected ADR schema. This reduces parsing errors and makes validation failures rarer. For RAG integration, you can optionally supply the model with your organization's existing SLO definitions, incident postmortems, or infrastructure cost data as additional context in the [CONTEXT] placeholder to ground the storage and cardinality recommendations in real constraints.

Human review is required before the generated ADR is committed to a decision log or used to provision infrastructure. The harness should present the validated output in a review UI that highlights: cardinality budget vs. current ingestion volume, any signal coverage gaps flagged by the validator, and the estimated monthly cost of the recommended storage tier. The reviewer must explicitly approve or request revision. Approved ADRs should be stored with a versioned prompt hash, model identifier, and reviewer identity for auditability. Avoid wiring this prompt directly into CI/CD pipelines without a human-in-the-loop gate, especially when the output drives resource provisioning or alerting rule creation.