Message Serialization Format Decision Prompt

This prompt is for an architect or technical lead who must choose a serialization format for a new service, event stream, or data pipeline. The job-to-be-done is producing a structured, evidence-backed comparison of Avro, Protobuf, and JSON Schema across dimensions that matter in production: schema evolution, performance, tooling ecosystem, and team capability. Use it when the decision will affect multiple teams, when the wrong choice creates costly migrations, or when the team is stuck in an advocacy loop without clear criteria.

Do not use this prompt for trivial decisions where any format works, for runtime performance micro-benchmarks that require actual profiling, or when the organization already has a mandated standard with a supported schema registry. The prompt is also inappropriate when the real problem is team skill gaps rather than format selection—no format comparison will fix a team that cannot write schemas. If the decision is urgent and irreversible, supplement the prompt output with a spike implementation that validates the top candidate against real payloads and consumer code.

Before running this prompt, gather the concrete requirements: expected message throughput, payload size ranges, consumer language diversity, existing schema registry infrastructure, backward-compatibility requirements, and any regulatory constraints on data format. Feed these into the [CONSTRAINTS] placeholder. The prompt works best when you provide at least two concrete [EXAMPLES] of messages the system must handle, so the analysis stays grounded in real shapes rather than abstract feature lists. After receiving the output, validate the comparison against your own benchmarks and review the 'premature optimization detection' eval criteria before committing to a decision.

The following template is designed to be copied directly into your AI harness. It forces a structured, criteria-weighted comparison rather than a generic summary. Every placeholder in square brackets must be replaced with your specific context before execution. The prompt is self-contained and can be used without the rest of this playbook, but you should wire it into a validation and review step as described in the Implementation Harness section.

textCopy
You are a principal software architect evaluating message serialization formats for a new or migrating system. Your task is to produce a structured, evidence-based comparison of Apache Avro, Protocol Buffers (Protobuf), and JSON Schema for the specific use case described below. Do not produce a generic summary. Your recommendation must be traceable to the weighted criteria.

## SYSTEM CONTEXT
[SYSTEM_DESCRIPTION: Describe the system, its scale, latency requirements, and primary consumers/producers.]

## EVALUATION CRITERIA
Rank the following criteria by importance for this decision (1-5, where 5 is critical):
- Schema Evolution Support: [RATING]
- Serialization/Deserialization Performance: [RATING]
- Message Size Efficiency: [RATING]
- Human Readability and Debugging: [RATING]
- Language Ecosystem and Tooling: [RATING]
- Team Familiarity: [RATING]
- Schema Registry Integration: [RATING]
- Dynamic Typing / Late Binding Needs: [RATING]

## CONSTRAINTS
- [CONSTRAINT_1: e.g., Must support Java and Python producers.]
- [CONSTRAINT_2: e.g., Messages must be inspectable via CLI tools.]
- [CONSTRAINT_3: e.g., Schema registry must be self-hosted.]

## OUTPUT FORMAT
Produce a JSON object with the following structure. Do not include any text outside the JSON.
{
  "decision_summary": "A 2-3 sentence recommendation with primary rationale.",
  "format_comparison": {
    "avro": {
      "score": 0.0-10.0,
      "strengths": ["strength_1", "strength_2"],
      "weaknesses": ["weakness_1", "weakness_2"],
      "best_when": "Scenario description."
    },
    "protobuf": { ... },
    "json_schema": { ... }
  },
  "weighted_analysis": {
    "criteria": [
      {
        "name": "Schema Evolution Support",
        "weight": 1-5,
        "avro_score": 0.0-10.0,
        "protobuf_score": 0.0-10.0,
        "json_schema_score": 0.0-10.0,
        "rationale": "Why these scores were assigned."
      }
      // Repeat for all criteria
    ]
  },
  "premature_optimization_warning": "A flag and explanation if performance differences are negligible for this use case, making team familiarity the deciding factor.",
  "migration_complexity": "If migrating, an assessment of the effort required for each format.",
  "unresolved_questions": ["Questions that must be answered before a final decision."]
}

To adapt this template, start by filling in the [SYSTEM_DESCRIPTION] with enough detail to constrain the analysis—mention throughput, message sizes, consumer diversity, and whether you have a schema registry. The criteria ratings are the most critical input; they directly control the weighted analysis. If you are not migrating, remove the migration_complexity field from the output schema. For high-stakes decisions where the recommendation will be committed to architecture documentation, always route the output through a human review step before finalizing.

This prompt is designed to operate as a decision-support tool, not an automated decision-maker. The output is a structured comparison table and recommendation that must be reviewed by a human architect before any format is adopted. Wire this prompt into an architecture review pipeline where it runs after a design document or RFC has been drafted, producing a serialization analysis that feeds into an Architecture Decision Record (ADR). The prompt expects a rich [CONTEXT] block containing team skill profiles, existing infrastructure details, throughput requirements, and schema evolution history—do not call it with a one-line question and expect a useful answer.

Validation and retry logic is critical here because the output schema is complex. Implement a post-generation validator that checks: (1) the comparison_matrix contains all three formats (Avro, Protobuf, JSON Schema) with non-empty entries for each evaluation dimension; (2) the recommendation field includes a primary_choice, runner_up, and dealbreaker_conditions; (3) the premature_optimization_warning boolean is present and consistent with the team's stated scale. If validation fails, retry once with the error details injected into the [CONSTRAINTS] block as a validation_failure_feedback field. After two failures, escalate to a human reviewer with the partial output and failure log. Log every run—including the input context hash, model used, token counts, validation results, and final reviewer—so you can audit decisions later.

Model choice matters. Use a model with strong reasoning and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller or older models that may collapse the comparison into generic pros/cons without format-specific detail. Set temperature to 0.1–0.2 to reduce variance in the recommendation while preserving enough flexibility for nuanced trade-off language. If your organization has a schema registry (e.g., Confluent Schema Registry), consider augmenting the prompt with a retrieval step that pulls the current schema count, compatibility mode, and team adoption stats before calling the model—this grounds the analysis in real infrastructure rather than hypotheticals. The [TOOLS] placeholder can accept a schema_registry_query function if you want the model to request live data during reasoning.

Human approval is mandatory before any format decision is finalized. The prompt output should be rendered into a review document (markdown ADR, Notion page, or pull request comment) with the comparison table, recommendation, and premature optimization flag clearly visible. Require at least one senior engineer or architect to sign off, with particular attention to the dealbreaker_conditions field—these are the scenarios where the recommendation should be reversed. After approval, store the full prompt input, output, and reviewer identity in your decision log. Avoid the temptation to skip this gate: serialization format choices are expensive to change later, and the prompt's job is to structure the conversation, not end it.