Structured Logging Schema Design Prompt

This prompt is for backend engineers, platform teams, and SREs who need to define a single, canonical structured logging schema that multiple services will adopt. Use it before writing code to prevent the observability tax of inconsistent field names, missing correlation IDs, and unparseable log payloads. The prompt produces a complete JSON Schema document with field definitions, types, required attributes, and descriptions that align with common log aggregation and querying platforms.

The ideal user brings a list of services that will emit logs, the log aggregation system in use (e.g., Datadog, Splunk, Grafana Loki, Elasticsearch), and any compliance or data governance constraints that restrict what fields can be logged. The prompt works best when you provide concrete examples of the queries and dashboards your team needs to support, such as 'trace all requests across services by request_id' or 'aggregate error rates by service and error type.' Without this context, the generated schema may be syntactically valid but operationally useless.

Do not use this prompt if you are only looking for a generic logging library configuration. It is designed for schema design, not code generation. If you need a logging strategy covering levels, sampling, and lifecycle, use the Logging Strategy Design Prompt instead. If you already have a schema and need to validate it against query patterns, use the Observability Dashboard Design Rubric Prompt. This prompt is also not a substitute for a data catalog or schema registry—it produces the schema artifact, not the governance around it.

Before running this prompt, gather: the list of services that will adopt the schema, the required correlation identifiers (e.g., trace_id, span_id, request_id), the standard fields your observability platform expects (e.g., @timestamp, severity, service.name), and any domain-specific fields that must be included (e.g., tenant_id, user_id, order_id). If you skip this preparation, the model will invent plausible but misaligned fields that create drift between the schema and your actual systems.

After generating the schema, validate it against your log aggregation platform's indexing and querying requirements. Common failure modes include: fields that collide with reserved names in your platform, nested objects that are too deep for your query engine to index efficiently, and string fields that should be enums but are left open-ended. Run a query simulation against the schema: can you answer the top five operational questions your on-call team needs? If not, iterate on the prompt with more specific query examples.

This prompt template asks the model to produce a canonical JSON Schema for structured logs across your services. It forces the model to define field names, types, required attributes, and example values before generating the schema, then validates the output against common parsing and querying requirements. Replace every square-bracket placeholder with your specific context before execution.

codeCopy
You are a backend engineer defining a canonical structured logging schema for [NUMBER] services in a [SYSTEM_DESCRIPTION].

Your task is to produce a JSON Schema that every service must follow when emitting structured logs.

## CONTEXT
- System description: [SYSTEM_DESCRIPTION]
- Services involved: [SERVICE_NAMES]
- Log consumers: [LOG_CONSUMERS, e.g., Elasticsearch, Datadog, Splunk, Loki]
- Compliance requirements: [COMPLIANCE_FRAMEWORKS, e.g., SOC2, HIPAA, GDPR, none]
- Existing logging format: [CURRENT_FORMAT_OR_NONE]
- PII/PHI in logs: [YES_NO_AND_SCOPE]

## CONSTRAINTS
- All fields must use snake_case naming.
- Timestamps must be ISO 8601 with timezone offset.
- Severity levels must follow [SEVERITY_STANDARD, e.g., RFC 5424 syslog, custom enum].
- The schema must include a `trace_id` field for distributed tracing correlation.
- The schema must include a `span_id` field.
- The schema must include a `service_name` field.
- The schema must include an `environment` field with allowed values: [ENVIRONMENTS, e.g., dev, staging, prod].
- The schema must include a `message` field for human-readable log text.
- The schema must include a `context` object for arbitrary key-value metadata.
- The schema must include an `error` object with `type`, `message`, and `stack_trace` fields, all optional.
- The schema must include a `request` object with `method`, `path`, `status_code`, `duration_ms`, and `user_agent` fields, all optional.
- No field may use `null` as a type; use optional fields instead.
- All enum fields must list their allowed values explicitly.

## OUTPUT_SCHEMA
Return a valid JSON object with this structure:
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Canonical Log Schema",
  "description": "...",
  "type": "object",
  "required": ["..."],
  "properties": { ... },
  "field_definitions": [
    {
      "field": "field_name",
      "type": "json_schema_type",
      "required": true_or_false,
      "description": "What this field captures and why it exists",
      "example": "A realistic example value"
    }
  ]
}

## VALIDATION RULES
Before returning the schema, verify:
1. Every required field appears in the `required` array.
2. Every field in `properties` has a corresponding entry in `field_definitions`.
3. No field uses `null` as its type.
4. All enum fields list their allowed values.
5. `trace_id` and `span_id` are present and use W3C Trace Context format examples.
6. Timestamps use `format: date-time`.

## EXAMPLES
[OPTIONAL_EXAMPLE_LOG_ENTRIES]

## RISK_LEVEL
[RISK_LEVEL, e.g., low, medium, high] — if high, add a note that human review is required before adoption.

After pasting this prompt, verify that the output JSON Schema passes a standard JSON Schema validator. For high-risk environments where logs contain PII or feed compliance audits, route the generated schema through a human review step before committing it to your logging library or telemetry pipeline. If the model omits required fields or produces invalid enum constraints, add those missing requirements to the CONSTRAINTS section and re-run.

This prompt is designed to be integrated into a design review or CI/CD pipeline, not used as a one-off chat interaction. The primary integration point is a pull request that introduces a new service or modifies an existing logging contract. When a developer opens a PR, a CI check can run this prompt against the proposed log schema or a sample of log output, returning a structured JSON review. The application harness should treat the model's output as a design review artifact, not an auto-mergeable change. The generated JSON schema should be committed to the repository as a source-of-truth contract, and any deviation in production logs should trigger a schema validation alert.

The harness must enforce a strict input contract. Before calling the model, validate that the provided [SERVICE_NAME] is a non-empty string, [SERVICE_DESCRIPTION] is a substantive paragraph, and [EXISTING_LOG_SAMPLES] is a valid JSON array of log objects. If the input fails validation, reject the request before consuming tokens. After receiving the model's response, validate the output against the expected JSON schema: it must contain a top-level $schema key, a properties object, and a required array. If the output fails JSON parsing or schema validation, implement a single retry with a repair prompt that includes the raw output and the validation error. If the retry also fails, fail the CI check and surface the raw output for human review. Log every interaction—input, output, validation result, and retry count—as structured metadata for prompt debugging.

For model selection, a capable mid-tier model like claude-sonnet-4-20250514 or gpt-4o is sufficient for this structured generation task. The prompt does not require tool use or retrieval-augmented generation (RAG) because the design context is fully contained in the input. However, you should wire the output into a downstream validation step: use a tool like ajv (for JSON Schema) to validate that the generated schema is itself a valid JSON Schema document. Additionally, run a check that every field in the required array exists in properties. These mechanical validations catch hallucinations that the model's self-critique might miss. The final artifact—the validated JSON schema—should be stored in a repository path like /services/[SERVICE_NAME]/log-schema.json and referenced by the service's logging library configuration.

The highest-risk failure mode is a schema that validates structurally but fails operationally—for example, a field typed as string that should be an ISO 8601 timestamp, or a missing trace_id field that breaks distributed tracing. To mitigate this, the harness should include an evaluation step that checks for the presence of required operational fields (timestamp, level, message, trace_id, service) and verifies that timestamp fields use a format constraint like "format": "date-time". If the generated schema is missing any of these, flag the review as requiring human attention. Do not auto-apply the schema to production logging configuration without a human approving the review artifact.