Log Event Extraction Prompt for SIEM Systems

This prompt is built for security engineers and SOC analysts who need to convert raw, unstructured log entries into structured events that a SIEM can ingest directly. The ideal user is someone managing a heterogeneous log stream—firewall syslog, IDS alerts, cloud audit trails (AWS CloudTrail, GCP Audit Logs), endpoint agent output, or application logs—where each source speaks a different dialect. The job-to-be-done is normalization at ingestion time: taking a raw log line and producing a strictly typed payload in Common Event Format (CEF), Log Event Extended Format (LEEF), or a JSON schema aligned with a common information model. The prompt requires the raw log text, the target output schema, and a known source identifier (e.g., source_product or deviceVendor) as minimum inputs. Without these, the model cannot produce a payload that downstream parsers will accept.

Use this prompt when you control the ingestion pipeline and can validate the output before forwarding to Splunk, QRadar, ArcSight, Microsoft Sentinel, or a similar platform. It is appropriate for batch replay of historical logs, real-time streaming enrichment, or migration projects where legacy log formats must be mapped to a new SIEM schema. The prompt assumes you have already established field-mapping rules or a common information model; it does not invent mappings from scratch. You must supply the mapping contract—either as explicit field lists in the output schema or as reference examples. The prompt handles severity classification, timestamp normalization, and source identification, but it relies on your provided rules for field-level transformations.

Do not use this prompt when the raw log contains sensitive data that cannot be sent to an external model endpoint, unless you are running a local or private deployment with appropriate data handling controls. It is not a replacement for a log parsing engine (e.g., Logstash, Fluentd, Cribl) when the log format is already known and deterministic; regex-based parsers are cheaper and more predictable for static formats. Avoid this prompt for real-time alert generation where latency must be sub-second and model inference time is unacceptable. Finally, do not use it when the target SIEM requires strict schema registry compliance (e.g., Avro with a registered schema) without a post-processing validation step—the prompt produces structured output, but schema enforcement belongs in the application harness, not in the model's generation step.

This prompt template is the core instruction set you will send to the model. It is designed to take a raw, unstructured log line and a set of configuration parameters, and return a strictly typed JSON event payload suitable for direct ingestion into a SIEM. The template uses square-bracket placeholders for all dynamic parts—your specific log source, your target schema, your severity mapping rules, and your normalization logic. Before using this prompt in any application, you must replace every placeholder with concrete values that match your environment's contracts. Leaving a placeholder unresolved will cause the model to hallucinate or default to an unpredictable behavior, which is unacceptable in a security pipeline.

textCopy
You are a precise log normalization engine. Your task is to parse the provided raw log line and extract a structured security event.

## Input
Raw Log Line:
[RAW_LOG_LINE]

## Configuration
- Log Source Type: [LOG_SOURCE_TYPE]
- Target Output Schema (JSON): [OUTPUT_SCHEMA]
- Timestamp Format in Log: [LOG_TIMESTAMP_FORMAT]
- Target Timestamp Format: ISO 8601 UTC (e.g., '2023-10-05T14:48:00.000Z')
- Severity Mapping Rules:
  [SEVERITY_MAPPING_RULES]
- Field Normalization Rules:
  [FIELD_NORMALIZATION_RULES]
- Default Values for Missing Fields:
  [DEFAULT_VALUES]

## Constraints
- [CONSTRAINTS]

## Instructions
1. Parse the timestamp from the raw log using the provided format and convert it to the target format. If parsing fails, use the current UTC time and set 'timestamp_confidence' to 'low'.
2. Map the log's native severity or priority to a standard scale (e.g., 0-10) using the provided rules. If no rule matches, default to [DEFAULT_SEVERITY] and set 'severity_confidence' to 'low'.
3. Extract all other fields required by the output schema. Apply the normalization rules.
4. For any required field that cannot be found or inferred, use the specified default value and set the corresponding '[field_name]_confidence' to 'absent'.
5. For any optional field that is not present, omit it from the output.
6. Add a top-level 'parser_confidence' field ('high', 'medium', 'low') based on the overall success of extraction and normalization.
7. Output ONLY the valid JSON object. Do not include any explanatory text, markdown fences, or code blocks.

## Output

To adapt this template, start by defining your [OUTPUT_SCHEMA]. This should be a strict JSON schema representing a single event, including required fields like timestamp, source_ip, event_type, and severity. The [SEVERITY_MAPPING_RULES] placeholder is critical; it should contain explicit instructions like 'Map "CRITICAL" to 10, "ERROR" to 8, "WARN" to 5...'. The [CONSTRAINTS] section is where you enforce your SIEM's contract, for example: 'The source_ip field must be a valid IPv4 or IPv6 address. If the log contains a hostname, do not resolve it; leave the field null.' This prompt is designed to be stateless and idempotent, making it safe for retry logic in your ingestion pipeline.

Before integrating this prompt into a production pipeline, you must pair it with a robust validation harness. The model's output should be validated against your [OUTPUT_SCHEMA] in application code. Any validation failure should trigger a retry with a more specific error message, or route the raw log to a dead-letter queue for human review. Never trust the model's parser_confidence field alone as a gate; always enforce schema compliance programmatically. For high-risk security environments, a human-in-the-loop review step is mandatory for all events with a 'low' confidence score or those that fail validation after a single retry.

This prompt is designed to sit inside a log normalization microservice or an event processing function that receives raw log lines and emits structured SIEM events. The typical call flow is: receive a raw log string, assemble the prompt with the log text and your SIEM field mapping contract, call the model, validate the output JSON against your expected schema, and either publish the event to your SIEM or route it to a dead letter queue for human review. Because security operations cannot tolerate hallucinated severity levels or missing timestamps, the implementation harness must enforce strict post-processing before any event reaches the SIEM.

Start by defining a JSON Schema that matches your SIEM's required fields. At minimum, validate that timestamp is present and parseable as ISO 8601, severity is one of your allowed enum values (e.g., INFO, LOW, MEDIUM, HIGH, CRITICAL), and source contains a non-empty identifier. Use a validation library like ajv (JavaScript) or pydantic (Python) to reject malformed outputs before ingestion. If the model returns a confidence score below your threshold—typically 0.7 for automated ingestion—route the event to a review queue rather than publishing it directly. For high-severity events (HIGH or CRITICAL), consider requiring human approval regardless of confidence, since false positives at those levels trigger on-call escalations.

Model choice matters here. Use a model with strong JSON mode support and low latency, since log pipelines often process high volumes. GPT-4o or Claude 3.5 Sonnet work well for low-to-medium throughput use cases; for higher throughput, consider a fine-tuned smaller model that has been trained on your specific log formats and field mappings. Implement retries with exponential backoff when the model returns invalid JSON or fails schema validation, but cap retries at three attempts to avoid queue backpressure. Log every extraction attempt—including the raw log, the model's raw response, validation errors, and the final published or quarantined event—so your security team can audit the pipeline and trace any missed or miscategorized events back to their source.

When wiring this into a streaming pipeline like Kafka, use the log source and timestamp as the partition key to preserve ordering for related events. If your SIEM expects CEF or LEEF format rather than JSON, add a post-processing transformation layer that converts the validated JSON output into the required key-value format. Never pass unvalidated model output directly to your SIEM; a single hallucinated field can corrupt dashboards, trigger false alerts, or mask real incidents. The prompt is the extraction engine, but the harness is what makes it production-safe.