JSON Lines Record Separation Repair Prompt

This prompt is for data pipeline operators and backend engineers who receive JSON Lines (JSONL/NDJSON) outputs from language models that have broken record boundaries. The core job is to reconstruct valid line-delimited JSON when the model merges multiple records onto one line, omits newline separators, produces partial lines, or interleaves non-JSON text with valid records. The prompt expects a raw, malformed text blob and returns clean, separated JSON records with a verified record count. It is not a general JSON repair tool—use the JSON Repair Prompt Template for API Payloads for single-object structural fixes, and use the Streaming Fragment Assembly Prompt when you are reassembling chunks from a real-time stream rather than repairing a complete but corrupted batch output.

Use this prompt when your pipeline has already received a complete model response but the JSONL parser rejects it due to line-boundary errors. Typical failure signals include: a JSONL parser returning fewer records than expected, JSONDecodeError on lines that contain multiple concatenated objects, or validation checks that find records with fields from adjacent records merged in. The prompt requires the raw corrupted output as [MALFORMED_JSONL] and the expected record count as [EXPECTED_COUNT]. It works best when the underlying JSON objects are individually valid but incorrectly separated—if the JSON objects themselves are malformed, run the Broken JSON Object Recovery Prompt first, then feed the repaired objects into this separation prompt. Do not use this prompt for streaming data where you lack the complete payload, or for formats other than JSONL/NDJSON.

Before wiring this into an automated pipeline, test it against your three most common failure patterns: merged records (two complete JSON objects on one line), missing newlines (a single line with many concatenated objects), and partial trailing records (a truncated final object). Measure whether the output record count matches [EXPECTED_COUNT] and whether each line parses as valid JSON independently. If the model consistently fails to separate records with high accuracy, consider adding a structural validation step that counts opening and closing braces to detect boundaries before calling the repair prompt, or escalate to human review when the confidence score drops below your pipeline's threshold.

This template is designed to be copied directly into your prompt management system, test suite, or application code. It instructs the model to act as a data recovery specialist, reconstructing valid line-delimited JSON from a malformed input. The prompt uses square-bracket placeholders for all dynamic inputs, ensuring you can swap in different files, schemas, and constraints without rewriting the core instruction. The primary goal is to recover as many complete records as possible while explicitly flagging any data that is unrecoverable.

textCopy
You are a precise data repair system. Your task is to repair a malformed JSON Lines (JSONL) file.

**Input:**
The input is provided in the [INPUT] section below. It is intended to be a valid JSONL file, but record boundaries are broken. Common issues include:
- Multiple JSON objects on a single line.
- A single JSON object split across multiple lines.
- Missing newline characters between records.
- Empty or whitespace-only lines.

**Instructions:**
1.  Parse the [INPUT] and identify all complete, valid JSON objects.
2.  Reconstruct the file so that each valid JSON object is on its own line, separated by a single newline character.
3.  Do not modify the internal structure, key names, or values of any valid JSON object you recover.
4.  If you encounter text fragments that cannot be assembled into a complete, parseable JSON object, do not guess or hallucinate data. Instead, append them as a single escaped string on a new line, prefixed with the marker [UNRECOVERABLE_FRAGMENT].
5.  If a [TARGET_SCHEMA] is provided, validate each recovered record against it. Append any records that fail validation as a string on a new line, prefixed with the marker [SCHEMA_VIOLATION], followed by the original JSON object and the validation error.

**Constraints:**
- [CONSTRAINTS]

**Output Format:**
Output ONLY the repaired JSONL content inside a single fenced code block with the language tag `jsonl`.

**Input Data:**
[INPUT]

**Target Schema (Optional):**
[TARGET_SCHEMA]

To adapt this template, replace the placeholders with your specific context. The [INPUT] placeholder should contain the raw, corrupted text. Use the [CONSTRAINTS] placeholder to add rules like a maximum number of records or a specific sorting order. The [TARGET_SCHEMA] is optional; if provided, it should be a valid JSON Schema object. If you don't have a schema, remove that entire instruction block. For high-stakes data pipelines, always follow this automated repair with a manual review step for any records flagged as [UNRECOVERABLE_FRAGMENT] or [SCHEMA_VIOLATION] to prevent data loss or corruption from propagating downstream.

The JSON Lines repair prompt is designed to sit inside a post-generation validation loop, not as a standalone one-shot call. When your pipeline receives a model output that fails JSONL parsing—typically because record boundaries are missing, merged, or truncated—this prompt becomes the recovery path. The harness should catch parse errors, extract the raw text and the expected record count, and invoke the repair prompt with both. Do not call the repair prompt on every output; gate it behind a parse attempt so you only pay the latency and token cost when repair is actually needed.

The implementation loop follows a clear sequence: (1) Attempt to parse the raw output with a streaming JSONL parser that can report the line number and character offset of the first failure. (2) If parsing succeeds, validate that the parsed record count matches the expected count from your upstream request. (3) On failure, construct the repair prompt call with [RAW_OUTPUT] set to the full model response text and [EXPECTED_RECORD_COUNT] set to the number of records you requested. (4) Parse the repair prompt's response, which should be valid JSONL. (5) Re-validate record count. If the repair output still fails, log the failure, increment a repair-attempt counter, and either retry once with a slightly different temperature or escalate to a dead-letter queue for human review. Never loop more than twice—diminishing returns set in fast, and infinite retry loops on malformed data will burn tokens and block your pipeline.

For validation, use a strict JSONL parser that rejects lines that aren't valid JSON objects. After parsing, run a record count check: len(records) == expected_count. Also verify that each record is a JSON object (not an array, string, or number) unless your schema permits top-level scalars. Log the repair rate as a metric—what percentage of outputs require repair—so you can detect model drift or prompt degradation over time. If the repair rate spikes above 5-10%, investigate whether your generation prompt is producing consistently broken boundaries and fix the root cause rather than relying on the repair harness indefinitely.

Model choice matters here. The repair task is structural, not creative, so prefer models with strong instruction-following and JSON manipulation capabilities. Temperature should be set low (0.0–0.1) to maximize deterministic repair behavior. If you're using a model that supports structured output modes, you can request that the repair response itself be valid JSONL, but the prompt template already instructs the model to output only the repaired records with no markdown fences or commentary. Parse the response by stripping leading/trailing whitespace and feeding each line through JSON.parse (or equivalent). Reject any response that contains markdown fences, explanatory text, or a different number of lines than expected.

Wire this into your observability stack. Log every repair attempt with: the original parse error message, the byte offset of the failure, the expected vs. actual record count before repair, the repair prompt's token usage, and the post-repair validation result. This data lets you distinguish between transient boundary issues (one-off newline drops) and systemic failures (the model consistently merges the last two records). If you see systemic patterns, fix the generation prompt or adjust your output parsing to be more tolerant before reaching for repair. The repair prompt is a safety net, not a substitute for reliable generation.