Inferensys

Prompt

Pairwise Source Comparison Prompt with Conflict Output Contract

A practical prompt playbook for using Pairwise Source Comparison Prompt with Conflict Output Contract in production AI workflows.
Wide-angle shot of a modern WeWork open floor plan with creative walls covered in AI system architecture diagrams, product team collaborating in standing desk area with industrial lighting.
PROMPT PLAYBOOK

When to Use This Prompt

Defines the ideal user, required context, and boundaries for deploying a pairwise source comparison prompt that produces machine-readable conflict records.

This prompt is designed for API developers and verification engineers who need to integrate a deterministic fact-checking step into a production pipeline. The core job-to-be-done is comparing two source documents—typically raw text or structured evidence records—and receiving a structured JSON output that identifies factual conflicts. The ideal user is building a system where downstream services, databases, or monitoring dashboards consume conflict records programmatically. You should use this prompt when you need typed conflict fields (e.g., direct_contradiction, numerical_discrepancy), explicit provenance links that map each conflicting claim back to its source document and location, and a machine-actionable verdict that requires no human parsing. The required context includes two complete source documents, a defined output schema, and a clear specification of what constitutes a reportable conflict versus an acceptable paraphrase or stylistic difference.

Do not use this prompt for open-ended editorial analysis where a human reader expects a narrative summary, nor for single-document review workflows that lack a comparative source. It is also unsuitable for end-user-facing chat interfaces where the raw JSON output would be confusing or where the latency of a structured generation step degrades the user experience. Avoid this prompt when the source documents contain primarily subjective claims, opinions, or speculative statements that lack a ground-truth reference; the prompt's conflict typing logic relies on verifiable factual assertions. If your workflow requires resolving contradictions across three or more sources, use a multi-source triangulation prompt instead, as this pairwise design does not handle weighted consensus or majority-rules logic. Finally, if your application cannot tolerate a structured output failure—for example, in a synchronous user-facing flow with no retry budget—consider whether a simpler classification prompt or a human-in-the-loop review step is more appropriate before committing to a strict schema contract.

To implement this successfully, you must pair the prompt with a validation harness that checks the output against the expected JSON schema before the response is passed downstream. The harness should verify required fields (conflict_id, claim_a, claim_b, conflict_type, verdict, provenance), validate enum values for conflict_type and verdict, and confirm that provenance links reference the correct source identifiers provided in the input. Integration test cases should include known contradictory document pairs, known consistent pairs, and edge cases such as empty documents, identical documents, and documents that differ only in formatting or synonym choice. For high-stakes verification domains—such as legal, financial, or clinical evidence review—always route the structured output through a human approval queue before any automated action is taken based on the conflict verdict. The prompt itself is a component, not a complete solution; its reliability depends on the surrounding validation, retry, and review infrastructure you build around it.

PRACTICAL GUARDRAILS

Use Case Fit

This prompt is designed for API developers who need machine-readable contradiction outputs from pairwise source comparison. It excels in structured verification pipelines but is not a general-purpose fact-checking chatbot. Understand where it fits and where it breaks before wiring it into production.

01

Good Fit: Structured Verification Pipelines

Use when: you need deterministic, schema-validated JSON conflict records with typed fields, provenance links, and machine-actionable verdicts. The prompt is built for downstream ingestion by databases, dashboards, or automated triage systems.

02

Bad Fit: Open-Ended Conversational Fact-Checking

Avoid when: the user expects a free-text explanation or a chat-based debate. This prompt is not designed for dialogue. Using it in a chatbot will produce rigid JSON that frustrates users expecting natural language.

03

Required Inputs: Paired Sources and a Strict Schema

What to watch: The prompt fails silently if source texts are missing, truncated, or not clearly delimited. Guardrail: Validate that both [SOURCE_A] and [SOURCE_B] are non-empty strings before calling the model. Enforce the output JSON schema with a post-generation validator.

04

Operational Risk: False Positives from Paraphrasing

What to watch: The model may flag semantically equivalent statements as contradictions due to lexical differences. Guardrail: Implement a secondary semantic similarity check or a human-review queue for low-confidence conflict types, especially 'direct_contradiction' vs. 'partial_mismatch'.

05

Operational Risk: Context Window Truncation

What to watch: Long source documents may be truncated, causing the model to miss the context that resolves an apparent contradiction. Guardrail: Chunk large documents and run pairwise comparisons on aligned sections. Log a context_truncated warning if source length exceeds a safe threshold.

06

Operational Risk: Over-Confidence on Ambiguous Claims

What to watch: The model may assign a high confidence score to a conflict verdict even when the underlying claims are hedged or vague. Guardrail: Calibrate the confidence field against a human-labeled test set. Route any verdict with a confidence score below your established threshold to a human review queue.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A copy-paste-ready prompt that compares two source documents and returns a structured JSON object identifying all factual conflicts, designed for direct integration into verification pipelines.

This prompt template is the core instruction set for a pairwise source comparison task. It is designed for API developers and verification engineers who need a machine-readable output contract, not a prose summary. The prompt forces the model to identify only factual conflicts—statements that cannot both be true—while ignoring differences in phrasing, scope, or interpretation. Use this when you need to feed conflict data directly into a downstream system, such as a fact-checking dashboard, a RAG pipeline's post-retrieval filter, or an automated evidence-grading service. Do not use this prompt for general document summarization or for generating human-facing reports without further processing; its value is in the strict JSON schema it enforces.

text
You are a precise contradiction detection engine. Your task is to compare two provided source documents and identify all factual conflicts. A factual conflict exists when two statements assert incompatible facts about the same specific entity, event, time, or measurement, such that both cannot be true simultaneously.

Ignore differences in phrasing, writing style, level of detail, or scope. Do not flag a conflict if one source is merely silent on a detail mentioned by the other. Do not flag opinions, predictions, or interpretations as conflicts unless they contain a core factual assertion that is directly contradicted.

## INPUTS
- Source A: [SOURCE_A_TEXT]
- Source B: [SOURCE_B_TEXT]
- Source A Identifier: [SOURCE_A_ID]
- Source B Identifier: [SOURCE_B_ID]

## OUTPUT SCHEMA
Return ONLY a valid JSON object conforming to this exact schema. Do not include any text outside the JSON object.

{
  "conflicts": [
    {
      "conflict_id": "string, unique identifier for this conflict, e.g., 'C01'",
      "claim_a": "string, the exact verbatim text from Source A that contains the conflicting assertion",
      "claim_b": "string, the exact verbatim text from Source B that contains the conflicting assertion",
      "entity_or_event": "string, the specific subject the conflict is about",
      "conflict_type": "enum, one of: 'direct_numerical', 'direct_factual', 'temporal_ordering', 'entity_identity'",
      "verdict": "string, a brief, machine-readable explanation of why these statements cannot both be true",
      "provenance": {
        "source_a_id": "[SOURCE_A_ID]",
        "source_b_id": "[SOURCE_B_ID]",
        "claim_a_char_start": "integer or null",
        "claim_a_char_end": "integer or null",
        "claim_b_char_start": "integer or null",
        "claim_b_char_end": "integer or null"
      }
    }
  ],
  "unresolvable_ambiguities": [
    {
      "claim_a": "string",
      "claim_b": "string",
      "reason": "string, explanation of why a conflict could not be definitively confirmed or ruled out"
    }
  ]
}

## CONSTRAINTS
- If no conflicts are found, return an empty 'conflicts' array.
- If a potential conflict is ambiguous due to missing context, place it in 'unresolvable_ambiguities', not 'conflicts'.
- For provenance fields, provide character start and end indices for the exact conflicting text within the original source strings. If you cannot determine the exact location, use null.
- Do not invent conflicts. Your precision is more important than recall.

To adapt this template, replace the square-bracket placeholders with your actual data before sending the request. The [SOURCE_A_TEXT] and [SOURCE_B_TEXT] fields should contain the full document bodies you are comparing. The [SOURCE_A_ID] and [SOURCE_B_ID] should be persistent, unique identifiers (like a document UUID or database key) that will travel with the output to maintain a clear chain of custody. In a production harness, you must validate the model's response against this JSON schema before accepting it. A common failure mode is the model omitting the provenance fields or adding commentary outside the JSON object; your application logic should retry or repair the output if schema validation fails. For high-stakes verification, always route outputs with a high number of conflicts or unresolvable ambiguities for human review before taking action.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Pairwise Source Comparison Prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to check the input at runtime before incurring model cost.

PlaceholderPurposeExampleValidation Notes

[SOURCE_A]

The first source document or text block to compare

According to the 2023 annual report, revenue grew 12%.

Must be a non-empty string. If the source is a document, include a stable identifier in [SOURCE_A_ID]. Truncate to the model's context window minus overhead.

[SOURCE_B]

The second source document or text block to compare

The press release stated revenue growth was 15% for the same period.

Must be a non-empty string. Ensure [SOURCE_A] and [SOURCE_B] are distinct inputs. If they are identical, the comparison is invalid and should be short-circuited.

[SOURCE_A_ID]

A stable identifier for the first source for provenance tracking

doc://annual-report-2023.pdf#page=4

Must be a non-empty string. Used in the output's provenance fields. If not available, use a generated UUID and log the mapping.

[SOURCE_B_ID]

A stable identifier for the second source for provenance tracking

doc://press-release-2024-01-15.txt

Must be a non-empty string. The ID should be resolvable back to the original source in your system of record.

[COMPARISON_FOCUS]

Optional. A specific claim, topic, or dimension to scope the comparison

Revenue growth rate for fiscal year 2023

If null, the prompt performs a general pairwise comparison. If provided, must be a non-empty string. Narrowing the focus reduces false positives from unrelated differences.

[OUTPUT_SCHEMA]

The strict JSON schema the model must conform to

See Conflict Output Contract definition

Must be a valid JSON Schema object. Validate that the schema includes required fields: conflict_type, claim_a, claim_b, verdict, confidence, provenance. Reject the prompt configuration if required fields are missing.

[CONFIDENCE_THRESHOLD]

Minimum confidence score for auto-accepting a conflict verdict

0.75

Must be a float between 0.0 and 1.0. Outputs below this threshold should be routed for human review. If null, default to 0.70. Do not set above 0.95 without human-in-the-loop fallback.

[MAX_TOKENS]

Hard limit on the model's response length

4096

Must be an integer. Set based on the expected size of the conflict output contract. If the output is truncated, the schema validation step will catch the malformed JSON and trigger a retry or repair.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Pairwise Source Comparison Prompt into a production application with validation, retries, and structured output guarantees.

The Pairwise Source Comparison Prompt is designed for API integration, not a one-off chat. Its primary value is the structured JSON conflict output contract, which downstream systems can parse, log, and act on. To use it in production, wrap the prompt in a service that manages the full lifecycle: input assembly, model invocation, output validation, retry logic, and result routing. The prompt itself expects two source texts ([SOURCE_A] and [SOURCE_B]), an optional [CLAIM_FOCUS] to narrow the comparison scope, and a [CONFLICT_OUTPUT_SCHEMA] that defines the exact JSON shape you need. The schema should include typed fields for the conflict verdict, conflicting excerpts, relationship classification, confidence score, and provenance links back to each source.

The implementation harness must enforce output contract compliance before the result reaches any consumer. After the model returns a response, run a JSON schema validator against the expected [CONFLICT_OUTPUT_SCHEMA]. Check that all required fields are present, enum values match allowed sets, confidence scores fall within the 0.0–1.0 range, and excerpt strings are non-empty when a conflict is flagged. If validation fails, implement a retry loop with a maximum of two additional attempts, appending the validation error to the prompt context so the model can self-correct. For high-stakes verification pipelines, route outputs with confidence scores in an ambiguous band (e.g., 0.4–0.6) to a human review queue with the full prompt context and raw model response attached. Log every invocation—including model, latency, token counts, validation pass/fail, and retry count—to enable prompt observability and regression testing over time.

Model choice matters for this workflow. The prompt requires precise, schema-adherent JSON generation and careful semantic comparison of potentially long source texts. Use models with strong instruction-following and structured output capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may hallucinate conflict fields or drop required provenance links. If your application compares sources that exceed the model's context window, pre-process with a chunking strategy that preserves paragraph boundaries and overlap, then run pairwise comparisons across aligned chunks and aggregate results. Do not use this prompt for real-time user-facing chat without first wrapping it in the validation harness—raw model output without schema enforcement will produce brittle, unparseable results in production.

IMPLEMENTATION TABLE

Expected Output Contract

Machine-readable JSON contract for pairwise source comparison. Use this table to validate output before ingestion into downstream verification pipelines.

Field or ElementType or FormatRequiredValidation Rule

comparison_id

string (UUID v4)

Must be a valid UUID v4 generated by the caller; reject if missing or malformed.

source_a

object

Must contain id (string), excerpt (string, max 2000 chars), and provenance (object with url or document_ref). Reject if excerpt is empty.

source_b

object

Same schema as source_a. Reject if id is identical to source_a.id (self-comparison).

conflict_verdict

enum string

Must be one of: direct_contradiction, partial_mismatch, compatible, unrelated. Reject any other value.

conflict_type

string or null

Required when verdict is direct_contradiction or partial_mismatch. Must be from allowed taxonomy: factual_inversion, numerical_disagreement, temporal_conflict, attribution_error, scope_mismatch. Null allowed otherwise.

confidence_score

number (0.0–1.0)

Must be a float between 0.0 and 1.0 inclusive. Reject if outside range. Values below 0.5 should trigger human review routing.

conflict_explanation

string

Must be non-empty, grounded in the provided excerpts, and cite specific phrases from source_a and source_b. Reject if explanation is generic or lacks excerpt anchoring.

resolution_status

enum string

Must be one of: unresolved, resolved_source_a_preferred, resolved_source_b_preferred, resolved_reconciled, requires_external_evidence. Reject if verdict is direct_contradiction and status is resolved_reconciled without explicit reconciliation logic.

PRACTICAL GUARDRAILS

Common Failure Modes

When deploying a pairwise source comparison prompt with a conflict output contract, these are the most common production failures and how to prevent them.

01

Paraphrasing Misclassified as Contradiction

Risk: The model flags semantically equivalent statements as contradictions because they use different wording, synonyms, or sentence structures. This inflates false positives and erodes trust in the conflict detection pipeline. Guardrail: Include few-shot examples of paraphrased vs. genuinely contradictory statement pairs in the prompt. Add a pre-check instruction: 'Before labeling a contradiction, verify the core factual claim differs, not just the phrasing.' Use an LLM judge eval calibrated on paraphrase-contradiction boundary cases.

02

Missing Required Fields in Structured Output

Risk: The model returns JSON that passes initial parsing but omits mandatory fields like conflict_type, provenance, or confidence_score, breaking downstream consumers that expect complete records. Guardrail: Validate output against the JSON schema immediately after generation. Implement a completeness check that rejects records with null or missing required fields. Use a retry prompt that echoes the missing field names and requests regeneration. Log schema violations by field for monitoring.

03

Context Window Truncation Silently Drops Evidence

Risk: When comparing long source documents, the model's context window may truncate one or both sources mid-content, producing contradiction verdicts based on incomplete evidence without signaling the truncation. Guardrail: Chunk documents before comparison and run pairwise checks on aligned segments. Add a source_completeness field to the output contract requiring the model to confirm both sources were fully processed. If the combined token count exceeds the safe window, split into multiple comparison calls and aggregate results.

04

Temporal Context Ignored in Conflict Classification

Risk: The model flags statements as contradictory when they are actually true at different points in time (e.g., 'revenue grew 10% in Q1' vs. 'revenue grew 5% in Q2'). Without temporal awareness, valid time-series data appears inconsistent. Guardrail: Require the prompt to extract and compare any associated timestamps, dates, or period references before classifying a conflict. Add a temporal_scope field to the output schema. Include temporal normalization rules in the prompt for relative dates ('last quarter', 'this year').

05

Confidence Scores Are Uncalibrated and Overconfident

Risk: The model assigns high confidence to incorrect contradiction verdicts, especially on ambiguous or domain-specific claims where it lacks expertise. Downstream systems treat these as authoritative, amplifying errors. Guardrail: Calibrate confidence thresholds against a human-labeled golden dataset. Add a requires_human_review boolean flag triggered when confidence falls below a calibrated threshold or when the claim domain is outside the model's known strengths. Log confidence distributions in production to detect drift.

06

Scope Mismatch Produces False Contradictions

Risk: Two sources make claims about the same topic but at different scopes (e.g., global vs. regional statistics, different product lines, different time periods), and the model incorrectly flags them as contradictory because it fails to detect the scope difference. Guardrail: Add a scope_context extraction step before contradiction classification. Instruct the model to identify and compare the scope of each claim (geography, product, population, timeframe) and only proceed to contradiction analysis when scopes overlap. Include scope-mismatch examples in few-shot demonstrations.

IMPLEMENTATION TABLE

Evaluation Rubric

Criteria for testing the Pairwise Source Comparison Prompt before production deployment. Each row defines a pass standard, a failure signal, and a concrete test method.

CriterionPass StandardFailure SignalTest Method

Schema Compliance

Output is valid JSON matching the [OUTPUT_SCHEMA] with no missing required fields.

JSON parse error, missing required field, or extra unexpected field.

Validate output against the JSON schema using a programmatic validator. Run 50 test pairs.

Conflict Classification Accuracy

Correctly labels direct contradictions, partial mismatches, and compatible statements in >= 90% of test cases.

Misclassifies a paraphrased statement as a contradiction or misses a direct numerical disagreement.

Run against a golden dataset of 100 labeled claim pairs. Calculate precision and recall per conflict type.

Provenance Link Integrity

Every conflict record includes a non-null [SOURCE_A_ID] and [SOURCE_B_ID] that match the input.

Null or empty source IDs, or IDs that do not correspond to the provided source metadata.

Parse output and cross-reference all source IDs against the input source list. Check for dangling references.

Evidence Excerpt Grounding

The [EVIDENCE_EXCERPT_A] and [EVIDENCE_EXCERPT_B] fields contain verbatim substrings from the provided source texts.

Excerpts are hallucinated, paraphrased, or cannot be located in the original source text.

Perform a fuzzy substring search for each excerpt in its respective source document. Flag any excerpt with a similarity ratio < 0.9.

Conflict Severity Calibration

Severity score matches the rubric definition (e.g., 'high' for factual contradictions, 'low' for stylistic differences) in >= 85% of cases.

A minor date format difference is scored as 'high' severity, or a direct factual negation is scored as 'low'.

Use an LLM-as-judge with the severity rubric to re-score a sample of 50 outputs. Measure agreement with the prompt's output.

Abstention on Unrelated Inputs

Returns a null or empty conflict list when two sources discuss completely unrelated topics.

Fabricates a conflict by over-interpreting or forcing a connection between unrelated statements.

Provide 20 pairs of documents on different topics. The test passes if the output conflict list is empty for all 20.

Handling of Identical Inputs

Returns a single conflict record with type 'compatible' or 'identical' and no false contradiction flag.

Flags the identical text as a 'direct contradiction' or 'partial mismatch'.

Input the same source text for both [SOURCE_A] and [SOURCE_B]. Verify the output type is not a contradiction class.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a frontier model (GPT-4o, Claude 3.5 Sonnet). Strip the JSON schema validation from your harness and accept raw model output for initial experimentation. Focus on getting the conflict type classification and evidence excerpts right before adding strict field completeness checks.

Watch for

  • Missing conflict_type enum values when the model invents categories
  • Evidence excerpts that paraphrase instead of quoting verbatim
  • provenance fields that reference document titles instead of stable IDs or chunk indices
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.