Multi-Hop Fact Verification Query Chain Prompt

The claim or statement to verify. Contains one or more factual assertions that may require independent evidence.

"The same gene variant linked to increased risk of Alzheimer's also appears to reduce the risk of certain cancers, according to recent studies."

Must be a non-empty string with at least one verifiable factual claim. Reject inputs that are purely opinion, subjective preference, or single-word queries. Check for compound claims that require decomposition.

Defines the output structure for individual claims extracted from the statement. Each claim becomes a separate verification target.

{"claim_id": "string", "claim_text": "string", "entities": ["string"], "verifiable": "boolean", "requires_temporal_scope": "boolean"}

Must be a valid JSON Schema object. Each claim record must include an id, text, and verifiable flag. Schema must support arrays of claims. Validate that the schema does not collapse distinct claims into a single record.

Defines the output structure for the full verification plan, including query chains and cross-reference rules.

{"plan_id": "string", "claims": [ClaimDecomposition], "query_chains": [{"claim_id": "string", "hops": [{"hop_number": "int", "query": "string", "target_source_type": "string", "expected_evidence_type": "string"}]}], "cross_validation_pairs": [{"claim_a_id": "string", "claim_b_id": "string", "relationship": "string"}]}

Must be a valid JSON Schema object. Query chains must map to claim IDs from the decomposition. Cross-validation pairs must reference valid claim IDs. Reject schemas that allow orphan queries or cross-references to non-existent claims.

Constraints that ensure each hop in a query chain retrieves evidence from sources independent of prior hops. Prevents circular verification.

"Each hop must target a different source domain or publisher than prior hops in the same chain. No hop may query the same URL or document identifier as a previous hop."

Must be a non-empty string or structured rules object. Rules must be parseable into executable checks. Validate that rules cover domain diversity, URL uniqueness, and publisher separation. Reject rules that allow self-referencing verification loops.

Upper bound on the number of retrieval hops per claim to prevent unbounded query chains and control latency.

3

Must be a positive integer between 1 and 10. Default to 3 if not specified. Validate that the value is within range before sending. Values above 5 should trigger a review flag due to latency and cost implications.

Optional date range or recency requirements that scope all generated queries. Ensures evidence is time-bounded.

{"start_date": "2023-01-01", "end_date": "2024-12-31", "prefer_recent": true}

If provided, must be a valid JSON object with ISO 8601 date strings. Null allowed. Validate that start_date precedes end_date. If prefer_recent is true, generated queries should include temporal filters. Reject malformed date strings.

Specifies whether each claim must be grounded to a retrievable source, or if the model can note when evidence is unavailable.

"Every claim must have at least one source citation. Claims without retrievable evidence must be flagged as UNVERIFIED with a reason."

Must be a non-empty string. Validate that the instruction explicitly addresses the UNVERIFIED case. Reject instructions that allow claims to pass without either a citation or an explicit unverified flag.

Language for generated queries and verification plan annotations. Queries should match the language of target retrieval corpora.

"en"

Must be a valid ISO 639-1 two-letter language code. Default to "en" if not specified. Validate against a supported language list. Reject unsupported codes that would cause retrieval mismatches.

object

Top-level object must contain claims array and cross_validation_queries array

array

Array length must be >= 1. Each element must have claim_text, sub_queries, and source_independence_score fields

string

Non-empty string. Must be a distinct extractable proposition from the input statement. Parse check: no duplicate claim_text values across array

array

Array length must be >= 1. Each element must be a non-empty string representing a retrievable query. No sub_query may be identical to the claim_text itself

number

Float between 0.0 and 1.0. Score represents how independently this claim can be verified from other claims. 1.0 = fully independent. Schema check: value must be >= 0.0 and <= 1.0

array

Array length must be >= 1. Each element must have query_text and target_claim_indices fields

string

Non-empty string. Must differ from all sub_queries in the claims array. Parse check: no exact string match against any claims[].sub_queries[] value

array

Array of integers >= 0. Each index must reference a valid index in the claims array. Array length must be >= 1. Schema check: no index out of bounds

Every factual claim in [STATEMENT] is extracted and assigned at least one verification query. No claim is skipped or merged into another claim.

Output misses a verifiable claim present in the input statement. A claim is paraphrased into a different meaning.

Diff the set of extracted claims against a human-annotated claim list for 20 test statements. Require 100% recall on unambiguous factual claims.

Each verification query targets a distinct retrieval need. No two queries are semantic duplicates or minor rewordings that would retrieve the same top-10 documents.

Two or more queries share >80% token overlap or produce identical retrieval results in a controlled test index.

Run each generated query against a fixed test corpus. Measure Jaccard similarity of top-10 result sets per query pair. Flag pairs above 0.7 similarity.

Cross-reference queries combine two or more previously independent claims into a single retrieval that tests their joint truth. Each cross-reference query is non-redundant with individual claim queries.

Cross-reference query is a trivial concatenation of two claim queries. Cross-reference query introduces a new claim not present in the original statement.

Manually inspect cross-reference queries against the claim list. Require that each cross-reference query references at least two claim IDs from the decomposition output.

Each query contains enough detail to retrieve relevant evidence without being so narrow that it returns zero results. Queries include key entities, dates, or constraints from the claim.

Query is a verbatim copy of the claim with no retrieval-oriented reformulation. Query is so vague it returns generic top-level pages.

Execute each query against a test retrieval index. Require mean reciprocal rank of a relevant document within top-10 results to be above 0.5 across a 50-statement test set.

Output is valid JSON matching the [OUTPUT_SCHEMA]. All required fields are present. Claim IDs are unique and sequential. Query list is non-empty.

Missing required field. Duplicate claim IDs. Query list is empty when statement contains verifiable claims. JSON parse error.

Validate output with a JSON Schema validator. Run across 100 test statements. Require 100% schema compliance. Reject any output that fails jsonschema.validate().

Every extracted claim is directly attributable to a phrase or clause in [STATEMENT]. No claim is invented, inferred, or added from external knowledge.

Output contains a claim that cannot be mapped back to a specific span in the input statement. Claim adds quantification not present in the source.

Require human annotators to map each extracted claim to a source span. Flag any claim without a span mapping. Target 0% hallucinated claims on a 50-statement test set.

The verification plan includes all required sections: claim decomposition, individual claim queries, cross-reference queries, and a suggested execution order. No section is truncated or omitted.

Cross-reference section is empty when statement contains multiple related claims. Execution order is missing or contradicts stated dependencies.

Check for presence of all top-level keys in the output object. Run a structural completeness check that asserts non-empty arrays for claim list and query list. Apply across 100 test statements.

When a claim is ambiguous or underspecified in [STATEMENT], the output notes the ambiguity and generates queries that cover multiple interpretations. Confidence is not asserted when evidence is likely to be mixed.

Output assigns high confidence to a claim that is clearly underspecified. Ambiguous claim is treated as a single precise claim without noting alternatives.

Include 10 deliberately ambiguous statements in the test set. Require that outputs for these statements include at least one ambiguity note or multiple interpretation queries. Flag outputs that treat ambiguous claims as single precise claims.