Spurious Relationship Pruning Prompt for Entity Extraction

This prompt operates as a post-extraction validation gate for entity resolution and graph construction pipelines. After an upstream model extracts entities and relationships from source text, this prompt reviews every proposed relationship edge and determines whether the source text actually supports the connection. Use it when your pipeline cannot tolerate hallucinated edges entering a knowledge graph, when downstream analytics depend on relationship accuracy, or when you need an audit trail of pruning decisions. This is not a replacement for the initial extraction prompt. It assumes you already have a set of extracted triples or relationship records and need to filter them before ingestion.

The ideal user is an engineering lead or data pipeline operator who already has a working extraction step but observes fabricated relationships in production—entities connected by relationships the model inferred rather than extracted. You should have the original source text and the extracted relationship set available as inputs. This prompt works best when each relationship carries a confidence marker or extraction rationale from the upstream model, giving the pruning step more signal to evaluate. Do not use this prompt when you lack the source text for comparison, when the extraction model already provides reliable grounding scores, or when your graph tolerates probabilistic edges with explicit uncertainty labels. In those cases, a simpler threshold filter or a human review queue may be more appropriate than a secondary LLM call.

Before wiring this into production, define your tolerance for false positives (removing valid relationships) versus false negatives (keeping hallucinated ones). The prompt includes a [CONSERVATIVENESS] parameter that lets you tune this trade-off. For high-stakes domains like legal contract analysis or clinical entity extraction, set conservativeness high and route borderline cases to human review. For exploratory knowledge graph construction where some noise is acceptable, lower conservativeness will preserve more edges. Always log pruning decisions with the justification text this prompt produces—that audit trail becomes essential when downstream consumers question why a relationship is missing from the graph.

This prompt template is designed to sit between your entity extraction step and your graph ingestion step. It receives a list of extracted entities and their proposed relationships, along with the source text that should ground those relationships. The model's job is to audit each relationship, determine whether it is supported by the source text, and return only the relationships that have clear textual evidence. Use this when your extraction pipeline produces relationship edges that downstream systems treat as fact—such as knowledge graphs, investigative link charts, or compliance databases.

textCopy
You are an evidence auditor for an entity extraction pipeline. Your task is to review a set of proposed relationships between entities and prune any relationship that is not directly supported by the provided source text.

## INPUT
Source Text:
[SOURCE_TEXT]

Extracted Entities:
[ENTITIES_JSON]

Proposed Relationships:
[RELATIONSHIPS_JSON]

## INSTRUCTIONS
1. For each proposed relationship, locate the exact span of text in the source that supports it. If no supporting span exists, mark the relationship for removal.
2. A relationship is supported only if the source text explicitly states or strongly implies the connection. Do not infer relationships from world knowledge, common sense, or statistical association.
3. If a relationship is partially supported (e.g., the entities co-occur but the specific relationship type is not stated), mark it as UNSUPPORTED.
4. Do not add new relationships. Only prune or keep the provided set.
5. If the source text is insufficient to verify any relationship, return an empty list and explain why.

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "kept_relationships": [
    {
      "source_entity": "string",
      "target_entity": "string",
      "relationship_type": "string",
      "evidence_span": "string (exact quote from source)",
      "confidence": "HIGH | MEDIUM"
    }
  ],
  "removed_relationships": [
    {
      "source_entity": "string",
      "target_entity": "string",
      "relationship_type": "string",
      "removal_reason": "NO_EVIDENCE | WEAK_IMPLICATION | ENTITY_NOT_IN_SOURCE | CONTRADICTED_BY_SOURCE"
    }
  ],
  "audit_summary": "Brief explanation of pruning decisions"
}

## CONSTRAINTS
- Do not fabricate evidence spans. If you cannot find a supporting quote, the relationship must be removed.
- Preserve the original entity identifiers exactly as provided in [ENTITIES_JSON].
- If the source text is empty or unrelated to the entities, return all relationships as removed with reason NO_EVIDENCE.
- For [RISK_LEVEL] = HIGH, require explicit, unambiguous textual support. For MEDIUM, allow strong implication.

Adaptation guidance: Replace [SOURCE_TEXT] with the raw document or passage that should ground the relationships. [ENTITIES_JSON] should contain the canonical entity list with IDs and types from your extraction step. [RELATIONSHIPS_JSON] should contain the proposed triples (source_entity, target_entity, relationship_type) that your extraction model produced. Set [RISK_LEVEL] to HIGH for regulated domains (legal, clinical, financial) where fabricated relationships carry compliance risk, or MEDIUM for exploratory graph construction where some noise is tolerable. The output schema is designed to feed directly into a graph database ingestion step—kept relationships go in, removed relationships go to a review queue or audit log.

Before shipping: Wire this prompt into a post-extraction validation harness. After the model returns the pruned relationship set, validate that every kept relationship's evidence_span is a substring of [SOURCE_TEXT] using an exact string match or fuzzy substring check. Log all removed relationships with their reasons for pipeline observability. For high-risk domains, route removed relationships with reason CONTRADICTED_BY_SOURCE to human review—these may indicate extraction errors upstream. Run this prompt against a golden dataset of source texts with known entity relationships to measure precision (did we keep only true relationships?) and recall (did we accidentally prune real ones?). Common failure mode: the model marks a relationship as supported but provides an evidence span that is a paraphrase rather than a quote—your substring validator will catch this.

This prompt is a post-extraction filter, not a standalone entity resolver. It expects a list of extracted relationships and their source texts as input. In production, wire it after your entity extraction step and before writing to your graph database or knowledge store. The prompt's job is to reject edges the model invented, not to add missing relationships. Keep the extraction step and the pruning step separate so you can tune each independently and log pruning decisions for audit.

Build a thin validation layer around the prompt call. The output schema should be a JSON array of relationship objects, each with relationship, verdict (one of PRESERVE or PRUNE), and evidence fields. Validate that every input relationship ID appears in the output exactly once, that no new IDs are introduced, and that verdict values are constrained to the allowed enum. If validation fails, retry once with the validation error injected into the prompt context. If the retry also fails, route the batch to a human review queue and log the failure with the raw model response and validator output. For high-stakes domains like legal or clinical entity graphs, require human approval on all PRUNE decisions before edges are removed from the graph.

Model choice matters here. This prompt requires strong instruction-following and low hallucination on structured comparison tasks. Claude 3.5 Sonnet and GPT-4o are good defaults. Avoid smaller or older models that may invent relationship verdicts or drop input entries. Set temperature to 0 and disable sampling to maximize deterministic pruning behavior. If you are processing large relationship batches, chunk the input into groups of 20-30 relationships per call to stay within reliable attention windows and to make retry scope manageable. Log every pruning decision with the model version, timestamp, and input hash so you can trace regressions if a model update changes pruning behavior.

Observability is critical because silent over-pruning breaks downstream graph queries. Emit metrics for: total relationships processed, number pruned, number preserved, validation failure rate, and retry rate. Set an alert if the prune ratio spikes above a historical baseline, which often indicates a prompt regression or a model behavior change. Store the full prompt and response for every call in a structured log so you can replay failures during debugging. If you are building a graph for search or analytics, consider keeping pruned relationships in a separate 'rejected edges' table with their evidence and verdict for periodic human audit rather than deleting them outright.