Fabricated Product Feature Detection Prompt

This prompt is designed for product documentation and competitive analysis teams who need to verify that model-generated content about a product—its features, capabilities, and specifications—is actually attested in the provided source documentation. The core job-to-be-done is preventing fabricated feature claims from reaching internal stakeholders, public-facing documentation, or competitive intelligence reports. The ideal user is a technical product manager, a competitive analyst, or a documentation engineer who has access to authoritative source material and needs a structured, auditable verification step before trusting or publishing AI-generated product descriptions.

Use this prompt when you have a set of source documents (product spec sheets, API references, official datasheets, release notes) and a model-generated output that makes claims about what a product can do. The prompt works best when the source material is complete and unambiguous—it will flag features that appear in the output but cannot be matched to any source passage. Do not use this prompt when the source documentation is incomplete, outdated, or itself speculative; the prompt cannot distinguish between a genuine documentation gap and a model hallucination. It is also unsuitable for evaluating subjective product qualities like 'ease of use' or 'market-leading performance' unless those claims are explicitly attested in the source material.

The prompt requires a specific input harness: a clearly delimited [SOURCE_DOCUMENTATION] block containing the authoritative text, and a [MODEL_OUTPUT] block containing the claims to verify. Each flagged feature receives a confidence score and a recommended action (verify or remove). For high-stakes workflows—such as publishing competitive comparisons or updating customer-facing documentation—always route outputs flagged as 'verify' to a human subject-matter expert before taking action. The prompt is a detection tool, not a replacement for product knowledge. Wire it into a post-generation validation step in your content pipeline, log every flag for auditability, and track false-positive rates over time to calibrate your confidence thresholds.

This prompt template is designed to be dropped into a post-generation validation step. It takes a model's output about a product and a set of source documents, then identifies any features, capabilities, or specifications that appear in the output but are not attested in the source material. The prompt is structured to return a machine-readable JSON report so that downstream systems can programmatically flag, remove, or escalate fabricated content before it reaches users or databases.

textCopy
You are a product documentation auditor. Your task is to compare a model-generated product description against a set of authoritative source documents and identify any features, capabilities, or specifications that are fabricated—meaning they appear in the output but are not supported by the source material.

## INPUT

**Source Documents:**
[SOURCE_DOCUMENTS]

**Model Output to Audit:**
[MODEL_OUTPUT]

**Product Context (optional):**
[PRODUCT_CONTEXT]

## INSTRUCTIONS

1. Extract every discrete product feature, capability, specification, or claim from the model output.
2. For each extracted item, search the source documents for supporting evidence.
3. Classify each item as one of:
   - **ATTESTED**: Directly supported by source text with a quotable passage.
   - **PARTIALLY_ATTESTED**: Partially supported but the output adds unsupported detail, scope, or quantification.
   - **FABRICATED**: No supporting evidence found in source documents.
   - **CONTRADICTED**: Source documents explicitly state something contrary to the output.
4. Assign a confidence score (0.0 to 1.0) to each classification.
5. For ATTESTED and PARTIALLY_ATTESTED items, include the exact source passage as evidence.
6. For FABRICATED and CONTRADICTED items, include a recommended action: REMOVE, FLAG_FOR_REVIEW, or ESCALATE.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:

{
  "audit_summary": {
    "total_claims_found": <integer>,
    "attested_count": <integer>,
    "partially_attested_count": <integer>,
    "fabricated_count": <integer>,
    "contradicted_count": <integer>,
    "overall_fabrication_risk": "LOW" | "MEDIUM" | "HIGH" | "CRITICAL"
  },
  "findings": [
    {
      "claim_id": <string, unique identifier>,
      "claim_text": <string, the exact claim from the output>,
      "classification": "ATTESTED" | "PARTIALLY_ATTESTED" | "FABRICATED" | "CONTRADICTED",
      "confidence": <float 0.0-1.0>,
      "source_evidence": <string or null, quoted source passage>,
      "discrepancy_note": <string or null, explanation of partial attestation or contradiction>,
      "recommended_action": "KEEP" | "REMOVE" | "FLAG_FOR_REVIEW" | "ESCALATE",
      "action_rationale": <string, brief reason for the recommendation>
    }
  ]
}

## CONSTRAINTS

- Do not invent source evidence. If a source passage does not exist, set source_evidence to null.
- If the model output contains no product claims at all, return an empty findings array with appropriate audit_summary counts set to zero.
- Do not classify stylistic choices, tone, or marketing language as features unless they make specific capability claims.
- When confidence is below 0.7, default to FLAG_FOR_REVIEW rather than REMOVE.
- Treat the source documents as the single source of truth. Do not bring external knowledge about the product.

## EXAMPLES

[EXAMPLES]

## RISK LEVEL

[RISK_LEVEL]

To adapt this template, replace the square-bracket placeholders with your actual inputs. [SOURCE_DOCUMENTS] should contain the full text of your authoritative product documentation, spec sheets, or knowledge base articles—concatenated with clear document boundary markers. [MODEL_OUTPUT] is the text you need to audit. [PRODUCT_CONTEXT] is optional but useful for providing the product name, version, or category to help the model disambiguate. [EXAMPLES] should include 2-4 few-shot examples showing correct classifications, especially for the tricky PARTIALLY_ATTESTED and CONTRADICTED categories. [RISK_LEVEL] should be set to "LOW", "MEDIUM", "HIGH", or "CRITICAL" to adjust the model's sensitivity—higher risk levels should make the model more conservative, flagging borderline cases for review rather than letting them pass. For high-stakes domains like regulated product claims or competitive analysis, always route FABRICATED and CONTRADICTED findings through human review before taking automated action.

This prompt is designed to operate as a post-generation validation step within a larger product documentation or competitive analysis pipeline. It should never be the sole gatekeeper for published content. Instead, wire it as a dedicated verification service that receives a model's output and the corresponding source documentation, returning a structured audit report. The implementation must enforce that both [MODEL_OUTPUT] and [SOURCE_DOCUMENTATION] are non-negotiable inputs—the prompt cannot function without them. If your application generates feature lists or capability summaries from a model, insert this prompt between the generation step and any user-facing surface or database write operation.

The harness should wrap the LLM call in a retry loop with a strict JSON schema validator. Define the expected output schema (an array of feature objects, each with feature_name, source_evidence, confidence_score, and recommendation fields) and reject any response that fails to parse or omits required fields. On validation failure, retry up to two times with an escalated [CONSTRAINTS] block that explicitly states the previous output was malformed. Log every raw response and its validation status for observability. For high-stakes use cases—such as publishing competitive battlecards or updating product pages—route all outputs with a confidence_score below 0.9 or a recommendation of verify or remove to a human review queue. This prompt is a detection tool, not an automated removal tool; the application layer should handle the actual stripping or flagging of content based on the prompt's structured verdict.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may struggle with the nuanced distinction between a feature implied by context and one outright invented. Do not use this prompt in a streaming context; you need the complete output to perform the comparison. If your source documentation is long, chunk it and run the prompt per section, then deduplicate the results. Finally, treat the prompt's output as evidence for a decision, not the decision itself—always build a human escalation path for any flagged fabrication before content reaches customers.