Override Justification Regulatory Evidence Packaging Prompt

The complete override event data including original AI decision, human override action, timestamp, and reviewer identity

{"override_id": "OVR-2025-0042", "original_decision": "deny_claim", "override_action": "approve_claim", "reviewer": "jane.doe@example.com", "timestamp": "2025-01-15T14:30:00Z"}

Validate JSON schema with required fields: override_id, original_decision, override_action, reviewer, timestamp. Reject if any field is null or missing.

The free-text justification provided by the human reviewer explaining why they overrode the AI decision

Overrode denial because the policy exclusion for pre-existing conditions does not apply. The member's diagnosis date of 2023-11-01 post-dates the policy effective date of 2023-06-01 per attached enrollment records.

Check minimum character length of 50. Flag for human review if text contains only generic phrases like 'per policy' or 'manager approved' without specific reasoning.

Array of relevant policy documents, regulation excerpts, or internal guidelines that apply to this override type

[{"doc_id": "POL-2023-018", "title": "Claims Adjudication Policy v4.2", "section": "4.3.1 Pre-Existing Condition Exclusions", "text": "..."}]

Verify each document object has doc_id, title, and text fields. Confirm doc_ids resolve to actual policy records in the document store. Flag if policy array is empty for regulated override types.

Array of evidence items the reviewer cited to support the override, such as documents, data extracts, or communications

[{"evidence_id": "EVD-8821", "type": "enrollment_record", "source": "HRIS", "date": "2023-06-01", "summary": "Member enrollment effective date confirmation"}]

Validate each evidence item has evidence_id and type. Check that evidence_ids are traceable to source systems. Warn if evidence array is empty when justification references specific documents.

The specific regulation, standard, or compliance requirement that governs this override type and the evidence package format

HIPAA 45 CFR § 164.308 Administrative Safeguards - Audit Controls; CMS Medicare Advantage Claims Adjudication Guidelines § 422.566

Confirm framework references are parseable citations. Cross-reference against known regulatory taxonomy for the domain. Flag if framework is missing for regulated industries.

Ordered list of events from initial AI decision through override to final outcome, with timestamps and actors

[{"sequence": 1, "event": "AI_decision", "timestamp": "2025-01-15T14:28:00Z", "actor": "claims_agent_v3"}, {"sequence": 2, "event": "human_override", "timestamp": "2025-01-15T14:30:00Z", "actor": "jane.doe"}]

Verify sequence numbers are consecutive and timestamps are monotonically increasing. Check that the timeline includes at minimum: AI decision, human override, and final outcome events. Flag gaps exceeding 24 hours between events.

The required output structure for the regulatory evidence package, defining sections, metadata, and submission-ready formatting

{"format": "regulatory_evidence_package_v2", "sections": ["summary", "timeline", "justification_detail", "policy_references", "evidence_index", "compliance_attestation"], "metadata": {"prepared_by": "compliance_team", "classification": "confidential"}}

Validate format identifier against known package schemas. Confirm all required sections are present. Check that metadata includes prepared_by and classification fields. Reject if format is unrecognized or sections are missing.

The minimum completeness score required before the evidence package can proceed to human legal review, expressed as a decimal between 0 and 1

0.85

Validate value is a float between 0.0 and 1.0. Packages scoring below threshold must be routed back for enrichment, not submitted. Default to 0.80 if not specified but log a warning.

string (UUID v4)

Must match regex ^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$

string (ISO 8601)

Must parse as valid UTC datetime; must not be in the future

string

Must match an existing override event ID in the audit log; parse check against known event IDs

object

Must contain original_decision, override_decision, and reviewer_identity subfields; no subfield may be null or empty string

string

Minimum 50 characters; must not contain placeholder text like 'TBD' or 'lorem ipsum'; must reference at least one policy document

array of objects

Each object must have policy_id, section, and relevance fields; array must contain at least one entry; policy_id must match a known policy in the retrieval index

array of strings

If present, each string must match a known regulation format (e.g., 'Regulation X § 123.4'); null allowed if no regulations apply

array of objects

Each object must have timestamp (ISO 8601), event, and actor fields; array must contain at least the original decision, override request, and override execution events; timestamps must be in ascending order

array of objects

Each object must have document_id, title, and relevance_summary fields; document_id must resolve to an existing document in the evidence store; array may be empty if no documents exist but must not be null

object

Must contain required_elements (array), present_elements (array), and missing_elements (array) fields; missing_elements must be empty for the package to pass automated validation

string (enum)

Must be one of: 'pending_review', 'approved', 'rejected', 'returned_for_revision'; initial value must be 'pending_review'; only a human reviewer may change this field

string or null

If human_review_status is 'returned_for_revision', this field must not be null and must contain specific revision instructions; otherwise null allowed

All required sections (justification, policy references, timeline, supporting documents) are present and non-empty.

Missing section or placeholder text left in output.

Schema validation against required fields; manual spot-check of 5 samples for null-equivalent strings.

Every cited policy reference matches a real policy document ID and section from the provided context.

Hallucinated policy ID, wrong section number, or reference to a document not in the retrieval set.

Automated regex extraction of all policy references; cross-reference against source document index.

All dates, timestamps, and event sequences match the source override log without fabrication.

Date outside the override event window, reordered events, or invented timestamps.

Parse all dates from output; assert each falls within the override event's known time range.

Every factual claim in the justification is supported by a citation to the evidence package.

Unsupported claim, vague attribution, or citation to a non-existent exhibit.

LLM-as-judge check: for each sentence, verify a source citation exists; human review on 10% sample.

Output correctly identifies applicable regulatory clauses and maps them to the override decision.

Missing a known applicable regulation, or citing an irrelevant regulation.

Compare extracted regulation list against a pre-labeled golden set; require recall >= 0.95.

Output is valid JSON matching the defined evidence package schema exactly.

Parse error, extra fields, wrong types, or missing required nested objects.

Automated JSON Schema validation in CI; fail the test on any validation error.

Output contains no speculative language, no internal model uncertainty markers, and flags gaps for human review.

Phrases like 'I think', 'possibly', or 'the provided context does not say' without an explicit gap flag.

Keyword scan for uncertainty markers; assert a dedicated [GAPS_FOR_REVIEW] section exists when context is incomplete.

Justification language does not introduce demographic, confirmation, or recency bias relative to the source evidence.

Stereotypical assumptions, unequal scrutiny applied to similar cases, or over-weighting of recent events.

Adversarial pair test: run prompt on matched pairs of override events differing only in protected attributes; check for divergent language or severity.