Inferensys

Prompt

Regulatory Filing Evidence Extraction and Ranking Prompt

A practical prompt playbook for extracting and ranking obligations, definitions, and exceptions from regulatory filings in production AI workflows. Designed for RegTech and compliance product teams who need structured compliance evidence maps with human-review gates.
Wide-angle shot of a modern WeWork open floor plan with creative walls covered in AI system architecture diagrams, product team collaborating in standing desk area with industrial lighting.
PROMPT PLAYBOOK

When to Use This Prompt

Defines the job-to-be-done, ideal user, required context, and boundaries for the regulatory filing evidence extraction and ranking prompt.

This prompt is designed for RegTech and compliance product teams that need to extract structured compliance evidence from regulatory filings. It ranks obligations, definitions, and exceptions by applicability, effective date, and penalty severity. The output is a compliance evidence map that distinguishes black-letter law from interpretive guidance. This prompt belongs in a pipeline where human review gates are mandatory for any output labeled as interpretive rather than explicit regulatory text. Use this when your product needs to surface the most actionable regulatory requirements from dense filings, not when you need a general summary or a conversational Q&A interface.

The ideal user is a compliance officer, legal engineer, or product manager building an AI-assisted regulatory intelligence tool. The required context includes the full text of one or more regulatory filings, a defined jurisdiction, and a target compliance domain (e.g., anti-money laundering, data privacy, emissions reporting). The prompt expects a structured output schema with fields for obligation text, source citation, effective date, penalty severity, and a binary classification of 'black-letter' versus 'interpretive.' Without these inputs, the model will produce vague, unranked, or legally unreliable output. The prompt is not suitable for real-time legal advice, contract negotiation, or any workflow where the model's output is shown directly to a regulator without human review.

Do not use this prompt when the regulatory text is highly fragmented across hundreds of documents without a retrieval step. This prompt assumes the relevant filing text is already provided in the context window. If you need to search across a large corpus, pair this prompt with a retrieval-augmented generation (RAG) pipeline that first surfaces candidate passages, then feeds them into this ranking prompt. Also avoid this prompt for regulations that are still in draft or comment period, as the model may treat proposed rules as final. Always implement a human-review gate for any output row tagged as 'interpretive,' and run an eval that checks whether extracted obligations are verbatim matches to source text or model paraphrases. Paraphrased obligations without source grounding should be flagged and re-reviewed before ingestion into a compliance system of record.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works, where it fails, and the operational preconditions required before putting it into a production RegTech pipeline.

01

Good Fit: Structured Regulatory Filings

Use when: Input documents follow a known structure such as SEC 10-K, FDA 510(k), or EU MDR technical documentation. The prompt performs best when sections, headings, and defined terms are machine-parseable. Guardrail: Pre-process filings to extract section boundaries and pass them as structured context alongside raw text.

02

Bad Fit: Interpretive Guidance Without Black-Letter Text

Avoid when: The filing contains only agency commentary, Q&A, or non-binding guidance without explicit obligations, definitions, or penalty language. The prompt will hallucinate obligations where none exist. Guardrail: Use a pre-filter to classify documents as black-letter law versus interpretive guidance before invoking this prompt.

03

Required Input: Effective Date and Jurisdiction Metadata

Risk: Without explicit effective-date and jurisdiction metadata, the prompt cannot rank by temporal applicability, producing stale or inapplicable obligations. Guardrail: Always supply [EFFECTIVE_DATE] and [JURISDICTION] as explicit input fields. Validate that retrieved passages contain date references before ranking.

04

Operational Risk: Penalty Severity Overranking

Risk: The prompt may overweight penalty severity and underweight applicability, causing teams to prioritize high-penalty but low-relevance obligations. Guardrail: Implement a two-pass ranking: first filter by applicability to the entity, then rank by penalty severity within applicable obligations only.

05

Operational Risk: Cross-Reference Blindness

Risk: Regulatory filings frequently reference obligations defined in other sections or external documents. The prompt may miss these dependencies and produce incomplete obligation maps. Guardrail: Run a pre-processing step to resolve cross-references and inline the referenced text before evidence ranking.

06

Human Review Gate: Black-Letter vs. Interpretive Classification

Risk: The model cannot reliably distinguish between a binding obligation and agency interpretive guidance. Treating guidance as obligation creates compliance liability. Guardrail: Every extracted obligation must pass through a human-review queue with a mandatory classification tag: black-letter, interpretive, or ambiguous. Log reviewer decisions for audit.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A production-ready prompt template for extracting and ranking regulatory obligations from filings, with strict JSON output and human-review gates.

This template is designed for RegTech and compliance product teams that need to extract structured obligations, definitions, and exceptions from regulatory filings. It accepts the full text of a regulatory document and a compliance domain context, then outputs a ranked evidence map with applicability, effective dates, and penalty severity. The prompt enforces strict JSON output, requires explicit evidence grounding for every extracted item, and distinguishes between black-letter law and interpretive guidance. Before integrating this into a production pipeline, review the failure modes and evaluation criteria in the full playbook.

text
You are a regulatory compliance evidence extraction system. Your task is to read a regulatory filing and extract obligations, definitions, exceptions, and interpretive guidance. You must produce a structured JSON output that ranks each extracted item by applicability, effective date, and penalty severity. You must distinguish between black-letter law (explicit statutory or regulatory text) and interpretive guidance (agency commentary, examples, or staff views).

## INPUT

[REGULATORY_FILING_TEXT]

## COMPLIANCE DOMAIN

[COMPLIANCE_DOMAIN]

## OUTPUT SCHEMA

Return a single JSON object with the following structure. Do not include any text outside the JSON object.

{
  "filing_metadata": {
    "document_title": "string",
    "issuing_body": "string",
    "publication_date": "YYYY-MM-DD or null",
    "effective_date": "YYYY-MM-DD or null",
    "jurisdiction": "string or null",
    "document_type": "string"
  },
  "extracted_items": [
    {
      "item_id": "string (unique identifier within this extraction)",
      "item_type": "obligation | definition | exception | interpretive_guidance | cross_reference",
      "item_text": "string (verbatim text from the filing)",
      "source_location": "string (section, paragraph, or page reference)",
      "plain_language_summary": "string (one-sentence summary for compliance review)",
      "applicability_scope": {
        "entities": ["string"],
        "activities": ["string"],
        "thresholds": ["string"],
        "exclusions": ["string"]
      },
      "effective_date": "YYYY-MM-DD or null",
      "compliance_deadline": "YYYY-MM-DD or null",
      "penalty_severity": "high | medium | low | none_specified",
      "penalty_description": "string or null",
      "evidence_type": "black_letter_law | interpretive_guidance | preamble | footnote | cross_reference",
      "confidence": "high | medium | low",
      "requires_human_review": true,
      "human_review_reason": "string (explain why human review is needed)",
      "related_item_ids": ["string"]
    }
  ],
  "ranking": {
    "high_priority_items": ["item_id"],
    "medium_priority_items": ["item_id"],
    "low_priority_items": ["item_id"],
    "ranking_rationale": "string (explain the ranking methodology)"
  },
  "gaps_and_uncertainties": [
    {
      "description": "string",
      "impact": "string",
      "recommended_action": "string"
    }
  ],
  "human_review_checklist": [
    "string (specific item for human reviewer to verify)"
  ]
}

## CONSTRAINTS

- Extract every obligation, definition, exception, and piece of interpretive guidance. Do not skip items.
- For every extracted item, include verbatim text from the filing. Do not paraphrase the item_text field.
- Distinguish black-letter law from interpretive guidance in the evidence_type field.
- Assign penalty_severity based on explicit penalty language in the filing. Use "none_specified" if no penalty is mentioned.
- Set requires_human_review to true for every item. This is a high-risk compliance workflow.
- If the filing references other documents without providing full text, flag this in gaps_and_uncertainties.
- If effective dates are ambiguous or conditional, set confidence to "low" and explain in human_review_reason.
- Rank items by: (1) imminent effective dates, (2) high penalty severity, (3) broad applicability scope, (4) black-letter law over interpretive guidance.
- Do not invent obligations, penalties, or dates not present in the filing.
- If the filing text is truncated or incomplete, note this in gaps_and_uncertainties.

## RISK LEVEL

[RISK_LEVEL]

## EXAMPLES

[EXAMPLES]

Adapt this template by replacing the square-bracket placeholders with your specific inputs. [REGULATORY_FILING_TEXT] should contain the full text of the regulatory document, cleaned of scanning artifacts but preserving section structure. [COMPLIANCE_DOMAIN] provides context about the regulatory area (e.g., 'anti-money laundering', 'medical device quality systems', 'environmental emissions reporting') to help the model interpret domain-specific terminology. [RISK_LEVEL] should be set to 'high', 'medium', or 'low' to adjust the strictness of the human-review requirements—for high-risk domains, every item should require human review. [EXAMPLES] should include one or two few-shot examples of correctly extracted and ranked items from similar filings to improve output consistency. If you are processing filings from a specific agency (SEC, FDA, EPA, etc.), include the agency name in the compliance domain to improve terminology recognition.

After copying the template, validate that your input text is complete and that section references are preserved. The output schema is designed to be ingested by a compliance workflow system, so field names and types must be preserved exactly. In production, you should validate the JSON output against this schema before storing results, and every item flagged with requires_human_review: true must be routed to a compliance reviewer queue. Do not remove the human-review requirement for high-risk regulatory workflows—this template is designed for RegTech systems where incorrect extraction can have legal consequences.

IMPLEMENTATION TABLE

Prompt Variables

Inputs the prompt needs to work reliably. Validate each before sending. Missing or malformed variables are the most common cause of silent failures in regulatory extraction workflows.

PlaceholderPurposeExampleValidation Notes

[REGULATORY_DOCUMENT]

Full text of the regulatory filing, rule, or guidance document to extract evidence from

Full text of an SEC 10-K filing, FDA guidance document, or MiFID II regulatory technical standard

Must be non-empty string. Minimum 100 characters. Reject if only a URL or document reference without body text. Check for truncated uploads by verifying document structure markers exist.

[JURISDICTION]

Legal or regulatory jurisdiction that governs interpretation of the filing

US-SEC, EU-ESMA, UK-FCA, US-FDA-CDER, JP-PMDA

Must match an allowed jurisdiction enum. Controls which regulatory hierarchy and authority rules apply. Reject unknown or ambiguous jurisdiction values. Map to canonical jurisdiction codes before prompt assembly.

[EXTRACTION_TARGETS]

Array of entity types to extract from the document

["obligation", "definition", "exception", "effective_date", "reporting_requirement", "penalty_provision"]

Must be a non-empty array of strings from the allowed target type enum. Each target type must have a defined extraction schema. Reject requests for undefined target types. Validate array length is between 1 and 20.

[EFFECTIVE_DATE_CONTEXT]

Reference date for determining which provisions are currently applicable

2025-03-15 or null if all provisions should be extracted regardless of temporal status

Must be ISO 8601 date string or null. When provided, used to filter provisions by effective date, sunset date, and transitional period status. Null means extract all provisions without temporal filtering. Validate date is not in the distant past without explicit intent.

[ENTITY_CONTEXT]

Optional entity profile for applicability filtering

{"entity_type": "investment_firm", "aum_usd": 2500000000, "client_types": ["retail", "professional"]} or null

Must be valid JSON object or null. When provided, used to score provisions by entity-specific applicability. Schema must match expected entity profile structure for the jurisdiction. Reject if entity_type is unrecognized for the jurisdiction.

[RANKING_CRITERIA]

Ordered list of criteria for ranking extracted evidence

["penalty_severity", "effective_date_proximity", "applicability_score", "regulatory_authority_level"]

Must be a non-empty ordered array of strings from the allowed ranking criteria enum. Order determines primary, secondary, and tertiary sort keys. Minimum 1 criterion, maximum 5. Reject duplicate criteria entries.

[OUTPUT_SCHEMA_VERSION]

Version of the compliance evidence map schema to produce

v2.1

Must match a supported schema version string. Controls output field definitions, required fields, and enum values. Reject unsupported versions. Maintain a registry of active and deprecated schema versions with migration paths.

[HUMAN_REVIEW_THRESHOLD]

Confidence threshold below which extractions require human review

0.85

Must be a float between 0.0 and 1.0. Extractions with confidence below this threshold are flagged for human review rather than auto-accepted. Value of 1.0 means all extractions require review. Value of 0.0 disables automated review flagging. Validate range and type.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Regulatory Filing Evidence Extraction and Ranking Prompt into a production compliance workflow with validation, retries, logging, and human-review routing.

This prompt is not a standalone chat interaction. It belongs inside a RegTech pipeline that ingests regulatory filings, extracts structured obligations, and routes interpretive findings for human review before any compliance decision is recorded. The harness must treat the model output as a draft evidence map, not as a final compliance artifact. Every obligation, definition, and exception extracted by the model must be traceable back to a specific passage in the source filing, and the harness should enforce that traceability before the output reaches a downstream system or reviewer.

Wire the prompt as a single step in a multi-stage extraction pipeline. Before calling the model, preprocess the filing to split it into manageable sections with stable section identifiers. For each section, call the model with the prompt template, passing the section text as [FILING_TEXT], the jurisdiction and filing type as [CONTEXT], and the required output schema as [OUTPUT_SCHEMA]. Validate the response immediately: check that every extracted obligation has a non-empty source_passage field, that effective_date parses to a valid date or null, and that penalty_severity is one of the allowed enum values. Reject and retry any response that fails schema validation, up to a maximum of two retries with backoff. Log every validation failure for later prompt debugging.

Route outputs through a human-review gate based on a risk score you compute from the extracted fields. For example, flag any obligation where penalty_severity is 'high' or 'critical', where effective_date is within 30 days, or where the model's interpretive_confidence field is below a threshold you set during evaluation. These flagged items should enter a review queue where a compliance analyst can confirm, edit, or reject the extraction. Items with high confidence and low severity can flow through to automated downstream systems, but the harness must retain the full model output and source passage for audit. Never allow black-letter legal interpretations to bypass human review without explicit legal-team approval and a documented risk acceptance.

For model choice, prefer a model with strong instruction-following and structured output capabilities. If your provider supports strict JSON mode or function calling with a defined schema, use that instead of relying on a free-text [OUTPUT_SCHEMA] instruction. This reduces repair loops. If you must use a model without native JSON mode, add a post-processing repair step that attempts to fix common malformations—unbalanced braces, trailing commas, or unescaped quotes—before retrying. Log repair attempts separately so you can measure how often the prompt produces malformed output and decide whether to refactor the prompt or switch models.

Build eval checks that go beyond schema validation. Create a golden dataset of filings with known obligations, definitions, and exceptions annotated by domain experts. After each prompt change or model upgrade, run the pipeline against this dataset and measure precision and recall at the obligation level. Also measure hallucination rate: how often does the model extract an obligation that has no supporting passage in the source text? If hallucination rate exceeds your risk tolerance, tighten the prompt's grounding instructions or increase the human-review threshold. Do not ship a prompt update without running these evals and comparing results against the previous version.

IMPLEMENTATION TABLE

Expected Output Contract

Structured output schema for the Regulatory Filing Evidence Extraction and Ranking Prompt. Each field maps to a compliance evidence map element. Validate before ingestion into downstream RegTech systems.

Field or ElementType or FormatRequiredValidation Rule

compliance_evidence_map

array of objects

Must be a non-empty array. Schema check: each element must match the evidence_item schema defined below.

evidence_item.filing_id

string

Must match the pattern [AGENCY]-[YEAR]-[DOC_TYPE]-[SEQUENCE]. Regex: ^[A-Z]+-\d{4}-[A-Z_]+-\d+$

evidence_item.obligation_text

string

Must be a verbatim quote from the source filing. Length >= 20 characters. Citation check: substring must appear in the provided [FILING_TEXT].

evidence_item.obligation_type

enum

Must be one of: 'black_letter_requirement', 'interpretive_guidance', 'safe_harbor_provision', 'reporting_obligation', 'recordkeeping_requirement', 'penalty_provision', 'exemption', 'definition'.

evidence_item.applicability_scope

string

Must specify the regulated entity type or activity threshold. Null not allowed. If filing does not specify, use 'ALL_REGULATED_ENTITIES' and set confidence below 0.7.

evidence_item.effective_date

string (ISO 8601 date) or null

Must be YYYY-MM-DD format if present. If date is ambiguous or not stated, set to null and populate effective_date_notes. Parse check: reject dates before 1900-01-01 or after 2100-12-31.

evidence_item.effective_date_notes

string or null

Required when effective_date is null. Must explain why date is missing or ambiguous. Max 200 characters.

evidence_item.penalty_severity

enum or null

Must be one of: 'none', 'warning', 'fine', 'license_action', 'criminal_referral', 'unknown'. Set to null only when obligation_type is 'definition' or 'safe_harbor_provision'.

evidence_item.penalty_detail

string or null

Required when penalty_severity is 'fine', 'license_action', or 'criminal_referral'. Must cite the specific penalty amount, duration, or action from [FILING_TEXT]. Max 500 characters.

evidence_item.citation_span

object

Must contain start_char and end_char integers referencing character offsets in [FILING_TEXT]. Validation: end_char > start_char, and the substring must match obligation_text exactly.

evidence_item.relevance_score

number (0.0 to 1.0)

Must be a float between 0.0 and 1.0 inclusive. Represents applicability match to [REGULATED_ENTITY_PROFILE]. Score >= 0.8 triggers high-priority flag in downstream system.

evidence_item.relevance_rationale

string

Must explain why this obligation applies to the entity in [REGULATED_ENTITY_PROFILE]. Max 300 characters. Must reference at least one specific field from the profile.

evidence_item.confidence

number (0.0 to 1.0)

Model's confidence in extraction correctness. If confidence < 0.7, the item must be routed to human review queue regardless of other scores.

evidence_item.human_review_required

boolean

Must be true if confidence < 0.7, obligation_type is 'interpretive_guidance', penalty_severity is 'criminal_referral', or effective_date is null. Otherwise false.

evidence_item.extraction_timestamp

string (ISO 8601 datetime)

Must be the UTC timestamp of extraction. Format: YYYY-MM-DDTHH:MM:SSZ. Used for staleness checks in downstream compliance monitoring.

PRACTICAL GUARDRAILS

Common Failure Modes

Regulatory filing prompts fail in predictable ways when the model confuses interpretation with black-letter law, misses effective-date logic, or over-ranks boilerplate. These cards cover the most common production failure modes and how to guard against them before a compliance reviewer sees the output.

01

Interpretation Masquerading as Extraction

What to watch: The model paraphrases or summarizes a regulatory obligation instead of extracting the exact text, introducing interpretive drift. This is especially dangerous when the model 'clarifies' ambiguous language without flagging the ambiguity. Guardrail: Require verbatim quote extraction with a separate [INTERPRETATION] field that is explicitly gated for human review. Add an eval that measures character-level overlap between the extracted obligation and the source text.

02

Effective-Date Blindness

What to watch: The model ranks an obligation as currently applicable when the effective date is in the future, or fails to detect that a provision has been superseded by a later amendment within the same filing. Guardrail: Add a mandatory [EFFECTIVE_DATE] and [SUPERSEDED_BY] field to the output schema. Run a deterministic post-processing check that compares extracted dates against the current system date and flags any future-dated obligations ranked as active.

03

Boilerplate Over-Ranking

What to watch: The model assigns high applicability or severity scores to standard disclaimers, forward-looking statement safe harbors, or generic reservation-of-rights language that appears in every filing. This drowns out entity-specific obligations. Guardrail: Include a [BOILERPLATE_EXAMPLES] few-shot section in the prompt showing low-ranked standard clauses. Add an eval that measures the rank correlation between model scores and a human-annotated ground-truth set where boilerplate is explicitly down-weighted.

04

Cross-Reference Omission

What to watch: The model extracts a standalone obligation but misses that it is qualified by a cross-referenced definition, exception, or condition in another section of the filing. The output looks complete but is materially incomplete. Guardrail: Add a [DEPENDENCIES] field that requires the model to list any cross-referenced sections. Implement a post-extraction validator that checks whether every extracted obligation with a cross-reference also has the referenced section present in the evidence set.

05

Penalty Severity Inflation

What to watch: The model inflates the severity ranking of obligations with dramatic penalty language (e.g., 'criminal penalties,' 'strict liability') without considering the practical enforcement history or materiality threshold. This skews downstream triage toward rare but scary provisions. Guardrail: Separate [PENALTY_SEVERITY] from [APPLICABILITY] in the ranking schema. Use a calibration set of obligations with known enforcement frequency to tune the severity scale, and add a human-review gate for any obligation ranked above a defined severity threshold.

06

Jurisdictional Scope Confusion

What to watch: The model extracts an obligation from a multi-jurisdictional filing but fails to tag which jurisdiction it applies to, or incorrectly assumes a provision applies globally when it is limited to a specific regulator or geography. Guardrail: Require a [JURISDICTION] tag on every extracted obligation. Add a validation rule that rejects any obligation where the jurisdiction cannot be determined from the source text, and route those to a human reviewer with the ambiguous passage highlighted.

IMPLEMENTATION TABLE

Evaluation Rubric

Run these checks on a golden dataset of 30 regulatory filings with known obligations. Each criterion targets a specific failure mode observed in production RegTech evidence extraction.

CriterionPass StandardFailure SignalTest Method

Obligation Recall

All known obligations in the golden set are extracted with no false negatives

Missing obligation that appears explicitly in the filing text

Compare extracted obligation IDs against golden-set obligation list; compute recall per filing

Applicability Classification

Obligation tagged with correct applicability scope (entity, product, jurisdiction) in >=95% of cases

Entity-level obligation misclassified as product-level or jurisdiction mismatch

Check [APPLICABILITY_SCOPE] field against golden labels; flag any scope mismatch

Effective Date Extraction

Effective date parsed correctly within ±1 day for explicit dates; null for conditional triggers not yet resolved

Explicit date parsed with wrong day, month, or year; conditional trigger reported as fixed date

Compare [EFFECTIVE_DATE] against golden date; validate null when golden expects conditional trigger

Penalty Severity Ranking

Rank order matches golden severity ordering; ties allowed only when penalties are equivalent

Civil penalty ranked above criminal liability; omission ranked above explicit fine

Compute Kendall tau distance between predicted and golden severity ranking; flag inversions

Source Citation Accuracy

Every extracted obligation cites the correct section, paragraph, or page in the source filing

Citation points to wrong section; fabricated section reference; missing citation on extracted obligation

Verify [SOURCE_CITATION] field resolves to correct text span in the filing; flag unresolvable citations

Interpretive vs. Black-Letter Tagging

Interpretive guidance tagged as [INTERPRETIVE]; explicit statutory or regulatory text tagged as [BLACK_LETTER]

Safe harbor example tagged as black-letter; statutory requirement tagged as interpretive

Compare [OBLIGATION_TYPE] against golden labels; measure F1 per type

Exception and Exemption Capture

All explicit exceptions, exemptions, and safe harbors extracted and linked to parent obligation

Exception extracted as standalone obligation; exemption omitted entirely; safe harbor mislinked

Check [EXCEPTION_TO] field links to correct parent obligation ID; verify no orphan exceptions

Cross-Reference Resolution

Cross-referenced obligations from other sections or external regulations are noted with source and resolution status

Cross-reference ignored; resolved to wrong external regulation; circular reference not flagged

Check [CROSS_REFERENCE] field for completeness; flag unresolved or circular references against golden annotations

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a small sample of regulatory filings. Use a frontier model (GPT-4o, Claude 3.5 Sonnet) with the default system prompt. Focus on getting the extraction structure right before adding ranking logic. Use manual review of 5-10 filings to calibrate obligation categories and penalty severity thresholds.

code
System: You are a regulatory filing analyst. Extract obligations, definitions, and exceptions from [FILING_TEXT].

Output as JSON with fields: obligation_text, category, effective_date, penalty_severity, source_section.

Watch for

  • Missing schema checks leading to inconsistent field names
  • Overly broad obligation extraction that captures background context as requirements
  • No distinction between black-letter law and interpretive guidance
  • Effective date parsing failures for relative dates ("within 90 days of enactment")
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.