Risk Score Threshold Policy Evaluation Prompt

This prompt is designed for risk governance teams, compliance officers, and fraud/security operations leads who need to evaluate whether their current risk score thresholds remain appropriate. It takes historical performance data, business impact metrics, and regulatory requirements as input, and produces a structured policy evaluation with threshold adjustment recommendations. Use this prompt during scheduled policy reviews, after a significant change in fraud patterns, or when preparing audit evidence for regulatory examinations. It is not a real-time scoring prompt; it is a retrospective policy analysis tool.

The ideal user brings structured inputs: a defined period of historical risk score data with outcomes, current threshold values, business impact categories (e.g., financial loss ranges, customer friction costs), and any applicable regulatory requirements or risk appetite statements. The prompt expects these as [HISTORICAL_PERFORMANCE_DATA], [CURRENT_THRESHOLDS], [BUSINESS_IMPACT_METRICS], and [REGULATORY_REQUIREMENTS]. Without these inputs, the evaluation will be speculative rather than evidence-based. The prompt also accepts optional [RISK_APPETITE_STATEMENT] and [PREVIOUS_POLICY_VERSION] placeholders to enable comparison against prior policy states and stated organizational tolerance.

Do not use this prompt for real-time transaction scoring, live anomaly detection, or operational decision-making. Those workflows require deterministic rule engines, real-time feature computation, and latency-optimized inference paths—not retrospective analysis. Similarly, avoid this prompt when you lack outcome-labeled historical data; without ground truth on which flagged transactions were actual fraud or which anomalies represented real threats, the threshold performance metrics will be unreliable. If your team needs a prompt for computing risk scores on live transactions, use the Transaction Risk Score Computation Prompt Template instead. For tuning detection sensitivity in production systems, use the Anomaly Detection Threshold Configuration Prompt.

After running this prompt, expect a structured output containing threshold performance metrics (precision, recall, false-positive rate, detection coverage), business impact analysis (estimated loss exposure, operational cost of reviews), and specific adjustment recommendations with supporting rationale. The output should also flag any regulatory alignment concerns and note where current thresholds diverge from stated risk appetite. Before acting on recommendations, validate the output against your own data, run sensitivity analysis on proposed threshold changes, and route any threshold adjustments that affect regulatory posture through your compliance review process. This prompt informs decisions—it does not make them.

This prompt template is designed for risk governance teams who need to systematically evaluate whether current risk score thresholds remain appropriate. It forces a structured analysis of historical performance, business impact, and regulatory alignment before producing a recommendation. The template expects you to supply historical performance data, current threshold values, business impact metrics, and any applicable regulatory requirements as input. The output is a structured policy evaluation with clear rationale and actionable recommendations.

codeCopy
You are a risk governance analyst evaluating the current risk score threshold policy. Your task is to assess whether the existing thresholds should be kept, adjusted, or replaced based on the provided evidence.

## INPUT DATA

### Current Threshold Policy
[THRESHOLD_POLICY]

### Historical Performance Data
[HISTORICAL_PERFORMANCE]

### Business Impact Metrics
[BUSINESS_IMPACT]

### Regulatory Requirements
[REGULATORY_REQUIREMENTS]

### Risk Appetite Statement
[RISK_APPETITE]

## OUTPUT SCHEMA

Produce a JSON object with the following structure:

{
  "evaluation_summary": {
    "overall_assessment": "string (keep | adjust | replace)",
    "confidence_level": "string (high | medium | low)",
    "key_findings": ["string"]
  },
  "threshold_performance": {
    "current_thresholds": [
      {
        "threshold_name": "string",
        "current_value": "number",
        "false_positive_rate": "number",
        "false_negative_rate": "number",
        "precision": "number",
        "recall": "number",
        "alert_volume": "number"
      }
    ],
    "calibration_assessment": "string",
    "drift_detected": "boolean",
    "drift_details": "string | null"
  },
  "business_impact_analysis": {
    "current_financial_exposure": "string",
    "operational_cost_of_reviews": "string",
    "customer_experience_impact": "string",
    "missed_detection_cost_estimate": "string"
  },
  "regulatory_alignment": {
    "compliance_status": "string (compliant | partially_compliant | non_compliant)",
    "gaps_identified": ["string"],
    "regulatory_risk_level": "string (low | medium | high | critical)"
  },
  "recommendations": [
    {
      "threshold_name": "string",
      "recommended_action": "string (keep | adjust | replace)",
      "proposed_value": "number | null",
      "rationale": "string",
      "expected_impact": {
        "false_positive_change": "string",
        "false_negative_change": "string",
        "alert_volume_change": "string",
        "risk_exposure_change": "string"
      },
      "implementation_risk": "string (low | medium | high)",
      "rollback_criteria": "string"
    }
  ],
  "implementation_plan": {
    "phased_approach": "string",
    "monitoring_requirements": ["string"],
    "approval_required": ["string"],
    "review_cadence_recommendation": "string"
  },
  "limitations_and_uncertainty": {
    "data_quality_concerns": ["string"],
    "analysis_limitations": ["string"],
    "assumptions_made": ["string"],
    "areas_requiring_human_judgment": ["string"]
  }
}

## CONSTRAINTS

- Base all analysis on the provided data only. Do not invent or assume performance metrics not present in the input.
- When data is insufficient to reach a conclusion, explicitly flag this in `limitations_and_uncertainty` and reduce `confidence_level` accordingly.
- For any recommendation to adjust or replace a threshold, provide specific `rollback_criteria` that would trigger reversion.
- Flag any recommendation that could increase regulatory exposure in `regulatory_alignment.gaps_identified`.
- If the current policy is compliant but the data suggests adjustment would improve performance, recommend the adjustment but note the compliance trade-off.
- Use `areas_requiring_human_judgment` to identify decisions that should not be automated based on this analysis alone.

## EVALUATION CRITERIA

Before finalizing your output, verify:
1. Every recommended threshold change has a measurable expected impact.
2. Regulatory gaps are mapped to specific requirements from the input.
3. Confidence level is consistent with data completeness and quality.
4. Rollback criteria are specific and observable, not generic.

To adapt this template, replace each square-bracket placeholder with your actual data. The [THRESHOLD_POLICY] should include current threshold names, values, and the rules that trigger escalation. [HISTORICAL_PERFORMANCE] should contain false-positive rates, false-negative rates, precision, recall, and alert volumes over a defined time window. [BUSINESS_IMPACT] should quantify the cost of reviews, missed detections, and customer friction. [REGULATORY_REQUIREMENTS] should cite specific regulations, their relevant clauses, and how the current thresholds map to compliance obligations. If any section lacks data, explicitly state that in the input rather than omitting the placeholder, so the model can flag the gap in limitations_and_uncertainty.

This prompt is designed for high-stakes governance decisions. Always route the output through a human review step before implementing any threshold changes. The implementation_plan.approval_required field should list the specific roles that must sign off. Pair this prompt with a regression testing harness that replays historical decisions through proposed thresholds to validate the expected impact before deployment. If the model's confidence level is low or the regulatory_risk_level is high or critical, escalate to a full governance committee review rather than proceeding with the recommendation.

This prompt is designed to be integrated into a scheduled governance workflow, not a real-time transaction path. It should run on a defined cadence—monthly, quarterly, or triggered by a significant shift in fraud patterns or regulatory requirements. The harness must treat the model's output as a draft policy evaluation that requires human review and sign-off before any threshold changes are applied to production systems. The prompt expects structured inputs including current thresholds, performance metrics, business impact data, and regulatory context, all of which must be assembled by the application layer before the model call is made.

The implementation harness should enforce a strict contract around the prompt's inputs and outputs. Before calling the model, validate that all required fields in the [THRESHOLD_CONFIG], [PERFORMANCE_METRICS], [BUSINESS_IMPACT_DATA], and [REGULATORY_CONTEXT] placeholders are populated with recent, sourced data. After receiving the model's response, run a schema validator against the expected output structure—particularly the threshold_adjustments, risk_appetite_alignment, and regulatory_checks fields. If validation fails, retry once with a repair prompt that includes the specific validation errors. Log every evaluation run, including the input snapshot, model response, validation result, and reviewer decision, to create an audit trail that demonstrates governance oversight. For model choice, use a model with strong reasoning capabilities and a low temperature setting (0.1–0.2) to favor consistency over creativity in policy analysis.

The most critical integration point is the human approval gate. The harness should never automatically apply threshold changes based on the model's recommendations. Instead, format the evaluation output into a review interface that highlights: (1) recommended adjustments with before/after values, (2) projected impact on false-positive rates and detection coverage, (3) any regulatory concerns flagged, and (4) confidence indicators for each recommendation. Require explicit approval from an authorized risk governance reviewer, and capture their decision, any overrides, and justification text. Store this approval record alongside the evaluation for audit purposes. Avoid wiring this prompt into any system that can directly modify production risk thresholds without human intervention—the prompt's purpose is to inform and accelerate human decision-making, not to replace it.