False Positive vs False Negative Trade-off Analysis Prompt

This prompt is designed for trust and safety engineering managers and detection system operators who need to make a deliberate, documented decision about where to set the operating point in a production abuse detection pipeline. Use it when you have a batch of scored detection outputs from an existing model or rule engine, a set of ground-truth labels for those outputs, and quantified business cost estimates for both false positives (incorrectly blocked legitimate content or users) and false negatives (missed abuse that reaches the platform). The core job-to-be-done is translating raw model confidence scores and labeled outcomes into a structured trade-off analysis that recommends a specific threshold adjustment, projects the resulting precision-recall shifts, and estimates the operational impact so the decision can be reviewed, challenged, and approved before any change is deployed.

Do not use this prompt to build a classifier from scratch, to compare different model architectures, or to generate training data. It assumes you already have a working detection system producing confidence scores and that the primary engineering task is threshold calibration, not model development. The prompt is also not a substitute for an A/B test or a gradual rollout; it provides the analytical justification for a threshold change, but the actual deployment should follow your standard change management process. The ideal user is someone who understands the detection pipeline's performance characteristics, has access to labeled evaluation data, and can translate business harm into concrete cost estimates per event. If you lack ground-truth labels or cannot estimate the relative cost of false positives versus false negatives, this prompt will produce unreliable recommendations.

Before using this prompt, gather your scored detection outputs, ground-truth labels, and cost estimates into a structured input. The prompt expects a distribution of confidence scores, not just aggregate metrics, because threshold tuning requires understanding the density of scores near the current operating point. After receiving the output, treat the recommendation as an engineering decision record input, not an automated command. Validate the projected precision-recall shifts against a held-out test set, review edge cases where the model was uncertain, and ensure the recommended threshold does not violate any regulatory or policy constraints on your platform. The next step is to feed the analysis into your change management or incident review process, not to apply the threshold change directly from the model's output.

This prompt template is designed to be pasted directly into your model interface after you replace the square-bracket placeholders with data from your detection system. It forces the model to act as a quantitative trust and safety analyst, grounding its recommendation in your provided score distribution, cost estimates, and operational constraints. The prompt is structured to produce a decision memo, not just a number, so you can review the reasoning before changing a production threshold.

textCopy
You are a quantitative trust and safety analyst. Your task is to analyze the provided abuse detection score distribution and recommend an operational threshold that balances the business cost of false positives against the harm of false negatives.

## INPUT DATA
- Score distribution (binned counts or raw scores): [SCORE_DISTRIBUTION]
- Estimated cost per false positive (user friction, support ticket, lost revenue): [FP_COST]
- Estimated cost per false negative (harm event, regulatory penalty, brand damage): [FN_COST]
- Current production threshold: [CURRENT_THRESHOLD]
- Operational constraints (max FP rate, min recall, review queue capacity): [CONSTRAINTS]

## OUTPUT SCHEMA
Return a JSON object with the following fields:
{
  "recommended_threshold": number,
  "projected_precision": number,
  "projected_recall": number,
  "projected_fp_rate": number,
  "projected_fn_rate": number,
  "estimated_daily_fp_count": number,
  "estimated_daily_fn_count": number,
  "total_projected_cost": number,
  "trade_off_rationale": "string explaining the shape of the trade-off curve and why this threshold is optimal given the cost inputs",
  "sensitivity_analysis": "string describing how the recommendation changes if FP cost or FN cost shifts by ±20%",
  "edge_case_risks": ["list of specific score ranges or input types where the model is least confident and human review is advised"],
  "implementation_notes": "string with guidance on phased rollout, monitoring metrics, and rollback criteria"
}

## ANALYSIS INSTRUCTIONS
1. Identify the precision-recall trade-off curve from the score distribution.
2. Calculate the total cost at each viable threshold using the formula: (FP_count * FP_COST) + (FN_count * FN_COST).
3. Select the threshold that minimizes total cost while respecting all operational constraints.
4. If multiple thresholds produce similar total costs, prefer the one with higher recall (fewer missed harms).
5. Flag any score ranges where the model's confidence is low or the distribution is sparse.

## CONSTRAINTS
- Do not recommend a threshold that violates any stated operational constraint.
- If no threshold satisfies all constraints, explicitly state the conflict and recommend the least-bad option with justification.
- Ground all projections in the provided score distribution. Do not invent data.
- If the distribution is insufficient to make a reliable recommendation, state what additional data is needed.

To adapt this template, replace each bracketed placeholder with real data from your system. The [SCORE_DISTRIBUTION] field should include enough granularity for the model to reason about the trade-off curve—binned histograms with counts work better than summary statistics alone. For [FP_COST] and [FN_COST], use concrete dollar figures or normalized cost units that reflect your actual business impact. The [CONSTRAINTS] field is where you encode hard operational limits, such as a maximum false positive rate your review team can handle or a minimum recall required by policy. After running the prompt, validate the output JSON against your expected schema before ingesting the recommendation into any automated threshold adjustment system. For high-stakes detection systems, always route the model's output through a human review step before changing production thresholds.

This prompt is designed to run as an offline analysis step, not as a real-time classification component. Wire it into your detection system evaluation pipeline or run it manually during threshold review cycles. The prompt expects a structured input payload containing detection statistics, edge-case examples, and business impact parameters. It produces a trade-off analysis with threshold recommendations, precision-recall shift estimates, and operational impact projections. Because the output influences production threshold changes that affect user experience and safety coverage, the harness must validate the output structure, log every analysis run for auditability, and require human approval before applying threshold changes.

The implementation harness should enforce a strict input schema before invoking the model. Required fields include current_threshold, detection_stats (with true_positives, false_positives, true_negatives, false_negatives), edge_cases (an array of representative misclassified examples with ground-truth labels), and business_parameters (with cost_per_false_positive, cost_per_false_negative, and regulatory_risk_level). Validate these inputs at the application layer before prompt assembly. After the model returns its analysis, validate the output against an expected schema: recommended_threshold, estimated_precision, estimated_recall, trade_off_rationale, operational_impact (with additional_review_volume, missed_violation_estimate), and confidence_level. If the output fails schema validation, retry once with the validation errors injected into the prompt as feedback. If the retry also fails, log the failure and escalate for manual analysis rather than silently accepting a malformed recommendation.

Model choice matters for this workflow. Use a model with strong reasoning capabilities and consistent JSON output support. The analysis requires comparing quantitative trade-offs and generating calibrated estimates, not just pattern matching. Set temperature low (0.0–0.2) to reduce variance in threshold recommendations. Log every analysis run with a unique analysis_id, timestamp, input statistics, model version, raw output, validated output, and the human reviewer who approved or rejected the recommendation. Store these logs in an append-only audit table. Never apply threshold changes automatically from this prompt's output. Always route the recommendation to a human reviewer with context about the current threshold, the proposed change, and the estimated impact. The reviewer should confirm or override the recommendation, and that decision should be logged alongside the analysis record for future calibration reviews.