Human-in-the-Loop (HITL)

The core of a HITL system is a programmatic gating mechanism that routes decisions to a human queue based on predefined, auditable criteria. This is not a random sampling; it is a rules-based engine that evaluates model outputs against business logic.

• Uncertainty Thresholds: Escalation occurs when a model's prediction confidence score falls within a specific band (e.g., 45%-65%), indicating ambiguity.
• Value Thresholds: Transactions exceeding a monetary amount are automatically flagged regardless of the model's score.
• Outlier Detection: Instances with a high anomaly score but a low classification probability are routed for review to resolve the contradiction.

A critical architectural distinction exists between active and passive human oversight, often confused in system design.

• Human-in-the-Loop (HITL): The human decision is a required, blocking step in the workflow. The automated process cannot proceed to a final action (e.g., blocking a transaction) until a human submits their judgment. This is the standard for high-risk fraud reviews.
• Human-on-the-Loop (HOTL): The human acts as a supervisory monitor. The system executes decisions autonomously, and the human intervenes only to override or halt the system. This is common in low-latency, high-volume screening where speed is paramount.

A HITL system is not just a decision gateway; it is a training data generation engine. Every human override or confirmation is logged as a new labeled data point.

• Ground Truth Creation: Human judgments resolve the ambiguity of borderline cases, creating a high-quality, labeled dataset of difficult examples.
• Model Retraining: This new data is fed back into the continuous model learning system to retrain the model, specifically improving its decision boundary in the uncertainty region.
• Drift Correction: Human feedback is the primary mechanism for correcting concept drift, as reviewers identify new fraud patterns that the model has not yet learned.

For a HITL system to satisfy SR 11-7 and model governance requirements, the human interaction must be captured in an immutable, chronological record.

• Decision Provenance: The audit trail must link the specific human reviewer, their decision, a timestamp, and any supporting notes to the exact model prediction and input features that triggered the escalation.
• Override Monitoring: The system must track the override rate per reviewer and per model to detect systematic biases, inadequate training, or potential internal collusion.
• Reconstructability: Regulators require the ability to replay any past decision sequence to prove that the human step was not bypassed and that the final outcome was determined by the documented workflow.

Integrating a human into a real-time fraud scoring pipeline introduces a significant and variable latency component that must be explicitly managed.

• Synchronous vs. Asynchronous: For real-time payment authorization, a synchronous HITL step is often impossible due to sub-second latency requirements. Instead, an asynchronous review model is used, where the transaction is held in a pending state while the human reviews it.
• Service Level Agreements (SLAs): The system must enforce a maximum review time. If the SLA expires, a default safe action (typically a conservative decline or hold) is triggered automatically to prevent an open risk window.
• Queue Prioritization: The system must dynamically prioritize the human review queue based on the transaction's value, the risk score, and the remaining time in the SLA window.

The efficacy of a HITL system is bounded by the cognitive load placed on the human reviewer. The interface must provide decision support, not just raw data.

• Explainability Integration: The interface must surface SHAP values or counterfactual explanations to show the reviewer exactly which features drove the model's high-risk score.
• Context Aggregation: It should aggregate related data—such as the user's 90-day transaction history, device fingerprint, and any linked accounts—into a single, coherent view to prevent the reviewer from needing to query multiple systems.
• Structured Judgment Capture: The interface should force a structured reason code for the human decision (e.g., "confirmed fraud," "false positive - legitimate business," "insufficient information") to enable downstream analytics and feedback loop quality.

The independent, evidence-based evaluation of a model's conceptual soundness and performance. In a HITL context, validation must assess whether the human override rate is within acceptable thresholds and whether operator decisions introduce systematic bias. Validators review the override monitoring logs to ensure the human-in-the-loop component is not degrading the model's overall risk posture.

The systematic tracking of every instance where a human operator reverses or modifies a model's automated recommendation. This is the primary control mechanism for HITL systems. Key metrics include:

• Override rate by operator, reason code, and time window
• Reversal analysis to detect poorly calibrated models
• Operator profiling to identify potential internal fraud or inadequate training Override logs form a critical part of the audit trail for regulatory review.

A chronologically secure, immutable record of all system activities, data accesses, and model decisions. For HITL workflows, the audit trail must capture:

• The model's original risk score and recommendation
• The human operator's identity, timestamp, and override rationale
• The final decision and any downstream consequences This tamper-proof record enables forensic reconstruction of decision logic and demonstrates compliance with SR 11-7 and EU AI Act human oversight requirements.

A human-interpretable statement identifying the minimal set of feature changes required to flip a model's adverse decision. In HITL review queues, counterfactuals empower operators by answering: 'What would need to be different for this transaction to be approved?' For example: 'If the transaction amount were $4,500 instead of $9,200, this alert would not have fired.' This technique bridges the gap between opaque model scores and actionable human judgment.

The end-to-end institutional framework for identifying, assessing, and mitigating risks arising from model use. HITL is often implemented as a compensating control within MRM—when a model's standalone performance cannot meet the bank's risk appetite, human review is mandated as a secondary safeguard. The MRM framework governs:

• The materiality threshold that triggers human review
• The qualification and training standards for operators
• The periodic validation of the HITL process itself

The EU AI Act mandates that high-risk AI systems—including creditworthiness and fraud detection models—be designed to allow effective human oversight. This requires:

• Operators must be able to understand the system's outputs
• Operators must be able to override or disregard the output
• The interface must prevent automation bias (blind reliance on the machine) HITL architecture directly operationalizes these regulatory mandates within financial fraud workflows.