Multi-Agent Based Automation of Duplicate Case Detection and Merging

Pharmacovigilance associates spend hours daily manually scanning global databases (e.g., Oracle Argus, Veeva Safety) for potential duplicates. A custom agent system automates this scan using fuzzy matching on patient demographics, drug, and event narratives, presenting only high-confidence merge proposals. This directly converts FTE hours from low-value searching to high-value medical review and signal assessment.

Manual duplicate processes often miss cases or create conflicting data across merged records, leading to audit observations and potential reporting inaccuracies. An automated workflow enforces a consistent, rules-based merge logic with a full audit trail. Every proposed and executed merge is logged with the agent's reasoning and reviewer approval, creating a defensible record for health authority inspections.

Duplicate cases stuck in a review queue delay the entire case processing timeline, impacting reporting deadlines. An orchestrated agent workflow identifies and routes potential duplicates within minutes of case intake. This allows for parallel processing and earlier case finalization, compressing the cycle time from detection to database lock and ensuring expedited reports (e.g., 7-day, 15-day) are built on complete, consolidated data.

Undetected duplicates skew disproportionality analyses (PRR, ROR) and aggregate reports (PSURs), leading to false signals or diluted true signals. A systematic merge ensures the safety database reflects unique patient events, improving the statistical validity of continuous monitoring and the accuracy of data pulled for periodic reports. This turns the database from a repository into a reliable source for strategic decision-making.

As product portfolios and global case volumes grow, manual duplicate checking becomes a bottleneck that demands more staff. A custom multi-agent system scales elastically with data volume, handling thousands of comparisons per hour without fatigue. This creates operational leverage, allowing existing PV teams to manage increased workload from new drug launches or geographic expansions without proportional cost increases.

Misaligned case data between licensor and licensee due to inconsistent duplicate handling creates massive quarterly reconciliation effort. Implementing a standardized, automated detection and merge workflow on both sides of the partnership ensures aligned data foundations. This drastically cuts the manual effort required to match cases and resolve discrepancies, streamlining a key contractual and compliance obligation.

Manual duplicate review is a high-volume, low-value task that consumes 15-30% of case processor time. This workflow automates the identification and merge proposal, directly targeting a 40-60% reduction in case processing labor for high-volume products. By preventing duplicate cases from entering aggregate analyses, it also improves the accuracy of signal detection and ensures a single, complete patient history for causality assessment, which is critical for regulatory audits.

The workflow is managed by a central orchestrator built on LangGraph, which coordinates a team of specialized agents through a defined state machine. It manages the lifecycle from case intake to merge resolution, handling retries, timeouts, and exception routing. This architecture ensures auditability, allows for the injection of human review gates at critical points, and provides a clear visualization of the process flow for compliance and debugging.

Matching Agent: Performs fuzzy matching across patient demographics (hashed), drug names (normalized to WHO-DD), and event narratives (embedded and compared). It uses a combination of deterministic rules and vector similarity.

Confidence Scoring Agent: Evaluates match clusters using a weighted model (e.g., patient match = high weight, event term similarity = medium weight) to assign a merge confidence score (High/Medium/Low).

Proposal Agent: For High/Medium confidence clusters, it drafts a merge proposal document, highlighting matching fields and discrepancies, and routes it to the designated reviewer queue in the safety database (e.g., ARISg, Veeva Safety).

The workflow ingests new and updated cases in near-real-time via API listeners or database triggers from the primary safety database. It must also query historical cases for comparison. Post-proposal, it updates case statuses and logs all actions—match attempts, scores, reviewer decisions—back to the database's audit trail. The key integration challenge is operating within the performance constraints and data models of legacy PV systems without disrupting core operations.

No merge is executed autonomously. All proposals require human reviewer approval within the safety database UI. The system enforces a four-eyes principle for high-impact merges. Every agent decision, data point used, and confidence score is written to an immutable audit log linked to the case IDs. Confidence thresholds and matching rules are configurable per product or region, allowing medical affairs to adjust sensitivity based on the product's safety profile.

1. Pilot on a Closed Dataset: Run the agents offline for 3-4 weeks on historical data for a single product, tuning matching logic against known duplicates.
2. Shadow Mode Integration: Deploy the orchestration layer to process live cases in parallel with manual operations, comparing outputs without affecting the production database.
3. Phased Go-Live: Enable the proposal workflow for a single product team, with robust monitoring for exception rates and reviewer feedback.
4. Scale and Optimize: Roll out to additional products and regions, incorporating feedback to refine agent prompts and confidence models.