Cross-Modality Image Registration and Fusion Automation Workflow

Manual side-by-side comparison of PET, CT, and MRI scans is a time-intensive cognitive task. A custom agentic workflow automates the spatial registration, overlays fused views, and highlights concordant/discordant findings. This reduces the radiologist's manual synthesis effort from 15-20 minutes to under 5 minutes per complex case, directly increasing reading throughput and allowing specialists to focus on interpretation rather than alignment.

Manual fusion is prone to misregistration errors, especially in serial studies. An automated workflow with rigorous validation (e.g., mutual information metrics, landmark verification) ensures pixel-perfect alignment. This creates a single, reliable source of truth for tumor board discussions and treatment planning, reducing the risk of missed interval changes or misinterpretation due to poor visual correlation. The system logs all transformation parameters for auditability.

In complex oncology (e.g., staging) and neurology (e.g., epilepsy focus localization), the diagnostic pathway often waits for sequential scan reviews and manual correlation. A custom orchestration layer, integrated with PACS and EHR, triggers fusion immediately upon study completion, pushing a pre-fused case to the specialist worklist. This eliminates coordination delays, potentially shortening the time from imaging to multidisciplinary review by multiple days, enabling faster treatment decisions.

The hidden cost of complex case review includes prolonged radiologist time, potential for repeat scans due to inconclusive reads, and administrative coordination. Automating fusion standardizes this high-value task. The ROI comes from labor leverage (handling more cases with the same specialist FTE), reduced need for repeat imaging, and decreased administrative overhead in preparing cases for tumor boards. The architecture pays for itself at scale across an academic medical center or large oncology network.

Fused images are the prerequisite for advanced analytics. Once PET-CT or MRI-US are aligned, a downstream workflow can automatically extract standardized uptake values (SUVs) from precisely matched anatomical regions or calculate perfusion metrics. This automation turns a subjective review into a quantitative, reproducible data pipeline, supporting clinical trial enrollment, treatment response assessment (PERCIST), and longitudinal research databases without manual ROI placement.

Subspecialists for complex fusion (e.g., neuro-radiology, nuclear medicine) are a constrained resource. This workflow makes their expertise more scalable. A centrally managed fusion service can pre-process studies from any network hospital, presenting a ready-to-read fused case to a remote specialist. This optimizes their cognitive effort, reduces burnout from manual tasks, and allows a single expert to support a broader geographic network, improving access to specialized diagnostics.

This agent orchestrates the secure, DICOM-compliant ingestion of scans from disparate PACS and modalities. It triggers automated quality control (QC) for artifacts, standardizes intensity values, and applies modality-specific preprocessing (e.g., N4 bias correction for MRI, SUV normalization for PET) to prepare images for registration. It routes suboptimal studies back to technicians via the RIS, preventing downstream processing errors.

The core computational engine executes advanced, often GPU-accelerated, deformable registration algorithms (e.g., B-spline, Demons) to align the moving image (e.g., PET) to the fixed reference (e.g., CT). It handles large anatomical variations and uses mutual information or cross-correlation metrics to optimize alignment. This agent outputs a dense deformation field and the transformed image, which are passed to the fusion and confidence-scoring layer.

This agent applies the deformation field to fuse functional/metabolic data (PET) with anatomical context (CT/MRI), creating a semantically aligned overlay. It manages color mapping, opacity, and visualization protocols (e.g., MIP, MPR) based on clinical context (oncology vs. neurology). The orchestrator generates the final fused DICOM object and a preliminary fusion quality score for downstream validation.

A critical safety and quality control component. This agent evaluates the registration and fusion output using quantitative metrics (e.g., Dice score on known landmarks, Jacobian determinant for deformation field sanity). It assigns a confidence score (High/Medium/Low) and automatically routes low-confidence results to a human-in-the-loop review queue within the radiologist's PACS worklist, preventing erroneous fused studies from reaching clinical interpretation.

To support longitudinal analysis, this agent queries the EHR (via FHIR/HL7) and imaging archives to retrieve relevant prior studies and clinical notes at the moment of fusion. It performs temporal alignment of prior exams and presents them alongside the newly fused study in the diagnostic viewer. This eliminates 5-10 minutes of manual search time per complex case and provides essential context for change detection.

The system integration backbone. This agent manages the end-to-end data flow, pushing the final fused study and its metadata back to the PACS with appropriate accession linking. It updates the RIS status, prioritizes the case on the radiologist's worklist based on urgency and subspecialty, and can trigger critical finding alerts. It ensures the automated workflow is a seamless, auditable part of the existing clinical IT ecosystem.

The workflow's trigger is a new study order in the Radiology Information System (RIS). An orchestration agent listens for HL7 ADT/ORM messages, retrieves the relevant prior exams (e.g., a baseline MRI for a new PET scan) from the Picture Archiving and Communication System (PACS), and routes the paired DICOM series to the registration engine. This eliminates manual search and retrieval, saving 5-8 minutes per complex case and ensuring all relevant data is assembled before the radiologist opens the study.

Core computational agents perform rigid and deformable image registration (e.g., aligning PET metabolic data to CT anatomy). A dedicated Quality Control (QC) agent runs post-registration metrics (mutual information, landmark error) and flags misalignments exceeding clinical tolerance (e.g., >2mm). Failed registrations are routed to a technical queue for manual correction or algorithm tuning, preventing flawed fusion from reaching diagnostic review.

Prior to fusion, an agent queries the Electronic Health Record (EHR) via FHIR APIs for critical context: patient positioning during prior scans, surgical history, or implanted devices that may cause artifacts. This context informs the fusion logic—for example, weighting MRI soft-tissue contrast more heavily in post-operative regions. This step reduces fusion errors by ~15% and creates a more clinically informed unified view.

The fused dataset and individual aligned modalities are pushed to the clinical viewer (e.g., Sectra, IntelliSpace). A fusion agent generates confidence heatmaps and toggle-able overlays directly within the radiologist's native workspace. This integration avoids context switching, allowing the clinician to assess alignment quality and diagnostic features in one interface, cutting synthesis time for tumor boards or progression tracking by 30-40%.

Upon radiologist approval of the fused view, the workflow triggers a handoff to the reporting system. An LLM agent, augmented with the aligned findings and prior report text, drafts an impression that synthesizes observations across modalities (e.g., "PET avidity corresponds to the enhancing margin on MRI"). This structured draft populates the radiologist's dictation software, reducing repetitive narrative synthesis and cutting report finalization time by ~25%.

Approved fused studies and their associated transformation matrices are archived in a research data lake with links back to PACS. This curated dataset fuels a continuous feedback loop: a monitoring agent tracks registration performance across patient cohorts and triggers model retraining pipelines when drift is detected (e.g., degraded performance on a new scanner model). This closed-loop system maintains diagnostic accuracy and compliance in a live clinical environment.