AI Integration for Multi-modality and Fusion Imaging

AI integration for multi-modality imaging focuses on three core functional surfaces within platforms like Sectra, Philips IntelliSpace, Intelerad, and GE PACS: the fusion viewer/workspace, the structured reporting module, and the quantitative analysis toolkit. The primary technical objective is to inject AI-derived insights—such as automated lesion co-registration across PET and CT, organ segmentation on the anatomical dataset for standardized uptake value (SUV) calculation, or synthetic image generation—directly into the radiologist's or nuclear medicine physician's fused review workflow. This requires integration via DICOM Secondary Capture (SC) or Structured Report (SR) objects for AI results, and often DICOMweb APIs for retrieving and sending fused series between the PACS and a containerized AI inference service.

A production implementation typically involves a queued workflow: 1) A completed PET/CT study is sent to the PACS. 2) A DICOM Modality Worklist or HL7 ORM/O01 trigger initiates an AI pipeline. 3) The AI service retrieves the DICOM series via a secure gateway, runs co-registration and segmentation models, and generates results (e.g., a new DICOM series with lesion contours mapped to both PET and CT, or an SR containing SUVmax measurements). 4) These results are sent back and linked to the original study, appearing as an overlay in the fusion viewer and auto-populating fields in a structured report template. This reduces manual segmentation time from 10-15 minutes to under a minute and minimizes registration errors between modalities.

Governance is critical. AI outputs must be clearly labeled as AI-derived within the viewer, with confidence scores available. A human-in-the-loop verification step should be enforced in the reporting workflow before final sign-off. Furthermore, integration must support audit trails for AI use and model versioning, as quantitative measurements from fused datasets directly influence staging and treatment decisions. Rollout should start with a single, high-value workflow—like auto-segmentation of tumors on CT for PET SUV analysis in oncology follow-up—within a controlled user group before expanding to other fused modalities like SPECT/CT or MR-PET.

Integrating AI for multi-modality workflows (e.g., PET/CT, SPECT/CT) requires a pipeline that ingests, co-registers, and fuses disparate DICOM series from different scanners before AI analysis. The architecture typically connects to the enterprise PACS or Vendor Neutral Archive (VNA) via DICOMweb or HL7 FHIR to retrieve the component studies. A dedicated fusion orchestration service handles the heavy lifting: it pulls the relevant series (e.g., a PET and a corresponding CT), uses AI or traditional algorithms for image registration and alignment, and creates a fused dataset. This fused volume is then passed to downstream AI models for tasks like automated lesion segmentation, standardized uptake value (SUV) quantification, or tumor burden analysis across the combined modalities.

The integration must embed results back into the clinical workflow seamlessly. AI-generated outputs—such as segmented volumes, quantitative metrics, and fused overlays—are packaged as DICOM Structured Reports (SR) and Secondary Capture (SC) objects and sent back to the PACS. These are linked to the original studies, making them viewable within the radiologist's advanced visualization workstation (e.g., IntelliSpace Portal, Sectra 3D). For reporting, key quantitative findings (e.g., SUVmax, Total Lesion Glycolysis) can be auto-populated into structured report templates via the PACS reporting module or speech recognition system, creating a cohesive report that references both the anatomic (CT) and functional (PET) findings. This closed-loop data flow ensures the AI acts as a quantitative assistant within the existing diagnostic pathway.

Governance and rollout require careful planning. AI models for fusion imaging must be validated on co-registered data from your specific scanner combinations to ensure accuracy. Implement automated QA checks in the orchestration pipeline to flag registration failures or poor-quality inputs. Access to these advanced AI tools should be controlled via the PACS or VNA's existing Role-Based Access Control (RBAC), typically granted to nuclear medicine physicians and specialized radiologists. Start with a pilot in a single clinical domain (e.g., oncology PET/CT) to refine the workflow, measure impact on report turnaround time and quantitative consistency, and build clinician trust before expanding to other fused modalities like SPECT/CT or PET/MR.

Governance begins at the data layer. For PET/CT or SPECT/CT fusion, AI models must be validated on co-registered DICOM series from both modalities. Implement a pre-processing pipeline that validates image registration quality, voxel alignment, and acquisition parameters before AI inference. Results should be written as DICOM Structured Reports (SR) or as annotations within the fused dataset in the PACS, preserving a clear audit trail linking the original series, the AI model version, and the generated findings. Access to AI tools should be controlled via the PACS's existing RBAC, ensuring only credentialed nuclear medicine physicians or radiologists can activate or approve AI-generated segmentations and quantitative metrics.

A phased rollout is critical. Start with a non-interruptive pilot in a research or QA context. For example, deploy an AI model for automated SUV quantification or tumor segmentation on retrospective PET/CT oncology studies. The AI outputs are saved to a separate series or SR but do not modify the primary radiologist's workflow. In Phase 2, integrate AI results as soft prompts within the advanced visualization workstation (e.g., Philips IntelliSpace Portal, Sectra 3D), presenting pre-segmented volumes or heatmaps for the physician to accept, modify, or reject. The final, production phase enables worklist prioritization, where studies with AI-detected high metabolic volumes or new lesions are flagged for expedited review.

Safety is engineered through continuous feedback loops. Establish a human-in-the-loop review process where all AI-generated quantitative analyses (e.g., Total Lesion Glycolysis, metabolic tumor volume) used for clinical reporting must be confirmed by the interpreting physician. Implement drift detection by periodically comparing AI outputs against a ground-truth set of manually segmented studies to catch performance degradation. Rollout plans must include specific training for technologists on proper acquisition protocols to ensure AI-ready data and for physicians on how to interpret and document their interaction with AI tools, maintaining the legal integrity of the final report.