Edge-Cloud Orchestration

The orchestrator executes time-critical inferences directly on the edge device to meet clinical real-time constraints. Simple, high-priority tasks—such as quality control checks, anatomical landmark detection, or flagging a critical hemorrhage—are processed locally without the variable latency of a network round-trip.

• Typical edge inference latency: < 50 milliseconds
• Eliminates dependency on network availability during acute care
• Contrast with cloud offloading, which may introduce 200-500ms of additional latency

When a case exceeds the edge device's capabilities, the orchestrator transparently offloads the workload to a cloud-based high-performance compute cluster. This is triggered by model confidence thresholds, image complexity metrics, or specific study protocols.

• Gigapixel whole slide image analysis is a canonical offload candidate
• Cloud-side models can be 10-100x larger than their edge counterparts
• The orchestrator manages stateful session continuity, ensuring the radiologist sees a unified result regardless of where inference occurred

A core function of the orchestration layer is to minimize data transfer over constrained clinical networks. Instead of streaming raw DICOM studies, the edge component transmits only compressed feature vectors, anonymized regions of interest, or inference metadata.

• Reduces upstream bandwidth consumption by 95-99% compared to full-study upload
• Employs progressive image compression for cases requiring remote specialist review
• Adheres to DICOMweb standards for seamless PACS integration of partial results

The orchestration system implements a tiered model cascade, where a lightweight edge model acts as a gatekeeper. If the edge model's uncertainty quantification exceeds a predefined threshold, the case is escalated to a more capable cloud model.

• Edge model: INT8 quantized, optimized for Jetson Orin or similar SoMs
• Cloud model: FP32 ensemble with test-time augmentation
• Cascading reduces cloud compute costs by only invoking expensive models on ambiguous or complex cases

Edge-cloud orchestration is designed for degraded network conditions. The local inference engine maintains full functionality during connectivity loss, queuing non-urgent offload requests for synchronization when the network is restored.

• Implements a local-first architecture with persistent edge-side storage
• Uses conflict-free replicated data types (CRDTs) to merge results upon reconnection
• Critical for ambulance-based stroke imaging and rural point-of-care ultrasound where connectivity is intermittent

The orchestration layer manages the secure lifecycle of models across the fleet. Instead of sending patient data to the cloud for retraining, the system participates in federated learning rounds, where only encrypted gradient updates are shared.

• Integrates with OTA update mechanisms for seamless model deployment
• Maintains a model registry tracking version, performance metrics, and drift status per device
• Enables continuous learning without violating HIPAA or GDPR data residency requirements