Jetson Orin

The Orin integrates an NVIDIA Ampere architecture GPU with up to 2048 CUDA cores and 64 Tensor Cores, delivering up to 275 TOPS of sparse INT8 inference performance. This enables real-time execution of 3D volumetric segmentation models and vision transformers directly on the scanner without cloud round-trips.

• Sparse tensor acceleration doubles throughput for pruned diagnostic models
• FP16 and INT8 mixed-precision support for quantization-aware deployment
• Concurrent execution of multiple DNNs for multi-task diagnostic pipelines

A dedicated Deep Learning Accelerator (DLA) provides a secondary inference engine optimized for energy efficiency. The DLA offloads repetitive convolutional operations from the GPU, achieving up to 5x better energy efficiency for sustained diagnostic workloads.

• Ideal for continuous whole slide image tiling and preprocessing
• Hardware-scheduled convolution reduces CPU intervention
• Supports INT8 and FP16 operations natively

Orin combines a 12-core Arm Cortex-A78AE CPU, Ampere GPU, DLA, and a Programmable Vision Accelerator (PVA) on a single SoC. This heterogeneous compute fabric allows diagnostic pipelines to be partitioned across processors for optimal throughput.

• CPU handles DICOM parsing and metadata extraction
• GPU runs heavy segmentation and classification models
• PVA accelerates image preprocessing like Hounsfield Unit normalization and resampling
• Hardware-based isolation enables safety-certified and non-certified workloads to coexist

Orin supports configurable power modes from 15W to 60W, allowing medical device manufacturers to balance performance against thermal and battery constraints. At 15W, the module still delivers over 60 TOPS, sufficient for many 2D diagnostic models.

• NVMON power rail monitoring enables real-time energy tracking
• Deterministic energy per inference measurement in millijoules
• Fanless thermal design compatible with sealed medical enclosures
• Critical for battery-powered portable ultrasound and point-of-care devices

The Orin platform includes hardware and software features for ISO 26262 ASIL-D functional safety, essential for diagnostic devices that influence clinical decisions. A dedicated Safety Cluster with lockstep-capable Cortex-R52 cores monitors the main compute complex.

• Hardware root of trust secures boot and prevents model tampering
• Encrypted OTA update support for deploying refined diagnostic models
• Memory ECC and bus parity protect against silent data corruption
• Enables FDA SaMD regulatory submissions with documented safety architecture

A compression technique that reduces the numerical precision of a neural network's weights and activations from FP32 to INT8 or FP16. This shrinks the model's memory footprint by up to 4x and accelerates inference on the Jetson Orin's tensor cores. Two primary approaches exist:

• Post-Training Quantization (PTQ): Converts a pre-trained model using a small calibration dataset without retraining
• Quantization-Aware Training (QAT): Simulates quantization during training, enabling the network to adapt and achieve higher accuracy after conversion For medical imaging, careful quantization preserves the subtle pixel-level features critical for detecting microcalcifications or small lesions.

A model optimization paradigm that incorporates the specific constraints of the Jetson Orin's Ampere GPU, Deep Learning Accelerators (DLA), and memory hierarchy directly into the neural network training process. Rather than designing a model in isolation and compressing it afterward, hardware-aware training jointly optimizes for accuracy, latency, and energy consumption. This is essential for scanner-side AI deployments where models must meet strict real-time deadlines for image reconstruction or quality control while operating within a fixed thermal envelope.

A critical efficiency metric measuring the total electrical energy, typically in millijoules (mJ), consumed to execute a single forward pass of a diagnostic model. On the Jetson Orin, this metric directly dictates:

• Battery life for portable ultrasound and point-of-care devices
• Thermal constraints in fanless, sealed medical enclosures
• Operational cost for always-on monitoring systems The Orin's heterogeneous architecture allows developers to route different model layers to the GPU, DLA, or CPU to minimize energy per inference while maintaining diagnostic accuracy.

A hybrid architecture that intelligently distributes AI workloads between the Jetson Orin at the edge and cloud-based infrastructure. Latency-sensitive tasks like real-time image quality checks and critical finding triage execute locally on the Orin, while computationally intensive analyses such as multi-study longitudinal comparisons or large-scale radiomics extraction are offloaded to the cloud. This orchestration ensures continuous operation during network outages and complies with data sovereignty requirements by keeping PHI processing local.

A safety mechanism that enables a model deployed on Jetson Orin to recognize input data fundamentally different from its training distribution. In diagnostic imaging, this prevents the AI from making unsupported predictions on:

• Unfamiliar anatomy from a body region not in the training set
• Novel scanner vendors with different image characteristics
• Rare pathologies the model was never exposed to When triggered, the system flags the study for mandatory human review rather than silently producing a potentially erroneous diagnosis, forming a critical safety net for FDA-cleared SaMD deployments.