Hexagon DSP

The Hexagon DSP is not a standalone accelerator but a core component of the Qualcomm AI Engine. This architecture intelligently orchestrates AI workloads across three primary compute units:

• Hexagon Processor (DSP/HTA): For scalar, vector, and tensor math on quantized models.
• Adreno GPU: For floating-point and high-throughput parallel operations.
• Kryo CPU: For control logic and lighter, non-vectorizable tasks. A system-level scheduler dynamically partitions neural network subgraphs, offloading dense linear algebra (like convolutions) to the Hexagon DSP for optimal power efficiency.

Modern Hexagon DSPs incorporate a dedicated Hexagon Tensor Accelerator (HTA) block. The HTA is a fixed-function hardware unit designed explicitly for low-precision matrix multiplications, which dominate neural network inference. Key characteristics include:

• Native INT8/INT16 Support: Executes quantized integer operations without the overhead of floating-point units.
• Large Vector Register Files: Enables processing of multiple data points per clock cycle.
• Systolic Array Design: An efficient dataflow architecture that minimizes data movement and maximizes compute density for tensor ops. This specialization allows the DSP to deliver high TOPS (Tera Operations Per Second) per watt, crucial for mobile and edge devices.

The Hexagon DSP employs a unified Scalar, Vector, and Tensor (SVT) execution model within a single processor core.

• Scalar Unit: Handles control flow and address calculations.
• Vector Unit: Processes Single Instruction, Multiple Data (SIMD) operations on 1D arrays, efficient for activations and element-wise ops.
• Tensor Unit (HTA): As described, performs 2D/3D matrix and tensor computations. This integrated design allows the DSP to execute entire neural network layers—including non-linearities, pooling, and data shuffling—without costly handoffs to other processors, reducing latency and power.

A defining feature is the DSP's placement on a low-power island within the Snapdragon SoC. This design allows it to operate independently while the main application processor cores are in a deep sleep state. This enables:

• Always-On Contextual AI: Continuous, ultra-low-power processing for use cases like voice wake-word detection, sensor fusion, and ambient intelligence.
• Dramatic System Power Savings: By avoiding wake-ups of power-hungry CPU/GPU clusters for simple inference tasks. The DSP can access dedicated memory and peripherals on this island, minimizing data movement across power domains.

Accessing the Hexagon DSP's capabilities requires a specialized software stack. The primary path is through the Snapdragon Neural Processing Engine (SNPE) SDK or the Android Neural Networks API (NNAPI).

• SNPE Runtime: Provides a DSP delegate that converts a model's computational graph (e.g., from TensorFlow Lite or ONNX) into optimized Hexagon vector instructions (HVX).
• Offline Preparation: SNPE includes tools for model quantization, graph analysis, and compilation to create a cache of pre-compiled kernels for the target DSP generation.
• NNAPI Driver: Qualcomm provides an NNAPI driver that allows frameworks like TensorFlow Lite to use the Android API to delegate operations to the Hexagon DSP transparently.

Hexagon Vector eXtensions (HVX) are the DSP's SIMD instruction set, critical for performance. HVX features:

• 1024-bit Vector Registers: Can process 128 INT8 or 32 INT32 elements in a single instruction.
• Zero-Overhead Hardware Looping: Reduces instruction fetch overhead for tight, data-parallel loops common in ML kernels.
• Direct Memory Access: Supports non-cached, streaming access patterns optimal for large tensor data. ML kernels (convolution, fully connected layers) are hand-optimized or compiler-generated to use HVX instructions, maximizing data throughput. The HTA builds upon this by adding dedicated hardware for the tensor operations that HVX would simulate in software.

The Android Neural Networks API (NNAPI) provides a hardware abstraction layer for on-device ML on Android. Qualcomm provides a vendor driver that allows NNAPI to dispatch workloads to the Hexagon DSP. This enables framework-agnostic acceleration:

• Framework Integration: TensorFlow Lite, PyTorch Mobile, and other frameworks that use NNAPI can leverage the Hexagon DSP without direct SDK calls.
• Automatic Delegation: The NNAPI runtime uses the Qualcomm driver to identify and offload supported operations to the DSP.
• Simplified Development: Developers using standard Android ML frameworks can benefit from Hexagon acceleration without learning a proprietary SDK, though fine-grained control is more limited compared to SNPE.

The TensorFlow Lite Hexagon Delegate is a plugin that allows TFLite models to execute quantized operations on the Hexagon DSP. It is an alternative to using SNPE directly for developers invested in the TFLite ecosystem.

• Optimized for Int8: Primarily accelerates 8-bit quantized models, leveraging the HTA.
• Seamless Integration: Works with the standard TFLite interpreter API; supported operations are automatically offloaded.
• Requires Shared Library: The delegate relies on a Qualcomm-provided Hexagon NN shared library, which must be present on the target device.
• Performance Profiling: Can be used alongside TFLite's benchmarking tools to measure DSP acceleration gains versus CPU execution.

A critical step in the Hexagon DSP workflow is converting and optimizing models from training frameworks into a deployable format. Key tools include:

• SNPE Converter (snpe-onnx-to-dlc, snpe-tensorflow-to-dlc, etc.): Converts models to the SNPE DLC format, applying graph optimizations and quantization.
• Quantization Encoding Tools: Tools like snpe-dlc-quantize perform post-training quantization, crucial for achieving optimal performance on the HTA.
• ONNX to SNPE Flow: A common pipeline involves exporting a model to ONNX, then using the SNPE tools for conversion and quantization.
• Model Analyzer (snpe-dlc-info, snpe-dlc-viewer): Utilities to inspect DLC files, view the graph, and understand which operations will run on which processor.

Managing execution across the heterogeneous Snapdragon platform requires runtime tools for orchestration and performance analysis.

• SNPE Runtime APIs: Allow selection of runtime target (DSP, GPU, CPU, or a combination), management of model caching, and control over performance profiles (e.g., power savings vs. burst performance).
• System Profiling: Tools like Snapdragon Profiler provide visibility into DSP utilization, memory bandwidth, and power consumption during model inference.
• Multi-Core Execution: SDKs support splitting a model's layers across different cores (e.g., some on DSP, some on GPU) to maximize throughput or meet latency goals.
• Benchmarking Suites: Qualcomm provides benchmarks to compare the performance of standard models across different runtime targets, helping developers make deployment decisions.

The Hexagon Tensor Accelerator is a dedicated hardware block within the modern Hexagon DSP core designed explicitly for low-precision tensor operations.

• Executes INT8, INT16, and FP16 matrix multiplications and convolutions with extreme energy efficiency.
• Features a systolic array architecture optimized for the small, quantized kernels common in mobile vision and language models.
• The HTA is what makes the Hexagon DSP particularly effective for quantized neural network inference, as opposed to traditional signal processing tasks.

Direct Hexagon Linking (often via Qualcomm Hexagon SDK libraries) is a low-level programming approach for the DSP.

• Developers write C/C++ code for the Hexagon core's scalar and vector units, compiled with the Hexagon toolchain.
• This is used for custom operators or ultra-optimized pre/post-processing pipelines that run entirely on the DSP, avoiding costly CPU<->DSP memory transfers.
• It's a more complex but powerful method compared to using higher-level frameworks like SNPE, offering maximum control over the DSP's resources.