Pipeline Parallelism

The core mechanism of pipeline parallelism. Instead of processing one entire signal burst at a time, the workload is split into stages (e.g., FFT, normalization, model layer 1, model layer 2). Multiple micro-batches of IQ samples flow through these stages simultaneously. While stage 2 is processing batch n, stage 1 is already working on batch n+1. This overlapping execution keeps all compute units busy, dramatically increasing overall system throughput measured in classifications per second.

A key inefficiency in naive pipeline parallelism. At the start of processing (the warm-up phase) and the end (the cool-down phase), some stages are idle while waiting for data. This idle time is called a bubble. The relative size of the bubble compared to total execution time decreases as the number of micro-batches increases. Advanced scheduling techniques, like 1F1B (one-forward-one-backward) used in training, can be adapted for inference to minimize these bubbles and improve hardware utilization.

The throughput of the entire pipeline is limited by its slowest stage, the bottleneck. Effective pipeline parallelism requires careful stage balancing to ensure each partition takes roughly the same amount of time. For a modulation classifier, this might mean:

• Partitioning a large neural network so each stage has a similar number of layers or FLOPs.
• Offloading a computationally heavy Polyphase Filter Bank to one FPGA core while a lighter Softmax layer runs on another.
• Profiling each stage's latency to identify and resolve imbalances.

The data transfer mechanism between pipeline stages is critical. To avoid the latency of copying data, high-performance pipelines use zero-copy buffers and shared memory regions. Data is passed between stages by simply transferring a pointer to the memory location of the IQ tensor. This is often implemented using a circular buffer or a ring buffer to manage the continuous flow of micro-batches without dynamic memory allocation, which would introduce non-deterministic jitter.

Pipeline parallelism maps naturally to heterogeneous hardware. A typical SDR system might assign stages to different compute units:

• Stage 1 (DSP): A Digital Down Converter (DDC) and Burst Detection running on an FPGA's programmable logic.
• Stage 2 (Feature Extraction): A CORDIC algorithm for instantaneous phase calculation on the FPGA.
• Stage 3 (Inference): A quantized INT8 neural network model executing on an Edge TPU or a dedicated NPU. This spatial mapping allows each task to run on the most efficient silicon.

A robust pipeline must handle the case where a downstream stage becomes temporarily slower than upstream stages. Without control, this leads to buffer overflow and data loss. Backpressure handling is a flow control mechanism where a saturated stage signals its immediate upstream neighbor to halt production. This signal propagates backwards, pausing the entire pipeline until the bottleneck clears. This is essential for maintaining data integrity in a deterministic latency system processing a continuous IQ stream.

The end-to-end, low-latency data path that ingests, processes, and moves raw In-phase and Quadrature samples from an RF receiver to a classification model in real-time. In a pipelined architecture, this is the first stage, responsible for digitizing the analog world and presenting a continuous flow of complex samples to the downstream DSP blocks. Key considerations include:

• Sample rate matching to the classifier's input requirements
• Buffer sizing to prevent overflow during processing spikes
• Timestamping for coherent multi-channel operation

The architectural practice of moving computationally intensive signal processing tasks from a general-purpose CPU to a Field-Programmable Gate Array. In a pipeline parallelism context, FPGAs form the preprocessing stage, executing operations like channelization, FFTs, and digital down-conversion with deterministic latency. This frees the CPU or GPU to focus exclusively on neural network inference, maximizing overall system throughput. Common offload tasks include:

• Polyphase filter banks for wideband channelization
• CORDIC-based digital mixing and NCOs
• CFAR algorithms for burst detection and triggering

A memory management technique where data is transferred between processing stages by passing pointers rather than physically copying the data. In a pipelined classifier, this is critical for minimizing CPU overhead and maintaining deterministic latency between the DSP stage and the inference engine. Without zero-copy, the act of duplicating high-bandwidth IQ streams can consume a significant fraction of the latency budget. Implementations typically rely on:

• Shared memory pools accessible by both producer and consumer
• Circular buffers for continuous streaming without allocation
• DMA engines on FPGAs for direct memory access

The maximum allowable time, typically in microseconds or milliseconds, allocated for a neural network to perform a single forward pass and return a modulation classification result. This budget dictates the pipeline stage boundaries and determines whether a stage must be split across multiple compute units. For real-time tactical systems, the budget is often sub-millisecond, requiring aggressive optimizations such as INT8 quantization, kernel fusion via TensorRT, and bare-metal inference to eliminate OS scheduling jitter.

A flow control mechanism that prevents data loss by signaling upstream producers to slow down when a downstream processing stage is saturated. In a pipelined modulation classifier, the inference engine may become a bottleneck during complex signal environments. Backpressure ensures that the IQ streaming pipeline does not overflow buffers, which would cause sample drops and missed detections. Common strategies include:

• Credit-based flow control where the consumer grants processing tokens
• Ring buffer watermarks that trigger throttling at predefined fill levels
• Adaptive decimation to reduce sample rate under load

A compression technique that reduces the numerical precision of a neural network's weights and activations, typically from 32-bit floating point to 8-bit integer. This is the primary enabler for fitting a modulation classifier into a tight pipeline stage on edge hardware. INT8 inference on an Edge TPU or FPGA can achieve 4x throughput improvement and 4x memory reduction compared to FP32, with negligible accuracy loss for most modulation recognition tasks. Post-training quantization is the most common approach for deployed RF systems.