Glossary

ViterbiNet

A model-based deep learning receiver integrating the Viterbi algorithm's structure with a neural network to decode sequences over channels with unknown, complex memory by learning the branch metric function.

Get in touch Learn more

ML engineer managing model training cluster on laptop, GPU utilization visible, technical deep learning setup.

MODEL-BASED DEEP RECEIVER

What is ViterbiNet?

ViterbiNet is a model-based deep learning receiver that integrates the structural algorithm of the Viterbi decoder with a neural network to perform sequence detection over channels with unknown, complex memory.

ViterbiNet is a symbol detection architecture that replaces the hand-crafted branch metric calculation in a classic Viterbi algorithm with a learned, data-driven function. By embedding a small neural network into the trellis processing loop, it retains the optimal dynamic programming structure for sequence estimation while learning the channel's conditional probability density function directly from pilot data.

This hybrid approach enables robust decoding over channels with unknown and analytically intractable memory, such as those exhibiting non-linear phase noise or hardware impairments. Unlike a purely learned black-box decoder, ViterbiNet provides the interpretability and reliability of a classical model-based algorithm while achieving the adaptability of a deep neural network, requiring significantly less training data than an end-to-end autoencoder.

ARCHITECTURE

Key Features of ViterbiNet

ViterbiNet is a model-based deep learning receiver that integrates the classic Viterbi algorithm's structure with a neural network, learning to decode sequences over channels with unknown, complex memory by replacing the hand-crafted branch metric calculation with a learned function.

Model-Based Deep Learning Architecture

ViterbiNet preserves the Viterbi algorithm's trellis structure as a non-trainable computational skeleton while replacing the branch metric calculation with a trainable neural network. This hybrid design combines the data efficiency of classical algorithms with the adaptability of deep learning, allowing the receiver to operate over channels with unknown memory without requiring explicit channel estimation.

Learned Branch Metrics via Classification

Instead of computing Euclidean distances to a known constellation, ViterbiNet trains a deep neural network classifier to estimate the probability of each possible transmitted symbol given the received sequence. The network outputs log-likelihood ratios that serve as the branch metrics for the Viterbi trellis search, effectively learning the channel's conditional distribution from pilot data.

Channel-Agnostic Operation

ViterbiNet does not require an explicit mathematical model of the channel. It learns to decode over channels with unknown intersymbol interference (ISI) , nonlinear distortions, or phase noise directly from training data. This makes it robust to hardware impairments and complex propagation environments where analytical channel models are unavailable or intractable.

Online Adaptation Capability

The neural branch metric network can be fine-tuned online during operation using received pilot sequences, allowing ViterbiNet to track time-varying channel conditions without redesigning the decoder. This meta-learning property enables rapid adaptation to new environments with minimal retraining overhead.

Computational Complexity Parity

ViterbiNet maintains the same trellis decoding complexity as the classical Viterbi algorithm, with the neural network inference replacing the branch metric computation. For channels with moderate memory length, the added neural computation is negligible compared to the trellis search, making it suitable for real-time implementation on embedded hardware.

Integration with Classical Decoders

ViterbiNet can be cascaded with channel decoders like LDPC or Turbo codes, providing soft-output log-likelihood ratios as input to iterative belief propagation decoders. This enables joint detection and decoding in a modular fashion, where the learned detector interfaces seamlessly with standardized error correction pipelines.

VITERBINET EXPLAINED

Frequently Asked Questions

Clear, technical answers to the most common questions about ViterbiNet, the model-based deep learning receiver that integrates the Viterbi algorithm's structure with a neural network for decoding over channels with unknown, complex memory.

ViterbiNet is a model-based deep learning receiver that integrates the structural skeleton of the classical Viterbi algorithm with a neural network to perform sequence decoding over channels with unknown, complex memory. Instead of relying on a hand-crafted, analytically derived branch metric based on an assumed channel model, ViterbiNet replaces this metric with a learned function implemented by a small neural network. This network is trained to estimate the likelihood of a state transition directly from a window of received symbols, effectively learning the channel's conditional probability distribution from pilot data. During inference, the trained network computes the branch metrics for the Viterbi trellis, allowing the algorithm to perform maximum-likelihood sequence estimation without explicit knowledge of the underlying channel dynamics, such as inter-symbol interference patterns or non-linear hardware distortions.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

VITERBINET ECOSYSTEM

Related Terms

Core concepts that contextualize ViterbiNet within the broader landscape of model-based deep learning and learned communication systems.

Model-Based Autoencoder

A transceiver architecture that integrates known physical layer algorithmic structures as non-trainable layers within a neural network. ViterbiNet is a canonical example, embedding the Viterbi algorithm as a structural prior while replacing the hand-crafted branch metric calculation with a learned function. This approach improves data efficiency and interpretability compared to black-box deep learning.

Key Insight: Combines expert knowledge with data-driven learning
Benefit: Requires fewer training samples than a pure neural receiver
Contrast: A standard DeepRx replaces the entire chain, not just one component

Blind Equalization Network

A neural network receiver that jointly performs channel equalization and symbol detection directly from a received signal sequence without requiring a separate pilot-based channel estimation step. ViterbiNet extends this concept to channels with unknown, complex memory by learning the trellis branch metrics.

Shared Goal: Operate without explicit channel state information (CSI)
ViterbiNet Advantage: Handles inter-symbol interference (ISI) with a structured decoder
Use Case: Communication over channels with unknown non-linear memory effects

Differentiable Channel Model

A mathematical or neural surrogate model of a physical communication channel that allows gradients to backpropagate from the receiver loss to the transmitter parameters. ViterbiNet relies on a differentiable loss function to train the branch metric neural network.

Training Requirement: The Viterbi algorithm itself is non-differentiable, but the loss is computed post-decoding
Alternative: Use a Gumbel-Softmax relaxation for end-to-end gradient flow through discrete trellis paths
Impact: Enables gradient-based optimization of the entire learned metric function

Neural Network Demapper

A receiver component that uses a neural network to compute soft bit estimates (log-likelihood ratios) directly from received I/Q symbols. ViterbiNet generalizes this concept to sequence detection, learning a non-linear metric for the entire received sequence rather than per-symbol demapping.

Per-Symbol: Neural demapper learns a decision boundary for a single symbol
Sequence-Level: ViterbiNet learns a metric that accounts for temporal dependencies
Output: Both produce soft information for downstream channel decoding

Non-Coherent Autoencoder

An end-to-end learned transceiver designed to operate without explicit channel state information, learning robust representations that are invariant to unknown channel phase and amplitude variations. ViterbiNet addresses the receiver-side of this problem for channels with memory.

Autoencoder Approach: Jointly learns transmitter and receiver invariances
ViterbiNet Approach: Learns a robust decoding metric for a fixed transmitter
Synergy: A non-coherent transmitter paired with a ViterbiNet receiver for fully blind operation

Transformer Codec

A sequence-to-sequence communication model based on the self-attention mechanism, designed to process long temporal dependencies in coded bit streams for iterative decoding. ViterbiNet and Transformer codecs represent two distinct philosophies for handling sequence memory.

ViterbiNet: Explicit Markovian memory via a trellis structure
Transformer Codec: Implicit long-range memory via self-attention
Trade-off: ViterbiNet is more parameter-efficient for known memory lengths; Transformers handle arbitrary dependencies

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.