Neural Bandit

Unlike LinUCB, which assumes a linear relationship between context features and expected reward, a neural bandit uses a deep neural network as the function approximator. This allows the model to capture complex, non-linear interactions between user features, item attributes, and session context. For example, the interaction between a user's past purchase category and their current time-of-day browsing pattern can be modeled as a non-linear combination, leading to more accurate click-through rate predictions.

A core advantage is the network's ability to learn useful latent representations of the context. The final hidden layers of the neural network act as a learned contextual feature vector, automatically extracting the most salient features for the decision task. This eliminates the need for extensive manual feature engineering. The shared representations learned for one action can also improve predictions for other actions with sparse data, mitigating the cold-start problem.

To balance the exploration-exploitation trade-off, neural bandits must estimate the uncertainty of their predictions. Common approaches include:

• Bayesian methods: Using dropout as a Bayesian approximation to model epistemic uncertainty.
• Ensemble methods: Maintaining a bootstrapped ensemble of neural networks and using the variance of their predictions as an exploration bonus.
• NeuralUCB: Using the network's gradient as a feature map for a linear Upper Confidence Bound (UCB) in the final layer, providing a tractable confidence interval for non-linear models.

Neural bandits operate in the online learning paradigm. Instead of batch retraining, the network's weights are updated incrementally via stochastic gradient descent (SGD) as each new bandit feedback event arrives. This allows the model to adapt continuously to non-stationary user behavior and contextual drift without requiring a full retraining cycle, maintaining high model freshness in dynamic retail environments.

Neural networks naturally handle large, structured action spaces where actions have rich metadata. By passing action features through the network alongside user context, the model can generalize across similar items. This is critical for dynamic assortment optimization in e-commerce, where the catalog contains millions of SKUs. The network learns to map item attributes to expected rewards, allowing it to make informed decisions even for items with limited historical interaction data.

In retail, a purchase may occur minutes or hours after a recommendation click. Neural bandits can be trained to handle delayed rewards by storing the context-action pair and using the eventual conversion signal as a supervised label. Techniques like reward shaping or training on proxy metrics (e.g., add-to-cart) provide immediate feedback, while the final purchase signal is back-propagated to refine the network's long-term reward signal estimation.

Neural Bandits power the personalized ranking of product carousels on e-commerce homepages. Unlike static collaborative filtering, the network ingests real-time session features (scroll depth, time-on-page, referral source) and user embedding vectors to predict click-through probability.

• Models complex, non-linear interactions between user affinity and item attributes.
• Continuously adapts to trending products without a full model retraining cycle.
• Balances showing proven best-sellers (exploitation) with surfacing new inventory (exploration).

Social media platforms use Neural Bandits to sequence content in infinite-scroll feeds. The deep network models the sequential dependency between items to maximize long-term engagement, not just immediate clicks.

• Input features include article embeddings, publisher authority scores, and user fatigue signals.
• The non-linear function approximator captures saturation effects where showing too many similar topics degrades the reward signal.
• Handles massive action spaces by combining candidate generation with neural scoring.

Demand-side platforms deploy Neural Bandits to select the optimal ad creative variant for each impression in programmatic auctions. The network evaluates contextual feature vectors including device type, geolocation, and browsing history.

• Predicts the uplift of a specific creative over a generic baseline.
• Learns quickly from sparse bandit feedback where only the clicked ad's outcome is observed.
• Uses Thompson Sampling on the output layer to naturally balance exploration without a fixed epsilon schedule.

Pharmaceutical researchers apply Neural Bandits to optimize dose-finding in adaptive clinical trials. The model learns the complex, non-linear efficacy-toxicity response surface across diverse patient subgroups.

• Context includes genetic biomarkers, demographic data, and metabolic indicators.
• Minimizes regret by reducing the number of patients assigned to suboptimal or toxic doses.
• The neural architecture captures synergistic drug interactions that linear dose-response models miss.

Mobile apps use Neural Bandits to determine the optimal moment and message content for push notifications. The network models the probability of app opens given the user's current context to prevent notification fatigue.

• Features include time since last session, current device motion state, and historical engagement patterns.
• Addresses the delayed reward problem where the notification's value is realized minutes or hours later.
• Explicitly models the negative reward of an uninstall to avoid aggressive messaging strategies.

Ride-sharing and hospitality platforms leverage Neural Bandits for real-time price elasticity modeling. The deep network ingests supply density, competitor pricing, and temporal features to set prices that maximize revenue per transaction.

• Captures the non-linear relationship between price multipliers and conversion probability.
• Operates in a non-stationary bandit environment where demand curves shift during weather events or holidays.
• Uses counterfactual evaluation to safely estimate the revenue impact of a new pricing policy before deployment.