Contextual Assortment Bandit

Unlike a standard multi-armed bandit, this agent conditions its action selection on a context vector. This vector encodes the current state, including user demographics, session behavior, device type, time of day, and local inventory levels. The policy learns a mapping from this high-dimensional context to the expected reward of each product, enabling truly personalized assortment curation rather than a one-size-fits-all popularity contest.

The algorithm continuously navigates the trade-off between exploitation (showing proven high-performers) and exploration (testing new or long-tail items). It uses a strategy like Thompson Sampling or Upper Confidence Bound to quantify uncertainty. For example, a new product with few impressions has high uncertainty, prompting the agent to explore it. As data accumulates, the agent shifts toward exploiting items with a high, statistically significant reward, all while minimizing regret.

The reward signal is not solely based on clicks or conversions. It integrates a stockout penalty and an overstock boost. The agent receives a negative reward for recommending an item that is out of stock, and a positive reward for accelerating the sell-through of overstocked or perishable goods. This constraint satisfaction ensures the assortment optimization directly serves supply chain and revenue management goals, not just engagement metrics.

The underlying policy is updated incrementally as new interaction data streams in. Using online learning algorithms, the model adapts to shifting trends—such as a sudden weather change driving demand for umbrellas—without requiring a full batch retraining cycle. This low-latency feedback loop ensures the displayed assortment reflects the most current demand signals, preventing stale recommendations during high-velocity retail periods.

The agent's action is not a single item recommendation but a combinatorial selection of a full product grid or carousel. It evaluates the joint reward of a set of items, accounting for diversity and cannibalization. A slate-based policy gradient method might be used to learn that displaying two very similar items together reduces overall basket value, thus optimizing the entire visual layout for maximum session-level revenue.

For new items with no interaction history, the bandit leverages informative priors derived from product metadata, such as brand, category, and price point. A Bayesian framework initializes the new item's reward distribution based on the performance of similar items in the catalog. This allows the agent to intelligently explore new inventory from the moment it is added, rather than waiting for organic data to accumulate, solving a critical retail cold start problem.