Online Learning

Unlike batch training, which processes the entire dataset at once, online learning updates model weights one sample at a time or in small mini-batches. This is typically achieved via Stochastic Gradient Descent (SGD) with a learning rate that controls the magnitude of each update. The model never sees the full dataset simultaneously, making it memory-efficient for massive, unbounded data streams. Each observation is processed, used to compute a gradient, and then discarded, allowing the model to learn from theoretically infinite data without storage constraints.

A defining capability of online learners is their ability to track non-stationary data distributions. When user behavior shifts—such as a sudden change in shopping patterns during a holiday season—the model automatically adjusts its decision boundary. Key mechanisms include:

• Forgetting factors: Exponentially decaying the influence of old observations
• Sliding windows: Maintaining a buffer of only the most recent N samples
• Change detection algorithms: Explicitly triggering model resets when drift is statistically detected This prevents the model from becoming stale and misaligned with current reality.

Online learning is theoretically grounded in regret analysis, which measures the cumulative difference between the model's performance and that of the best fixed strategy in hindsight. Algorithms like Online Gradient Descent and Follow-The-Regularized-Leader (FTRL) provide formal guarantees that the average regret approaches zero as the number of rounds increases. This framework is particularly relevant for ad placement and dynamic pricing, where every suboptimal decision incurs immediate revenue loss.

Online learning eliminates the need for periodic, resource-intensive batch retraining jobs. The model maintains only its current parameter vector in memory, not the historical dataset. This makes it ideal for edge deployment and resource-constrained environments. The computational cost per update is constant and predictable, enabling deployment on streaming infrastructure like Apache Kafka or Apache Flink, where models are embedded directly in the data pipeline and updated with micro-batch latency.

A primary vulnerability of naive online learning is catastrophic forgetting, where the model overwrites previously learned patterns when exposed to new data. This is especially problematic when data is non-stationary but exhibits recurring patterns (e.g., seasonal trends). Mitigation strategies include:

• Elastic Weight Consolidation (EWC): Penalizing changes to parameters important for prior tasks
• Experience replay: Interleaving new data with a small buffer of historical samples
• Progressive neural networks: Freezing old subnetworks while adding new capacity

Because online learners evolve continuously, they demand rigorous real-time observability. Unlike static batch models validated offline, online models can silently degrade. Essential monitoring includes:

• Prediction drift detection: Tracking the distribution of model outputs over time
• Feature drift monitoring: Alerting when input distributions shift beyond acceptable thresholds
• Performance metric streaming: Computing moving averages of business KPIs like click-through rate or conversion rate
• Canary deployments: Routing a small percentage of traffic to updated model versions before full rollout

The phenomenon where the statistical properties of the target variable change over time, rendering a static model obsolete. In online retail, this manifests as shifting user preferences during seasonal trends or a competitor's viral campaign. Virtual drift occurs when the input distribution P(X) changes, while actual drift indicates a change in the conditional relationship P(Y|X). Online learning is the primary mitigation strategy, allowing the decision boundary to continuously adapt without manual intervention.

The event-driven backbone that ingests, transforms, and delivers real-time data to an online learner. Architectures like Apache Kafka or Apache Flink sessionize raw clickstreams into structured training instances. Key requirements include:

• Exactly-once semantics to prevent duplicate updates
• Windowing logic to aggregate features over sliding time intervals
• Back-pressure handling to decouple ingestion from model training throughput

A critical failure mode where a neural network abruptly overwrites previously learned knowledge upon ingesting new data. In a product recommender, this could cause the model to forget a stable user preference after a single anomalous session. Mitigation strategies include elastic weight consolidation (EWC), which penalizes changes to parameters important for prior tasks, and experience replay, which interleaves historical samples with the live stream.

A centralized platform serving pre-computed and real-time features with low latency to online models. It bridges the training-serving skew gap by ensuring the exact same transformation logic is used at inference and update time. Core components include:

• Online store: In-memory database for sub-millisecond serving
• Offline store: Historical feature warehouse for backfilling
• Transformation registry: Versioned, reusable feature definitions

The partial-label learning signal inherent to personalization, where you only observe the outcome of the action taken, not the counterfactual outcomes of alternatives. Online learning algorithms must correct for this selection bias using techniques like inverse propensity scoring (IPS). Without this correction, a model trained naively on logged data will amplify existing biases, creating a self-reinforcing filter bubble.

The safety infrastructure required to prevent a continuously updating model from degrading silently. A champion-challenger architecture runs the candidate model in shadow mode, comparing predictions against the production champion on live traffic. Automated canary analysis triggers an instant rollback to the last known-good checkpoint if key metrics like NDCG@10 or conversion rate breach a predefined threshold.