Online Learning

Unlike batch learning, online learning processes data one observation at a time as it arrives. The model ingests a stream of examples (x_t, y_t) and updates its parameters immediately after each observation. This eliminates the need to store massive historical datasets and enables learning from unbounded data streams where the total volume may be unknown or infinite. Common algorithms include Stochastic Gradient Descent (SGD) applied incrementally and Passive-Aggressive classifiers.

Online learners naturally handle non-stationary distributions by continuously adjusting to new patterns. When user behavior shifts—such as a sudden change in purchasing trends during a holiday season—the model adapts without manual intervention. Techniques include:

• Exponential forgetting: applying decay factors to down-weight old observations
• Sliding windows: maintaining a fixed-size buffer of recent examples
• Change detection: triggering model resets when performance degrades beyond a threshold

Online learning algorithms are evaluated using regret, which measures the cumulative difference between the algorithm's performance and the best fixed model in hindsight. A no-regret algorithm guarantees that the average regret per round approaches zero as the number of rounds grows. This theoretical framework, rooted in convex optimization and game theory, provides formal guarantees even when facing adversarial data sequences rather than i.i.d. assumptions.

Online learning operates with O(1) memory per update for linear models, making it ideal for resource-constrained edge devices and high-throughput production systems. The model maintains only its current parameter vector rather than a full training corpus. This efficiency enables:

• Real-time personalization on mobile devices
• Continuous training on IoT sensor streams
• Cost-effective model updates without GPU clusters for retraining

Online learning is the foundational paradigm behind contextual multi-armed bandits. In a bandit setting, the model must learn from partial feedback—observing only the reward for the chosen action, not counterfactual outcomes. Online learning algorithms like Online Gradient Descent are adapted to the bandit setting through techniques such as inverse propensity scoring (IPS) to debias the observed rewards. This connection enables real-time personalization systems that balance exploration and exploitation.

Online learning systems in production typically follow a closed-loop architecture:

• Event ingestion: user actions streamed via Kafka or Kinesis
• Feature computation: real-time feature stores serve context vectors
• Model update: the learner updates weights after observing the outcome
• Prediction serving: updated model immediately serves new predictions This contrasts with the traditional offline-batch-online-serve pattern where models are retrained periodically and pushed to production in a separate deployment step.

Online learning is the backbone of programmatic advertising. Models continuously update click-through rate (CTR) predictions based on streaming impression data, adjusting bids in milliseconds. A contextual bandit selects ad creatives while an online logistic regression model updates its weights with each new click or conversion event. This closed-loop system processes millions of events per second, with model freshness measured in minutes rather than days.

E-commerce platforms and ride-sharing services use online learning to adjust prices in response to real-time supply and demand signals. A stochastic gradient descent (SGD) regressor updates its coefficients as each transaction completes, incorporating features like current inventory levels, competitor pricing, and time-of-day patterns. The model must handle concept drift gracefully—a sudden weather event or competitor flash sale shifts the demand curve instantly, and the online learner adapts without manual retraining.

Financial institutions deploy online learning to catch fraudulent transactions as patterns evolve. Unlike batch models that retrain weekly, an online anomaly detector updates its decision boundary with each flagged transaction. Key techniques include:

• Incremental PCA for updating feature distributions
• Online gradient boosting for adapting ensemble weights
• Exponential moving averages to decay stale observations This architecture catches zero-day fraud patterns that static models miss entirely.

Content platforms use online learning to personalize feeds as user interests shift throughout the day. A factorization machine trained via Follow-the-Regularized-Leader (FTRL) updates user and item embeddings with each scroll, click, or dismiss action. The system handles the cold-start problem by rapidly incorporating signals from new articles and users, while exploration heuristics inject diversity to prevent filter bubbles. Model updates propagate in under 60 seconds from user action to refreshed recommendations.

Industrial IoT systems stream sensor telemetry to online learning models that predict equipment failure. A Hoeffding Tree classifier—a decision tree designed for data streams—updates its split criteria incrementally as vibration, temperature, and pressure readings arrive. The model never stores raw historical data, making it memory-efficient for edge deployment. When a new failure mode emerges, the tree adapts its structure without catastrophic forgetting of previously learned patterns.

Self-driving systems use online learning to adapt perception models to changing environmental conditions. A continual learning framework updates object detection weights as the vehicle encounters new weather patterns, road types, or lighting conditions. Techniques like elastic weight consolidation (EWC) prevent catastrophic forgetting of previously learned scenarios while allowing rapid adaptation. The model learns from each mile driven, improving lane detection in heavy rain without forgetting how to navigate clear highways.