LASSO Regression

LASSO adds a penalty equal to the absolute value of the magnitude of coefficients (L1 norm) to the ordinary least squares loss function. This constraint, controlled by the hyperparameter lambda (λ), forces the optimization to shrink coefficients toward zero. Unlike Ridge regression's L2 penalty, the L1 constraint creates a diamond-shaped feasible region that intersects the elliptical contours of the error surface at the axes, naturally driving some coefficients to exactly zero.

The defining characteristic of LASSO is its ability to perform embedded feature selection during model training. As λ increases, more coefficients are driven to exactly zero, effectively removing irrelevant predictors from the model. This is critical in alpha factor discovery where researchers test hundreds of candidate signals:

• Eliminates multicollinearity problems by selecting one representative from correlated groups
• Reduces overfitting in high-dimensional settings where p > n
• Produces sparse models that are computationally efficient in production

LASSO intentionally introduces bias into coefficient estimates to dramatically reduce variance. The shrinkage mechanism trades a small amount of in-sample accuracy for substantial out-of-sample stability. In financial applications, this is particularly valuable because:

• Unregularized models often fit noise in historical returns, leading to alpha decay
• The variance reduction improves the Information Coefficient (IC) on unseen data
• The bias introduced is a small price for avoiding catastrophic overfitting in regime-shifting markets

The regularization strength λ controls the degree of shrinkage and must be carefully tuned. Common approaches include:

• K-fold cross-validation: Partition data, train on k-1 folds, validate on held-out fold, and select λ that minimizes mean cross-validated error
• Information criteria: Use AIC or BIC to balance model fit against complexity, penalizing the number of non-zero coefficients
• One-standard-error rule: Choose the largest λ within one standard error of the minimum cross-validation error to favor sparsity

In walk-forward optimization for trading strategies, λ is typically re-estimated at each rebalancing period using an expanding or rolling window.

Despite its power, LASSO has important limitations for quantitative finance:

• Correlated predictors: When features are highly correlated, LASSO arbitrarily selects one and zeros out others, potentially discarding economically meaningful signals
• Linear assumptions: LASSO captures only linear relationships, missing complex non-linear interactions that neural network alpha models can exploit
• Post-selection inference: Standard p-values and confidence intervals are invalid after LASSO selection because the model has already peeked at the data

For these reasons, practitioners often use Elastic Net (combining L1 and L2 penalties) or employ LASSO as a screening step before applying non-linear models.

A typical LASSO workflow for alpha discovery follows these steps:

1. Data preparation: Standardize all features to zero mean and unit variance, as LASSO is scale-sensitive
2. Universe construction: Define the stock universe and point-in-time factor values to avoid look-ahead bias
3. Cross-sectional fitting: Fit LASSO at each time period to predict forward returns, producing a sparse coefficient vector
4. Signal aggregation: Combine the selected factors into a composite alpha signal
5. Orthogonalization: Optionally orthogonalize the resulting signal against known risk factors to isolate pure alpha

Libraries like scikit-learn's Lasso and LassoCV in Python, or the glmnet package in R, provide efficient coordinate descent implementations suitable for large-scale factor research.

LASSO uses L1 regularization (absolute value penalty), which can shrink coefficients to exactly zero, performing automatic feature selection. In contrast, Ridge regression uses L2 regularization (squared magnitude penalty), which shrinks coefficients toward zero but never eliminates them entirely. The geometric difference: L1 constraints create diamond-shaped feasible regions where solutions hit corners (zero coefficients), while L2 creates circular regions. Elastic Net combines both penalties to handle correlated features better than LASSO alone.

In alpha factor discovery, predictor variables are often highly correlated (e.g., multiple value metrics). LASSO's variable selection property directly addresses multicollinearity by selecting one representative from a group of correlated features and zeroing out the rest. This prevents the unstable coefficient estimates that plague ordinary least squares. However, LASSO's selection can be arbitrary among correlated features—the Elastic Net variant adds an L2 penalty to encourage grouped selection, making it preferred when factor correlation is high.

The lambda (λ) hyperparameter controls the strength of the L1 penalty. Higher λ values force more coefficients to zero, creating sparser models. Practitioners use k-fold cross-validation to find the optimal λ:

• Split data into k folds, train on k-1, validate on the held-out fold
• Repeat for a grid of λ values
• Select λ that minimizes mean squared error on validation sets
• The 'one standard error rule' often selects a slightly larger λ for a more parsimonious model In finance, walk-forward cross-validation is preferred to respect temporal ordering and avoid look-ahead bias.

LASSO is sensitive to feature scales because the L1 penalty applies uniformly to all coefficients. Without proper scaling, features with larger magnitudes dominate the penalty term. Essential preprocessing steps:

• Standardization: Transform each feature to mean=0, standard deviation=1
• Normalization: Scale features to [0,1] range
• Apply the same scaling parameters from training data to test data
• Never fit scalers on the full dataset before train/test split—this introduces look-ahead bias in financial applications Unscaled data leads to biased variable selection where LASSO incorrectly favors features with naturally larger numeric ranges.

The standard optimization method for LASSO is coordinate descent, which solves the non-differentiable L1 objective efficiently:

• Updates one coefficient at a time while holding others fixed
• Uses soft-thresholding operator: sign(β)(|β| - λ)+
• Cycles through all features until convergence
• Computationally efficient for high-dimensional problems (p >> n)
• Handles the L1 corner cases naturally without subgradient methods Modern implementations like glmnet use cyclical coordinate descent with active set strategies, ignoring zeroed-out features to accelerate convergence on sparse solutions.

In quantitative finance, LASSO addresses the curse of dimensionality when testing hundreds of potential alpha factors:

• Start with a large universe of candidate signals (value, momentum, quality, etc.)
• LASSO automatically selects the subset with genuine predictive power
• Zeroed-out coefficients indicate factors that add no incremental value
• Surviving non-zero coefficients represent the parsimonious alpha model
• Reduces overfitting risk compared to stepwise regression
• Provides interpretable models that quants can explain to portfolio managers Caveat: LASSO assumes linear relationships; for non-linear factor interactions, consider tree-based methods or neural networks with L1 regularization.