Feature Engineering

Mathematical operations applied to raw variables to adjust their scale or distribution for optimal model performance. Normalization rescales values to a 0-1 range, while standardization centers data around a mean of 0 with unit variance. Log transformations reduce skewness in heavy-tailed distributions like transaction amounts. These techniques prevent gradient-based optimizers from oscillating and ensure distance-based algorithms like k-means aren't biased toward features with larger magnitudes.

Converting non-numeric data into vectors that algorithms can process. One-hot encoding creates binary columns for each category, suitable for low-cardinality fields like 'color'. Target encoding replaces categories with the mean target value, capturing predictive power for high-cardinality fields like 'zip code'. Embeddings learn dense, low-dimensional representations that capture semantic relationships between categories, essential for deep learning recommenders.

Deriving predictive signals from timestamps to capture behavioral patterns. Key extractions include:

• Recency: Time since last purchase
• Frequency: Count of events in a rolling window
• Monetary value: Total spend over a period
• Cyclical encoding: Transforming hour-of-day or day-of-week into sine/cosine coordinates to preserve the circular nature of time for neural networks

Synthetic attributes created by combining two or more raw variables to capture non-linear relationships. Cross-product features multiply two numerical values, such as 'session duration × pages viewed' to quantify engagement depth. Feature crosses concatenate categorical values, like 'country × device type', enabling linear models to learn segment-specific behavior without complex architectures. These are critical for capturing multiplicative effects in click-through rate prediction.

Strategies for handling absent data without discarding valuable records. Mean/median imputation fills gaps with central tendency, suitable for random missingness. Predictive imputation trains a separate model to estimate missing values from other features. Indicator variables create a binary flag marking imputed rows, allowing the model to learn that 'missingness' itself may be a signal—for example, a missing 'income' field might correlate with a specific user segment.

Algorithmic generation of candidate features to accelerate discovery. Deep Feature Synthesis iteratively applies mathematical primitives (sum, mean, max, trend) across relational tables to create thousands of features. Featuretools and similar frameworks traverse entity relationships to generate features like 'average time between orders for a customer's product category'. This systematic approach often surfaces non-intuitive but highly predictive patterns that manual analysis would miss.