Feature Encoding

Creates a binary column for each unique category, marking presence with a 1 and absence with a 0. This method prevents models from assuming an ordinal relationship where none exists.

• Mechanism: Transforms a single categorical feature with n distinct values into n sparse binary features.
• Use Case: Low-cardinality categorical data like country_code or product_color.
• Trade-off: Explodes feature dimensionality for high-cardinality columns, leading to the curse of dimensionality and increased memory consumption.

Assigns a unique integer to each category. While memory-efficient, this technique is primarily suited for tree-based models (like XGBoost or Random Forests) that can learn non-linear splits.

• Critical Warning: Using label encoding for linear models or neural networks introduces a false ordinal signal (e.g., 'Red'=2 is not 'greater than' 'Blue'=1).
• Best Practice: Reserve for ordinal categories with a natural rank order (e.g., education_level: High School < Bachelor's < Master's).

Replaces a categorical value with a blend of the posterior probability of the target variable for that category and the prior probability across the dataset. This directly captures the predictive signal of the category.

• Formula: Encoded_Value = (n * category_mean + m * global_mean) / (n + m) where m is a smoothing factor.
• Risk: Prone to data leakage if the target mean is calculated on the validation or test set. Strict cross-validation within the training set is mandatory.

Learns a dense, low-dimensional vector representation for each category during model training. Unlike static encoding, embeddings capture semantic similarity—similar categories end up close together in vector space.

• Architecture: An embedding layer maps each category index to a trainable weight vector. The weights are updated via backpropagation.
• Advantage: Handles high-cardinality features (e.g., product_sku, user_id) efficiently and reduces dimensionality compared to one-hot encoding.

Applies a hash function to a categorical value to deterministically map it to a fixed-size vector index. This eliminates the need for a vocabulary dictionary, making it ideal for streaming and online learning.

• Hashing Trick: index = hash(feature_value) % n_features
• Collision Handling: Multiple categories can map to the same index. The sign hash variant uses the sign of the hash to mitigate collision impact.
• Use Case: Text classification with massive, dynamic vocabularies where maintaining a dictionary is infeasible.

Maps categories to ordered integers based on a predefined, meaningful rank. This is distinct from label encoding because the numerical order reflects a true business logic sequence.

• Example: customer_tier mapped as Bronze=0, Silver=1, Gold=2.
• Model Compatibility: Safe for linear models and neural networks because the magnitude of the integer directly corresponds to the magnitude of the attribute.
• Contrast: Do not confuse with arbitrary label encoding where the integer assignment is random.

Creates a binary column for each unique category, marking a 1 where the category is present and 0 elsewhere. This prevents models from assuming an ordinal relationship between unrelated categories like 'Red', 'Blue', and 'Green'.

• Sparsity: Produces a wide, sparse matrix for high-cardinality features.
• Dummy Variable Trap: Requires dropping one column to avoid multicollinearity in linear models.
• Use Case: Ideal for nominal categories with low cardinality (< 15 unique values).

Assigns a unique integer to each category, such as Red=0, Blue=1, Green=2. While memory-efficient, it introduces an artificial ordinal relationship that can mislead non-tree-based models into interpreting Green as 'greater than' Blue.

• Tree Compatibility: Safe for tree-based models (XGBoost, Random Forest) that split on arbitrary thresholds.
• Risk: Catastrophic for linear regression or neural networks without subsequent embedding layers.
• Use Case: Target variables or high-cardinality features destined for embedding layers.

Replaces a category with a blended statistic of the target variable, such as the mean of the target for that category. This directly captures the predictive relationship but carries a high risk of data leakage if not cross-validated properly.

• Smoothing: Applies a blending factor to prevent overfitting on rare categories.
• Leakage Prevention: Must be calculated strictly within each cross-validation fold.
• Use Case: High-cardinality categorical features in supervised learning tasks.

Learns a dense, low-dimensional vector representation for each category during model training. Unlike static encoding, embeddings capture semantic similarity—categories that behave similarly in relation to the target will have vectors close together in latent space.

• Dimensionality Reduction: Maps thousands of categories to a compact vector (e.g., 50 dimensions).
• Transfer Learning: Learned embeddings can be reused across different models.
• Use Case: High-cardinality features like user_id or product_sku in deep learning recommenders.

Applies a hash function to map categories to a fixed-size vector, avoiding the need to maintain a vocabulary. Collisions occur when different categories map to the same index, but this trade-off is often acceptable for memory-constrained, streaming environments.

• Stateless: Requires no storage of category mappings, ideal for online learning.
• Collision Rate: Controlled by the chosen output dimensionality.
• Use Case: Text classification with massive, unbounded vocabularies in production feature pipelines.

Maps categories to ordered integers based on a defined ranking, such as Low=1, Medium=2, High=3. This explicitly preserves the meaningful order that label encoding falsely implies for nominal data.

• Manual Mapping: Requires a predefined, logical ordering of categories.
• Monotonic Relationship: Asserts a positive or negative trend with the target.
• Use Case: Education level, customer tier, or survey responses where the order is intrinsic to the data.