Data Preprocessing

The process of identifying and correcting errors, inconsistencies, and missing values in a dataset. For multimodal data, this is complex as issues vary by modality.

• Handling Missing Data: Techniques include mean/median imputation for numerical features, mode imputation for categorical data, or using predictive models (k-NN) to estimate missing values. For sequences, forward/backward fill may be used.
• Outlier Detection: Statistical methods like the Interquartile Range (IQR) or Z-score identify anomalous data points. In images, outliers could be corrupted files; in text, they might be gibberish strings.
• Deduplication: Critical for preventing model bias. Involves identifying and removing exact or near-duplicate records across text, image, or tabular data.

Rescaling numerical features to a common range or distribution to ensure stable and efficient model training, especially when features have different units or scales.

• Min-Max Normalization: Scales features to a fixed range, typically [0, 1]. Formula: X_scaled = (X - X_min) / (X_max - X_min). Common for pixel values in images.
• Z-Score Standardization: Transforms data to have a mean of 0 and a standard deviation of 1. Formula: X_standardized = (X - μ) / σ. Essential for algorithms like SVMs and linear models that assume centered data.
• Robust Scaling: Uses the median and interquartile range, making it resistant to outliers. Crucial for sensor data or financial metrics with extreme values.

Converting non-numeric categorical variables into a numerical format that machine learning algorithms can process.

• One-Hot Encoding: Creates new binary columns for each category. A value of 1 indicates the presence of that category. Suitable for nominal data (e.g., city names) without an intrinsic order. Can lead to high dimensionality (the "curse of dimensionality").
• Ordinal Encoding: Assigns each unique category an integer value. Used for ordinal data where categories have a meaningful order (e.g., "low," "medium," "high").
• Target Encoding (Mean Encoding): Replaces a category with the mean of the target variable for that category. Powerful but risks data leakage if not calculated carefully using out-of-fold or cross-validation methods.

The art of creating new, informative input features from raw data using domain knowledge, often to improve model performance.

• Domain-Specific Features: From timestamps, extract day-of-week, hour, or time-since-event. From text, create features like sentence length, presence of specific keywords, or sentiment score.
• Dimensionality Reduction: Techniques like Principal Component Analysis (PCA) or t-SNE transform high-dimensional data (e.g., image pixels, word embeddings) into a lower-dimensional space while preserving variance or structure, aiding visualization and efficiency.
• Polynomial Features: Creating interaction terms (e.g., feature1 * feature2) or polynomial terms (e.g., feature^2) can help linear models capture non-linear relationships in the data.

Artificially expanding the training dataset by applying realistic transformations to existing samples, improving model generalization and robustness.

• For Images: Random rotations, flips, cropping, color jitter (brightness, contrast), and adding noise.
• For Text: Synonym replacement, random insertion/deletion, back-translation (translating to another language and back), and sentence shuffling for some tasks.
• For Audio: Adding background noise, shifting time, changing pitch or speed.
• Crucial Note: Augmentations must be label-preserving. Flipping a "6" might turn it into a "9," changing its class. For multimodal data, augmentations must be applied consistently across aligned modalities (e.g., same crop to an image and its corresponding text caption).

Strategically partitioning the dataset into distinct sets for training, validation, and testing to reliably evaluate model performance.

• Train-Validation-Test Split: The standard partition (e.g., 70%-15%-15%). The training set is used to learn model parameters. The validation set is used for hyperparameter tuning and model selection. The hold-out test set provides a final, unbiased estimate of generalization error.
• Stratified Sampling: Ensures that the class distribution (or a key feature's distribution) is preserved across all splits. Vital for imbalanced datasets to prevent a split from missing a rare class entirely.
• Temporal Splitting: For time-series data, splits must be chronological (train on past, validate/test on future) to avoid data leakage from the future and simulate real-world deployment.

The process of using domain knowledge to create new input features from raw data or transform existing ones to improve model performance. This is a creative, often manual step that precedes automated preprocessing.

• Examples: Creating a 'time_of_day' feature from a timestamp, calculating ratios between existing columns, or applying polynomial expansions.
• Contrast with Preprocessing: While preprocessing (e.g., normalization) makes data consumable, feature engineering makes it informative. It directly addresses the model's ability to learn patterns.

A set of techniques to artificially expand a training dataset by applying random, realistic transformations to existing samples. It acts as a regularization method to improve model generalization.

• For Images: Random rotations, flips, cropping, color jitter, and noise addition.
• For Text: Synonym replacement, random word insertion/deletion, back-translation.
• For Audio: Adding background noise, shifting pitch or speed, time masking.
• Purpose: Increases effective dataset size, teaches invariance to irrelevant variations, and reduces overfitting.

An automated sequence of processes (a Directed Acyclic Graph or DAG) that ingests, transforms, validates, and moves data from source systems to a destination like a model training service. Preprocessing is a core stage within this pipeline.

• Key Components: Source connectors, transformation logic (cleaning, normalization), quality checks, and sinks (e.g., feature store, model training job).
• Orchestration Tools: Apache Airflow, Prefect, and Kubeflow Pipelines define and schedule these workflows.
• Importance: Ensures reproducible, reliable, and scalable flow of prepared data from raw sources to production models.

The process of temporally, spatially, or semantically synchronizing data points from different modalities or sources so they correspond to the same real-world event. This is critical for multimodal preprocessing.

• Temporal Alignment: Syncing audio waveforms with video frames or sensor telemetry timestamps.
• Spatial Alignment: Registering different camera views or LiDAR point clouds to a common coordinate system.
• Semantic Alignment: Ensuring a text caption accurately describes the content of a paired image.
• Challenge: Different data streams have varying sampling rates, latencies, and formats, requiring sophisticated interpolation and matching algorithms.

A sequence of deterministic transformations applied to rescale feature values to a standard range or distribution (e.g., [0,1] or a mean of 0, std of 1). This is a foundational preprocessing step for numerical data.

• Z-Score Normalization: (x - mean) / standard deviation. Assumes a roughly Gaussian distribution.
• Min-Max Scaling: (x - min) / (max - min). Scales features to a fixed range, often [0,1].
• Robust Scaling: Uses median and interquartile range, making it resistant to outliers.
• Why it's needed: Prevents features with large numerical ranges from dominating the model's loss function and accelerates gradient descent convergence.

The process of breaking down raw sequential data (text, audio, genomics) into smaller, discrete units called tokens, which serve as the fundamental input for models. This is modality-specific preprocessing.

• Text: Using algorithms like Byte-Pair Encoding (BPE) or WordPiece to split text into subwords.
• Audio: Converting raw waveform to spectrograms, then patching them into visual tokens for audio models.
• Video: Decomposing into individual frames (image tokens) and possibly applying a spatial patchifier.
• Output: A sequence of integer IDs that map to entries in a model's vocabulary, ready for embedding lookup.