Model Versioning

The primary, serialized output of the training process. This is the executable file containing the learned parameters (weights and biases) and the computational graph. Common formats include:

• SavedModel (TensorFlow)
• .pt or .pth files (PyTorch)
• ONNX (Open Neural Network Exchange)
• Pickle (Python serialization, use with caution) This artifact is loaded by the model serving infrastructure to perform inference.

The exact source code, libraries, and configuration files used to produce the model artifact. This ensures reproducibility and includes:

• Training script with the model architecture definition.
• Hyperparameters (e.g., learning rate, batch size, optimizer settings).
• Environment specification (e.g., requirements.txt, Dockerfile, conda.yaml).
• Pipeline definitions (e.g., Kubeflow Pipelines YAML, Airflow DAGs). Without this, a model artifact is a black box and cannot be reliably retrained or debugged.

A precise reference to the dataset used for training. In practice, storing the full dataset is often impractical, so versioning focuses on provenance:

• Data version identifiers (e.g., commit hash in a Data Version Control system, object store path with timestamp).
• Data schema defining feature names, types, and constraints.
• Data preprocessing and transformation logic (e.g., scaling parameters, tokenizer vocabularies).
• Lineage linking back to raw data sources. This is critical for diagnosing data drift and for compliance audits.

Quantitative and qualitative assessments of the model's performance at the time of version creation. This establishes the performance baseline for future comparison and includes:

• Primary metrics (e.g., accuracy, F1-score, perplexity) on hold-out validation and test sets.
• Slice-based evaluation showing performance across key user segments or data cohorts.
• Bias and fairness metrics (e.g., demographic parity, equalized odds).
• Generated outputs from a standard evaluation prompt set for LLMs. These reports are stored as immutable artifacts (e.g., JSON, HTML) within the model registry.

Structured and unstructured information that provides context for the model version. This is essential for discovery, governance, and operational use.

• Basic Metadata: Author, creation timestamp, version tag (e.g., v1.2.3), parent version.
• Operational Metadata: Expected latency, hardware requirements, inference cost profile.
• Governance Metadata: Intended use case, known limitations, license information.
• Model Card: A standardized document detailing performance characteristics, ethical considerations, and recommended usage guidelines, as pioneered by Google's Model Cards for AI.

The complete software context required to load the artifact and run inference or retraining. This guarantees environment parity between development and production.

• Framework versions (e.g., TensorFlow 2.15.0, PyTorch 2.2.0, Transformers 4.36.0).
• Hardware drivers and libraries (e.g., CUDA 12.1, cuDNN 8.9).
• System libraries and OS details. This is typically captured via containerization (a Docker image digest) or a detailed environment manifest, forming the basis for the model packaging process.

Model versioning creates an immutable audit trail linking every model artifact to the exact code, data, and hyperparameters that produced it. This is critical for:

• Regulatory Compliance: Providing evidence for audits under frameworks like the EU AI Act.
• Debugging & Root Cause Analysis: Precisely recreating a model to diagnose performance drops or failures.
• Scientific Rigor: Ensuring experimental results can be independently verified, a cornerstone of Evaluation-Driven Development. Without versioning, models are 'black boxes' whose origins and behavior cannot be reliably explained.

Versioning enables low-risk innovation by allowing teams to test new models without jeopardizing production stability. It supports deployment strategies like:

• Canary Deployments: Releasing a new version to a small user segment.
• Shadow Deployment: Processing live traffic in parallel without affecting users.
• Instant Rollback: If a new version (Model Challenger) underperforms, traffic can be instantly reverted to the stable Model Champion. This creates a safety net that encourages continuous model improvement and Continuous Retraining.

A centralized Model Registry powered by versioning acts as a single source of truth for all ML assets. This scales team collaboration by:

• Eliminating 'Works on My Machine' Issues: Ensuring all engineers use the same artifact versions.
• Streamlining Approval Workflows: Linking model versions to formal Governance Policies and Validation Gates required for promotion.
• Enabling Model Lineage: Clearly tracing which dataset version led to which model, which is essential for managing Data Drift and Concept Drift.

By tracking model performance per version, organizations can make data-driven decisions that directly impact infrastructure spend. Benefits include:

• Preventing Costly Redundancy: Avoiding duplicate training runs for models that already exist.
• Informed Retirement: Archiving underperforming or unused models to free up storage and compute resources.
• Efficient Rollouts: Using version comparison to deploy only models that demonstrate a clear performance or efficiency gain, optimizing Inference costs. This is a key concern for CTOs and FinOps teams.

Model versioning is the enabler for CI/CD for ML (MLOps Pipelines). By automating the tracking and promotion of models, it reduces manual toil and cycle times:

• Automated Promotion: Models that pass automated tests can be versioned and promoted from staging to production without manual intervention.
• Parallel Development: Multiple data scientists can experiment independently, merging their versioned models into a shared pipeline.
• Faster Incident Response: When Drift Detection triggers a Retraining Trigger, the new model can be versioned, validated, and deployed through an automated pipeline, minimizing downtime.

Treating models as versioned, immutable artifacts protects against unauthorized changes and corruption. This supports:

• Tamper-Evident Logs: Any attempt to alter a production model artifact creates a new version, leaving a clear record in the Audit Trail.
• Supply Chain Security: Versioning dependencies (like training libraries) as part of the model's metadata helps identify vulnerabilities.
• Reliable Recovery: In the event of a system failure or a security incident, a known-good model version can be rapidly redeployed from the registry, ensuring business continuity.

A comprehensive, immutable record that traces the origin, transformations, and dependencies of a specific model version. It answers the question: "What created this model?"

• Data Provenance: Links the model to the exact dataset and preprocessing code version used for training.
• Code Versioning: Records the commit hash of the training script and library dependencies.
• Hyperparameter Tracking: Captures the configuration used for the training run.
• Artifact Chain: Shows relationships between intermediate outputs (e.g., feature encoders, preprocessors) and the final model. This is critical for auditability, debugging, and reproducibility, especially under regulatory scrutiny.

Any file or object produced during the machine learning lifecycle that constitutes a versioned component of the model. Versioning must encompass all artifacts, not just the final weights. Core Artifacts include:

• Serialized Weights: The trained model parameters (e.g., .pt, .h5, .joblib files).
• Model Schema: A file defining the expected input features, data types, and output structure.
• Preprocessing Pipelines: Fitted transformers (e.g., scalers, encoders) required to prepare raw data for the model.
• Evaluation Reports: Performance metrics, confusion matrices, and fairness assessments for that specific version.
• Model Card: Documentation detailing intended use, limitations, and ethical considerations.

The technical process of converting a trained model's in-memory state (weights, architecture, configuration) into a persistent, storable file format. This creates the primary artifact for versioning. Key Formats and Frameworks:

• Pickle (Python): A common but potentially insecure format for scikit-learn models.
• ONNX (Open Neural Network Exchange): A standardized, framework-agnostic format for deep learning models, promoting interoperability.
• TensorFlow SavedModel: A comprehensive format that includes the computation graph, weights, and assets.
• PyTorch torch.save(): Saves model state dictionaries or entire scripted modules. The choice of serialization format directly impacts deployment flexibility, performance, and long-term maintainability of versioned models.

The practice of periodically saving the intermediate state of a model during a training run. These checkpoints are versioned snapshots that enable:

• Fault Recovery: Resume training from the last checkpoint in case of hardware failure.
• Hyperparameter Exploration: Compare the performance of different intermediate states without retraining from scratch.
• Early Stopping: Select the best-performing checkpoint based on validation metrics, not just the final epoch.
• Fine-Tuning: Use a specific checkpoint as a starting point for further training on new data. Checkpoints are a form of temporal versioning within a single experiment, capturing the model's evolution over training time.

A core principle of robust model versioning where a versioned model artifact, once created and registered, cannot be altered. Any change necessitates creating a new, distinct version. This immutability provides:

• Deterministic Reproducibility: The exact artifact used in production can always be retrieved and redeployed.
• Reliable Rollbacks: Reverting to a previous version is guaranteed to restore the identical model behavior.
• Strong Audit Trails: Eliminates ambiguity about what was deployed at any point in time. Immutable versioning is typically enforced by content-addressable storage (e.g., using a hash like SHA-256 as the version ID) within a Model Registry.