Dark Launch

The core principle of a dark launch is that the new functionality executes silently within the production environment without affecting the user experience or visible output. User requests are processed by both the old and new systems, but only the results from the old, stable system are returned to the end-user. This allows for zero-impact testing on real user traffic.

A primary use case is to validate new infrastructure under real-world load conditions before a user-facing launch. This involves:

• Testing database queries, caching layers, and external API calls at production scale.
• Identifying bottlenecks in network I/O, memory usage, or GPU utilization for new models.
• Verifying autoscaling policies and resource provisioning under peak traffic. This prevents performance degradation when the feature is officially enabled.

In machine learning, dark launch is synonymous with shadow mode or shadow deployment. A new model version processes live inference requests in parallel with the champion model. Its predictions are logged and compared offline against the live model's outputs and ground truth (when available). Key metrics evaluated include:

• Prediction latency and throughput.
• Statistical differences in output distributions.
• Business metric performance (e.g., click-through rate, conversion) via offline replay.

Dark launches facilitate the collection of production-grade telemetry and validation data for the new system. This includes:

• Logging inputs, outputs, and internal state for analysis.
• Gathering a labeled dataset by recording model predictions and subsequent user actions (e.g., whether a recommended item was purchased).
• Validating assumptions about data schemas and distributions that may differ from staging environments.

Dark launch is typically controlled by feature flags or toggles. This allows for:

• Granular control: Enabling the dark launch for specific user segments, geographic regions, or percentages of traffic.
• Instant rollback: Disabling the new code path immediately if critical issues are detected, without a full deployment rollback.
• Phased graduation: Seamlessly transitioning from dark launch (0% user-facing) to a canary release (1% of users) to a full rollout.

This technique decouples the deployment of code from the release of functionality. The new code is shipped and runs in production, but its business logic is dormant. This allows:

• Reduced risk during deployment events, as the system's behavior does not change.
• The operations team to validate deployment health (e.g., smoke tests) before the product team 'flips the switch' to release.
• Compliance with strict release windows, as code can be deployed ahead of time and activated at a scheduled moment.

The new model runs in parallel with the current champion model, processing identical inputs. Its outputs are logged and compared offline against the live model's results and ground truth (when available). This generates a completely unbiased performance benchmark using live data, free from the selection bias that can affect offline testing datasets. Key metrics evaluated include:

• Prediction accuracy and business KPIs
• Output distribution shifts
• Latency profiles and P99 tail latency
• Resource efficiency (cost per inference) This data is essential for a confident go/no-go decision on a full rollout.

Dark launch is used to test complex new features or data pipelines that feed into the model, where failures could be catastrophic. For example, a new retrieval-augmented generation (RAG) system can be dark-launched: live user queries trigger the full RAG pipeline (query vectorization, semantic search, context augmentation, and generation), but the final answer is discarded. This validates the entire integration—including the vector database, context window management, and prompt formatting—ensuring no runtime errors or performance cliffs exist before the feature is user-visible.

Before a model can be improved, it needs data from its target domain. A dark launch allows the collection of a high-fidelity inference dataset from the exact production environment. This dataset includes:

• Raw input features as seen in production (post-any upstream transformations)
• The model's predictions
• Subsequent ground truth labels (collected via user feedback or system outcomes) This dataset is invaluable for diagnosing future concept drift, creating representative test sets, and performing targeted fine-tuning or model editing based on real-world edge cases.

Deploying the observability stack for a new model is as critical as deploying the model itself. A dark launch provides a safe period to verify that telemetry, logging, metrics, and alerting are functioning correctly. Teams can confirm:

• Latency histograms are being populated accurately
• Prediction drift detectors are receiving data
• Business metric calculations are correct
• Error tracking captures failed inferences This ensures that when the model goes live, the engineering team has a verified, operational dashboard to monitor its health, rather than discovering monitoring gaps during an incident.

Dark launch provides a controlled environment to inject failures and test the resilience of the new model serving system. Engineers can safely test the integration of circuit breakers, fallback models, and graceful degradation pathways. For instance, by artificially throttling the new model's dependencies (e.g., a feature store), teams can verify that the system correctly fails over to a cached value or a default, rather than crashing or timing out. This proactive validation of failure modes is a core practice in building reliable ML-powered services.

A deployment state where a new model processes live production traffic in parallel with the currently serving model, but its outputs are logged for analysis and not used to affect user-facing decisions. This is a core implementation pattern for a dark launch.

• Primary Use: Safely gather performance metrics (latency, throughput) and prediction quality data under real load.
• Key Distinction: Unlike a canary, users are unaware and unaffected by the new model's existence.
• Outcome: Provides a direct, apples-to-apples comparison against the incumbent model before any traffic switching.

A gradual, user-facing rollout where a new model version is initially deployed to a small, specific subset of production traffic or users. Performance and business metrics are closely monitored before expanding the rollout.

• Primary Use: Validate stability and correctness with real users before full deployment.
• Key Distinction: Unlike a dark launch, the canary model's predictions directly affect the selected users' experience.
• Progression: Typically follows a successful shadow mode/dark launch phase to further de-risk the release.

The infrastructure mechanism that directs a controlled percentage of inference requests to different model versions. It is the enabling technology for A/B tests, canary releases, and gradual rollouts.

• Implementation: Typically managed by a load balancer, service mesh (e.g., Istio, Linkerd), or an ML feature store/gateway.
• Control Levers: Splits can be based on random sampling, user attributes, geographic location, or device type.
• Purpose: Allows for simultaneous evaluation of multiple model variants in production under identical conditions.

A software configuration mechanism (often a toggle or switch) that dynamically controls the activation of code paths or features, including which model version is invoked for a given request.

• Primary Use: Decouple deployment from release. A new model can be deployed to production but kept disabled until a flag is flipped.
• Granularity: Flags can be set at global, user-segment, or even individual user levels.
• Critical Role: Serves as the kill switch for instant rollback without needing a code redeploy if a model fails.

A release strategy that maintains two identical, full-scale production environments (labeled Blue and Green). At any time, one environment serves all live traffic while the other hosts the new version.

• Primary Use: Achieve zero-downtime updates and instantaneous rollbacks by switching traffic at the router/load balancer level.
• Process: The new model is deployed to the idle (e.g., Green) environment, validated, and then traffic is switched from Blue to Green.
• Contrast with Dark Launch: This is an infrastructure-swapping technique, whereas dark launch is about hiding logic within a single environment.

A network-level technique where a copy (or mirror) of live production requests is sent to a secondary service or model endpoint. The mirrored traffic does not block the primary request/response cycle.

• Primary Use: A common method to implement shadow mode. The new model receives a duplicate of every request sent to the primary model.
• Key Benefit: Allows load testing and performance profiling of the new model with absolutely no risk to the user experience.
• Infrastructure: Often configured within a service mesh or API gateway.