Zero-Downtime Deployment

The foundation of zero-downtime is intelligent traffic routing. A load balancer distributes user requests across multiple, identical application instances. During a deployment, the orchestrator (like Kubernetes) directs the load balancer to:

• Drain connections from old instances before termination.
• Register new instances only after they pass health checks and readiness probes.
• Maintain at least N-1 instance availability at all times. This ensures incoming traffic is always served by a healthy instance, making the replacement of individual instances invisible to users.

This pattern maintains two identical production environments: Blue (current version) and Green (new version).

• The entire new version is deployed to the idle Green environment.
• After rigorous testing, a router or load balancer instantly switches all traffic from Blue to Green.
• The old Blue environment is kept on standby for immediate rollback if issues are detected. Key Advantage: The switch is atomic and instantaneous, eliminating the "in-between" state of a rolling update. It requires double the infrastructure capacity during the cutover.

The most common strategy in containerized environments. New application instances (pods) are gradually rolled out while old ones are terminated.

• The orchestrator starts a new pod with the updated version.
• Once the new pod is healthy and ready, the orchestrator terminates an old pod.
• This process repeats until all pods are replaced. Critical Controls: The strategy is governed by maxSurge (how many extra pods can be created) and maxUnavailable (how many pods can be down during the update). Setting maxUnavailable: 0 is a strict zero-downtime configuration.

Automated validation is essential to prevent faulty versions from receiving traffic.

• Liveness Probes determine if a container is running. Failure triggers a restart.
• Readiness Probes determine if a container is ready to serve requests. A pod is only added to the load balancer's pool after this probe succeeds.
• Startup Probes handle slow-starting containers. For LLM deployments, a readiness probe might check that the model is loaded into GPU memory and can perform a simple inference. Without these checks, users could be routed to a broken instance, causing errors.

Stateless application updates are simpler. For stateful services (like databases) or LLMs with fine-tuned adapters, strategies include:

• Backward-Compatible Schema Changes: Database migrations must be applied in phases that are compatible with both old and new application versions.
• Dual-Writing: The new version writes to both the old and new data structures during transition.
• Feature Toggles: Runtime flags can activate new data access paths only after migrations are complete.
• Model Weights: Swapping LLM model files or adapters often requires a brief, scheduled read-only window unless served from a redundant endpoint.

Zero-downtime requires confidence to proceed. This is built through real-time observability and safety mechanisms.

• Canary Analysis & Traffic Splitting: A small percentage of traffic is routed to the new version while monitoring key Service Level Indicators (SLIs) like latency, error rate, and business metrics.
• Automated Rollback Triggers: If SLIs violate a Service Level Objective (SLO), the system automatically triggers a rollback to the previous known-good version.
• Comprehensive Logging & Tracing: Distributed tracing helps identify if new errors are correlated with the deployment. This turns deployment from a manual event into a controlled, observable process.

A deployment strategy that maintains two identical production environments (labeled blue and green). Only one environment serves live traffic at a time. The new version is deployed to the idle environment, tested, and then traffic is switched instantaneously via a router or load balancer. This provides instant rollback by switching traffic back to the old environment if issues arise. It is a foundational pattern for achieving zero-downtime releases.

A risk-mitigation strategy where a new application version is released to a small, controlled subset of users or infrastructure (the 'canary') before a full rollout. Key aspects include:

• Traffic Splitting: Routing a percentage of requests (e.g., 5%) to the new version.
• Real-time Monitoring: Observing key metrics like error rates, latency, and business KPIs.
• Progressive Rollout: Gradually increasing traffic to 100% if the canary performs well, or rolling back if anomalies are detected. This allows for validation in production with minimal user impact.

A default deployment strategy in orchestrators like Kubernetes where application instances are gradually replaced without downtime. The process:

1. A new pod (instance) with the updated version is started.
2. Once healthy and passing its readiness probe, it is added to the load balancer pool.
3. An old pod is terminated.
4. This cycle repeats until all pods are updated. The key advantage is resource efficiency, as it does not require a full duplicate environment. The risk is that both versions may run simultaneously, requiring backward compatibility.

A software development technique that uses conditional runtime toggles to enable or disable functionality. This decouples deployment from release, enabling:

• Zero-downtime feature activation: Code is deployed but dormant until the flag is flipped.
• Controlled rollouts: Features can be enabled for specific user segments (e.g., internal teams, beta users).
• Instant kill switches: Problematic features can be disabled without a redeploy. Flags are managed via external configuration services and are essential for progressive delivery.

The practice of routing user requests to different service versions based on defined rules. It is the underlying mechanism for canary deployments and A/B testing. Implementations include:

• Load Balancer Rules: Configuring weights (e.g., 90% to v1, 10% to v2).
• Service Mesh: Using a mesh like Istio or Linkerd to apply fine-grained routing policies based on headers, cookies, or percentages.
• API Gateway: Directing traffic at the entry point. This allows for parallel running of versions and data-driven decision-making for rollouts.

Kubernetes health checks that are critical for automated, zero-downtime updates.

• Liveness Probe: Determines if a container is running. If it fails, the kubelet restarts the container.
• Readiness Probe: Determines if a container is ready to serve traffic. If it fails, the pod is removed from service endpoints until it passes. During a rolling update, the new pod must pass its readiness probe before receiving traffic, and the old pod continues serving until terminated, ensuring continuous availability.