Agent Rolling Update

The update process replaces agent instances sequentially, not all at once. The orchestrator (e.g., Kubernetes) terminates an old pod, schedules a new one with the updated version, and waits for it to become healthy before proceeding to the next. This creates a phased transition where both old and new versions run concurrently during the update window.

• Key Benefit: Maintains a minimum number of available agents to serve requests.
• Contrasts with a recreate strategy, which terminates all old instances before starting new ones, causing a full service outage.

The primary objective is to maintain service-level agreements (SLAs) during deployment. By carefully managing the sequence and health of instances, the overall system remains available to end-users.

• Traffic Routing: A load balancer or service mesh (e.g., Istio) directs traffic only to healthy, ready instances.
• Readiness Probes: New instances must pass their readiness check before being added to the traffic pool. If a new instance fails, the update pauses or rolls back, preventing a cascade of failures.

The update's pace is governed by liveness and readiness probes. The orchestrator uses these checks to decide when to move to the next pod.

• Max Surge: Defines the maximum number of extra pods (beyond the desired replica count) that can be created during the update. A value of 1 means you can have one new pod and one old pod running simultaneously.
• Max Unavailable: Defines the maximum number of pods that can be unavailable during the update. A value of 0 enforces that at least the full desired number of pods are always ready, a strict requirement for critical services.

If the new agent version exhibits failures (e.g., crashes on startup, fails health checks), the orchestrator can automatically or manually initiate a rollback. This reverts the deployment to the previous stable version using the same rolling update mechanism in reverse.

• Automatic Rollback: Some systems trigger a rollback after a configurable number of new pods fail consecutively.
• Versioned History: The orchestrator maintains a revision history of the deployment, allowing operators to revert to any known-good configuration instantly.

The update strategy is defined declaratively in the agent's deployment manifest, not via imperative commands. This specification is version-controlled and applied by the orchestration system.

Example Kubernetes Deployment Spec:

yamlCopy
strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0

• Declarative State: The orchestrator's reconciliation loop continuously works to match the live cluster state to this declared desired state, managing the complex update process automatically.

Rolling updates differ from other deployment strategies in their granularity and traffic control.

• vs. Blue-Green: Blue-green maintains two complete, separate environments. Traffic is switched all at once from the old (blue) to the new (green). Rolling updates blend versions within a single environment.
• vs. Canary: A canary release directs a small, specific subset of traffic (e.g., 5% of users) to the new version for validation. A rolling update typically replaces instances across the entire user base, just gradually. Canary is often a precursor to a full rolling update.

A Kubernetes policy that constrains voluntary disruptions during operations like rolling updates or node maintenance. It ensures a minimum number of available agent pods or a maximum number of unavailable pods.

• Function: Orchestrators respect the PDB, evicting pods gradually to maintain service availability.
• Example: maxUnavailable: 1 ensures no more than one pod in a deployment is down during an update.
• Critical For: Stateful agents where quorum or persistent connections must be maintained.

Periodic diagnostic probes (liveness and readiness) used by the orchestrator to determine an agent's operational state. Essential for the safety of rolling updates.

• Liveness Probe: Determines if the agent is running. Failure triggers a restart.
• Readiness Probe: Determines if the agent is ready to accept traffic. A failing pod is removed from service load balancers.
• Update Logic: The orchestrator waits for the new pod's readiness probe to pass before terminating the old pod, ensuring continuous service.

The orchestration capability to automatically detect and recover from agent failures. This works in tandem with rolling updates to maintain system resilience.

• Mechanism: Combines health checks with restart policies (e.g., Always, OnFailure) and rescheduling to healthy nodes.
• During Updates: If a new pod fails its health checks repeatedly, the update may be automatically halted, and the previous stable version continues serving traffic.

The practice of defining the desired state of agents (image version, replica count, resources) in version-controlled manifest files. Rolling updates are triggered by changes to this declared state.

• Principle: The orchestrator's control loop continuously reconciles the actual cluster state with the declared state.
• Workflow: A developer commits a new agent image tag to Git. A GitOps operator (e.g., ArgoCD) applies the manifest, initiating a controlled rolling update in the cluster.