Rolling Update

Rolling updates rely on readiness and liveness probes to determine the health of new instances. The orchestration engine (e.g., Kubernetes) will only proceed to update the next instance after the newly launched one passes its readiness check. This prevents defective versions from cascading through the system. If a new agent instance fails its health check, the rollout can be automatically paused, triggering an alert for investigation—a key feature of self-healing systems.

A robust rolling update mechanism includes automated rollback capabilities. If the updated version exhibits critical failures—such as a high crash rate or failed health checks beyond a defined threshold—the system can automatically revert to the previous stable version. This is often tied to metrics from orchestration observability tools. This fail-safe is essential for maintaining service level agreements (SLAs) and is a complementary practice to chaos engineering, which tests these rollback procedures.

During the update, the system's load balancer or service mesh (like Istio or Linkerd) plays a crucial role. It must dynamically adjust traffic routing away from instances being terminated and toward healthy ones, both old and new. This requires seamless integration with the orchestration layer's service discovery. In a multi-agent context, this ensures that agent-to-agent communication is not disrupted and that client requests are not sent to terminating agents.

During the rollout, multiple versions of the application or agent logic run simultaneously. This temporary state requires careful design to ensure backward and forward compatibility, especially for state synchronization and shared data. Agents must handle communication with peers running different API versions. For stateful agents, this may involve careful data migration strategies or the use of conflict-free replicated data types (CRDTs) to manage state during the transition.

A periodic probe or request (e.g., HTTP /health endpoint, heartbeat signal) sent to a service or agent to verify its operational status and readiness. In a rolling update, the orchestrator uses health checks to determine if a newly updated instance is healthy before proceeding to update the next one.

• Liveness Probe: Determines if the agent is running. Failure triggers a restart.
• Readiness Probe: Determines if the agent is ready to accept work. An agent failing its readiness check is removed from the load balancer.
• Critical Role: Health checks are the primary feedback mechanism that gates the progression of a rolling update, preventing faulty versions from cascading.

The automatic process of switching to a redundant or standby system, component, or agent when the currently active one is detected as failed. While rolling updates proactively manage change, failover reactively handles runtime failures to maintain availability.

• Active-Passive Failover: A standby replica takes over when the primary fails. Common in database clusters.
• Active-Active Failover: Traffic is redistributed among remaining healthy nodes. Common in stateless agent fleets.
• Relationship to Rolling Updates: During a rolling update, the orchestrator may trigger a failover if a newly deployed instance fails its health check, rolling back to the previous version for that specific node.

A design philosophy where a system is built to maintain partial, core functionality when some of its components fail or are under maintenance. A rolling update is an operational tactic that enables graceful degradation by ensuring some instances are always available.

• Contrast with Fault Tolerance: Fault tolerance aims for full functionality despite failures. Graceful degradation accepts a reduced but acceptable service level.
• Example in Multi-Agent Systems: During a rolling update of a specialized 'planner' agent, the system might degrade to using a simpler, fallback planning algorithm on updated nodes, while non-updated nodes continue full service.