Deployment

A Deployment's primary function is to declare the desired state of an application, such as the container image, number of replicas, and resource limits. The Kubernetes control plane's state reconciliation loop continuously works to match the actual cluster state to this declared specification. This is foundational to GitOps workflows, where the Deployment manifest in a Git repository serves as the single source of truth.

• Key Fields: spec.replicas, spec.template (Pod spec), spec.selector.
• Example: A Deployment manifest declares 10 replicas of an object detection model; Kubernetes schedules and maintains 10 Pods, restarting any that fail.

Deployments enable sophisticated, automated update strategies crucial for zero-downtime model version upgrades on edge fleets. The rolling update strategy incrementally replaces old Pods with new ones, controlled by parameters like maxUnavailable and maxSurge. If a new version fails health checks, an automatic rollback to the previous stable version can be triggered.

• Update Triggers: Changes to the Pod template (e.g., a new model image tag).
• Versioned History: Kubernetes maintains a revision history of the Deployment, enabling rollback to any previous revision.
• Edge Consideration: Updates must be staged and validated across heterogeneous device groups using canary deployments.

In Edge AI orchestration, a Deployment is rarely a standalone object. It integrates with other Kubernetes resources to manage the full model lifecycle:

• ConfigMaps & Secrets: Inject model configuration, feature flags, and secure API keys into the inference Pod.
• Horizontal Pod Autoscaler (HPA): Automatically scale the number of inference Pod replicas based on custom metrics like request latency or queue depth.
• Service: Provides a stable network endpoint and load balancing for the dynamic set of inference Pods, both within the cluster and for external access.
• Resource Limits: Critical for edge devices; Deployments define CPU/memory requests/limits to prevent resource starvation.

Edge environments consist of heterogeneous hardware (CPU, GPU, NPU). Deployments use node affinity/anti-affinity rules and tolerations to ensure inference Pods are scheduled on appropriate devices.

• Node Affinity: requiredDuringSchedulingIgnoredDuringExecution rules force Pods onto nodes with specific labels (e.g., accelerator: nvidia-jetson).
• Taints and Tolerations: A master node or a node reserved for system workloads may have a taint. Inference Pods must have a matching toleration to be scheduled there.
• Pod Topology Spread Constraints: Used to spread Pods across failure domains (e.g., different physical edge locations) for high availability.

For stateful, complex Edge AI applications (e.g., managing an on-device training pipeline), the basic Deployment may be insufficient. The Operator Pattern extends Kubernetes using Custom Resource Definitions (CRDs) and custom controllers. An "InferenceEngine" CRD, managed by a custom controller, can use Deployments as building blocks while adding domain-specific logic for model warming, hardware-specific compilation, and health checks beyond simple process liveliness.

• Controller Logic: Watches for custom objects and creates/manages standard Kubernetes resources (like Deployments) in response.
• Example: The Kubeflow Training Operator manages distributed training jobs, which are more complex than stateless inference.

Understanding when to use a Deployment versus other Kubernetes workload controllers is key to effective orchestration:

• Deployment: Manages stateless, replicated applications. Provides rolling updates. Ideal for scalable model inference servers.
• StatefulSet: For stateful applications requiring stable, unique network identifiers, persistent storage, and ordered deployment/scaling (e.g., a distributed vector database for edge agents).
• DaemonSet: Ensures a copy of a Pod runs on all (or some) Nodes. Ideal for node-level monitoring agents, log collectors, or storage drivers on an edge fleet.
• Job/CronJob: For finite, batch-execution tasks like one-time model batch inference or scheduled dataset cleanup.

A Pod is the smallest deployable unit in Kubernetes, representing a single instance of a running process. It encapsulates one or more application containers, shared storage volumes, and a unique network IP. The Deployment controller manages the lifecycle of Pods to match the declared desired state.

• Atomic Unit: Pods are scheduled, scaled, and managed as a single entity.
• Shared Context: Containers within a Pod share the same network namespace and can communicate via localhost.
• Ephemeral Nature: Pods are designed to be disposable; Deployments ensure a specified number of identical Pods are always running.

A ReplicaSet is a Kubernetes controller that ensures a specified number of identical Pod replicas are running at any given time. A Deployment creates and manages ReplicaSets to achieve the desired scale and provides declarative updates over them.

• Scalability: Maintains a stable set of Pod replicas, automatically creating or deleting Pods to match the count.
• Pod Selector: Uses label selectors to identify which Pods it manages.
• Underlying Mechanism: Deployments implement features like rolling updates by creating new ReplicaSets and scaling down old ones.

A Rolling Update is the default deployment strategy managed by a Deployment object. It incrementally updates Pod instances with new ones, ensuring zero downtime and allowing rollback if failures are detected.

• Incremental Replacement: New Pods are brought up while old Pods are terminated, controlled by maxSurge and maxUnavailable parameters.
• Health Checks: Relies on Pod readiness probes to ensure new instances are healthy before proceeding.
• Version Tracking: The Deployment maintains a revision history, enabling rollbacks to previous stable states.

The Horizontal Pod Autoscaler is a Kubernetes controller that automatically scales the number of Pods in a Deployment or ReplicaSet based on observed CPU utilization, memory consumption, or custom metrics.

• Dynamic Scaling: Adjusts replica counts to meet demand, defined by target utilization thresholds.
• Metric Sources: Can scale based on resource metrics from the Kubernetes metrics server or custom metrics from applications like Prometheus.
• Tight Integration: Works directly with the Deployment's underlying ReplicaSet to modify the spec.replicas field.

A Service is a Kubernetes abstraction that defines a stable network endpoint and a policy to access a logical set of Pods, typically those managed by a Deployment. It provides load balancing and service discovery.

• Stable IP/DNS: Provides a consistent virtual IP (ClusterIP) or DNS name that persists despite Pod churn.
• Load Balancing: Distributes network traffic across all healthy Pods matching the Service's label selector.
• Loose Coupling: Decouples the network identity of a workload from the dynamic set of Pods a Deployment manages.

State Reconciliation is the continuous control loop process executed by the Kubernetes Deployment controller. It constantly observes the actual state of the cluster and takes actions to drive it toward the declared desired state in the Deployment manifest.

• Control Loop: The core principle of all Kubernetes controllers.
• Self-Healing: If Pods crash or are deleted, the reconciliation loop detects the deviation and creates new ones to match the spec.replicas count.
• Declarative Model: Users specify the what (desired state), and the controller handles the how (reconciliation actions).