Pod

The Deployment controller is the primary workload manager for stateless applications. It provides declarative updates for Pods and their ReplicaSets, enabling:

• Rolling updates and rollbacks for model versions with zero downtime.
• Desired replica count maintenance, automatically replacing unhealthy Pods.
• Ideal for stateless inference servers where any Pod instance is interchangeable.

A DaemonSet ensures a copy of a Pod runs on all (or a subset of) nodes in a cluster. This is critical for edge AI to deploy node-level services:

• Log collectors and monitoring agents (e.g., for GPU telemetry).
• Local storage provisioners or network plugins required for model execution.
• When a new node joins the cluster, the DaemonSet automatically schedules the Pod onto it.

The StatefulSet controller manages stateful applications, providing guarantees about:

• Stable, unique network identifiers (hostname) for each Pod.
• Persistent storage that follows the Pod, even after rescheduling.
• Ordered, graceful deployment and scaling. Essential for edge AI applications requiring local, persistent model caches or embedded vector databases that must maintain identity and data.

These controllers manage finite, batch-oriented tasks.

• A Job creates one or more Pods and ensures a specified number successfully terminate. Used for one-off model batch inference, data preprocessing, or on-device validation tasks.
• A CronJob runs Jobs on a time-based schedule (e.g., 0 */6 * * *). Used for scheduled model retraining on edge data, periodic health checks, or reporting telemetry.

The ReplicaSet is a lower-level controller that maintains a stable set of identical Pod replicas running at any given time. It is the mechanism used by Deployments to ensure scalability and availability.

• Key Functions:
  • Scaling: Increases or decreases the number of Pods to match a replicas spec.
  • Self-Healing: Replaces Pods that fail, are deleted, or are evicted from a node.
• While rarely used directly, it is the foundational layer for Pod replication.

A Kubernetes controller that ensures a copy of a specified Pod runs on all (or some) nodes in a cluster. Essential for edge deployments of system-level services.

• Use Case: Deploying a monitoring agent, log collector, or local inference server on every edge device in a fleet.
• Key Property: Tied to the node lifecycle; when a node is added to the cluster, the DaemonSet automatically schedules the Pod onto it.

A Kubernetes workload controller for managing stateful applications. It provides guarantees about ordering, unique network identity, and persistent storage for Pods.

• Edge AI Relevance: Critical for deploying databases (e.g., for local vector stores), message brokers, or any edge AI service that requires stable network names (pod-0, pod-1) and durable data that survives Pod rescheduling.
• Contrast with Deployment: Unlike stateless Deployments, Pods in a StatefulSet are not interchangeable.

An abstract way to expose an application running on a set of Pods as a network service. It provides a stable IP address and DNS name for Pods, which are inherently ephemeral.

• Core Function: Load balances traffic across all healthy Pods matching a label selector.
• Edge Consideration: For internal communication between microservices (e.g., a frontend Pod calling a model inference Pod) within the edge cluster, regardless of individual Pod lifecycles.

Kubernetes objects used to manage configuration data and sensitive information separately from Pod definitions.

• ConfigMap: Stores non-confidential data (e.g., model parameters, feature flags, environment variables) in key-value pairs.
• Secret: Holds sensitive data like API keys, passwords, or TLS certificates, stored encoded or encrypted.
• Pod Integration: Both can be mounted into a Pod as files or exposed as environment variables, enabling immutable infrastructure where the Pod image remains constant but configuration is injected.

A Kubernetes controller that automatically scales the number of Pod replicas in a Deployment, StatefulSet, or other resource based on observed CPU/memory utilization or custom metrics.

• Edge AI Application: In edge clusters with varying inference load, HPA can scale inference server Pods up during peak demand and down to conserve resources during idle periods.
• Custom Metrics: Can scale based on application-specific metrics like queries-per-second or inference latency, gathered from a metrics server like Prometheus.

Specialized containers that run before the main app containers in a Pod. They must run to completion successfully before the main containers start.

• Critical Edge AI Tasks:
  • Model Download: Pulling the latest model weights from a registry to a shared volume before the inference engine starts.
  • Dependency Setup: Validating hardware accelerators (e.g., NPU drivers) or performing security checks.
  • Data Preprocessing: Preparing local datasets or configuration.
• Separation of Concerns: Keeps initialization logic out of the main application image.