DaemonSet

The core function of a DaemonSet is to saturate nodes with a specific Pod. When a DaemonSet is created, the Kubernetes control plane automatically schedules a Pod on each node that matches the DaemonSet's node selector. If a new node joins the cluster, the DaemonSet controller immediately deploys a Pod to it. This is ideal for deploying per-node infrastructure services like:

• Log collectors (e.g., Fluentd, Logstash)
• Monitoring agents (e.g., Prometheus Node Exporter)
• Network plugins (e.g., Calico, Cilium)
• Local storage providers

Like all Kubernetes controllers, a DaemonSet operates on a declarative desired state. You define the Pod template and the nodes it should run on, and the DaemonSet controller continuously reconciles the actual state of the cluster with this specification. If a DaemonSet Pod crashes or is manually deleted, the controller immediately creates a replacement on the same node to maintain the desired state. This provides built-in resilience for critical node-level services, ensuring they are always present without manual intervention.

DaemonSets are not limited to all nodes. You can target a subset of nodes using node selectors or affinity/anti-affinity rules. This is essential in heterogeneous edge clusters where only specific nodes have the required hardware (e.g., a GPU for AI inference) or are designated for a particular role.

Example Selector:

yamlCopy
nodeSelector:
  accelerator: nvidia-tesla-t4

This ensures the DaemonSet Pod (e.g., a GPU monitoring agent) only runs on nodes with that label.

DaemonSets are the primary method for deploying Pods onto nodes with Kubernetes taints, such as master nodes. By adding corresponding tolerations to the DaemonSet's Pod spec, you can run system Pods on these restricted nodes.

For updates, DaemonSets support rolling updates. You can update the Pod template (e.g., to deploy a new version of a log collector), and the controller will update Pods on each node in a controlled manner, respecting configurable update strategies (RollingUpdate or OnDelete).

In edge AI architectures, DaemonSets are the standard pattern for deploying node-local AI inference servers or model monitoring sidecars. Instead of a centralized cloud endpoint, each edge node runs its own local inference service via a DaemonSet, providing:

• Ultra-low latency: Inference happens on the device generating the data.
• Offline operation: No dependency on cloud connectivity.
• Data locality: Sensitive data never leaves the node. A typical pattern involves a DaemonSet Pod that contains a lightweight model server (e.g., TensorFlow Serving, Triton Inference Server) and a mounted volume with the latest quantized model.

It's critical to distinguish DaemonSets from other Kubernetes workloads:

• Deployment: Manages a replicated set of Pods for user-facing applications (e.g., a web API). The scheduler places Pods on any available node based on resource needs.
• StatefulSet: Manages stateful applications (e.g., databases) where Pods have unique, stable identities and persistent storage.
• DaemonSet: Ensures one Pod per node for system-level services. The placement is tied to the node itself, not to scaling replicas. Use a DaemonSet when the workload's purpose is intrinsically linked to the node, not when you need to scale a service independently of the number of nodes.

DaemonSets manage CSI (Container Storage Interface) drivers and local storage provisioners on nodes with attached storage (e.g., NVMe, SD cards). This is critical for Edge AI workloads that require:

• High-performance I/O: Low-latency access to training data or video streams stored locally.
• Data locality: Processing sensor data where it lands, avoiding network transfer costs.
• Ephemeral caching: Providing fast temporary storage for model artifacts or intermediate computation results.

DaemonSets deploy Kubernetes Device Plugins that advertise specialized edge hardware to the Kubelet, allowing pods to request exclusive access. This is essential for:

• NPU/GPU acceleration: Making neural processing units available to AI inference pods.
• Sensor integration: Providing access to unique device peripherals (e.g., LiDAR, specialized cameras).
• Resource accounting: Enabling the scheduler to place model pods only on nodes with the required hardware accelerators present.

A Deployment is a Kubernetes controller that manages a set of identical, stateless pods, providing declarative updates, scaling, and rollback capabilities. It is the primary controller for running application services.

• Stateless by design: Ideal for scalable microservices and web servers.
• Rolling updates: Updates pods incrementally to ensure zero-downtime deployments.
• Scaling: Easily scales the number of pod replicas up or down.
• Key difference from DaemonSet: Deployments run a desired number of pods across the cluster, while DaemonSets run one pod per node.

A StatefulSet is a Kubernetes workload controller used to manage stateful applications, providing stable, unique network identifiers, ordered deployment and scaling, and persistent storage.

• Stable identity: Each pod gets a persistent hostname and stable storage, even after rescheduling.
• Ordered operations: Pods are created, scaled, and terminated in a strict order (ordinal index).
• Use cases: Essential for databases (e.g., MySQL, Cassandra), message queues, and any edge AI service requiring persistent state or ordered startup.
• Contrast with DaemonSet: DaemonSets are for node-level agents; StatefulSets are for clustered, stateful applications.

A Job is a Kubernetes controller that creates one or more pods to perform a specific task to completion, rather than running continuously.

• Run to completion: Ensures a pod successfully terminates after finishing its work.
• Parallel execution: Can create multiple pod instances to process work items in parallel.
• Use cases: Perfect for batch processing, data ETL pipelines, one-time model training jobs, or running a diagnostic script on demand across the cluster.
• Related controller: A CronJob schedules Jobs to run at specified times, useful for periodic model retraining or data aggregation tasks.

A Pod is the smallest and simplest deployable object in Kubernetes, representing a single instance of a running process. It is a group of one or more containers with shared storage and network.

• Atomic unit: The fundamental building block managed by all higher-level controllers (Deployments, DaemonSets, etc.).
• Shared context: Containers in a pod share an IP address, port space, and can communicate via localhost. They can also share volumes.
• Ephemeral: Pods are created and destroyed to match the state specified by a controller. A DaemonSet's core function is to ensure a specific pod specification is present on all relevant nodes.

The Operator Pattern is a method of packaging, deploying, and managing a Kubernetes application using custom controllers and custom resources. It encodes human operational knowledge into software.

• Automates complex tasks: Handles installation, configuration, backup, recovery, and upgrades for stateful applications.
• Extends Kubernetes API: Uses Custom Resource Definitions (CRDs) to define application-specific resources (e.g., InferenceService, EdgeModel).
• Relevance to Edge AI: Operators are increasingly used to manage the full lifecycle of AI/ML workloads on Kubernetes, automating tasks beyond the scope of standard DaemonSets or Deployments, such as model version rollout and hardware-specific optimizations.

These are mechanisms for controlling which nodes a pod can be scheduled onto, crucial for targeting specific edge hardware.

• Node Selector: A simple field in a pod spec that schedules pods only onto nodes with matching labels (e.g., accelerator: nvidia).
• Taints and Tolerations: A more advanced feature that repels pods from nodes unless the pod explicitly tolerates the taint.
  • Taint: Applied to a node (e.g., dedicated=edge:NoSchedule).
  • Toleration: Applied to a pod, allowing it to be scheduled on a tainted node.
• DaemonSet Usage: DaemonSets often use tolerations to run critical pods (like log collectors) on all nodes, including master nodes which are typically tainted.