Glossary

DaemonSet

A Kubernetes controller that ensures a copy of a pod runs on all (or some) nodes in the cluster, typically used for cluster-level services like logging agents.

Get in touch Learn more

Developer demonstrating multi-agent tool use, agent tool selection interface on laptop, casual tech demo moment.

KUBERNETES DEPLOYMENT

What is DaemonSet?

A DaemonSet is a Kubernetes controller that ensures a copy of a specified Pod runs on all (or a subset of) nodes in a cluster.

A DaemonSet is a Kubernetes workload controller that guarantees an identical copy of a Pod is present and running on every node in the cluster that matches its node selector. Unlike a Deployment or ReplicaSet, which scale Pods based on a desired replica count, a DaemonSet's scaling is intrinsically tied to the number of eligible nodes. This makes it the primary mechanism for deploying cluster-level infrastructure services that must run on every node, such as logging collectors (e.g., Fluentd), monitoring agents (e.g., Prometheus Node Exporter), or network plugins (e.g., Calico). When a new node joins the cluster, the DaemonSet controller automatically schedules its Pod onto that node.

In the context of Agent Deployment Observability, a DaemonSet is the foundational pattern for deploying mandatory telemetry and monitoring sidecars. It ensures every compute node hosting autonomous agents runs the necessary observability daemon, providing a uniform data collection layer for agent telemetry pipelines and distributed trace collection. For partial node coverage, a DaemonSet uses node selectors or affinity/anti-affinity rules. It manages Pod lifecycle identically to other controllers, supporting rolling updates and rollbacks for version upgrades. This deterministic deployment model is critical for establishing a baseline of agent state monitoring and agentic anomaly detection across an entire fleet.

KUBERNETES CONTROLLER

Key Characteristics of a DaemonSet

A DaemonSet is a Kubernetes controller that ensures a copy of a Pod runs on all (or a subset of) nodes in a cluster. It is a core primitive for deploying cluster-level infrastructure services.

Node-Level Saturation

The primary function of a DaemonSet is to saturate nodes with a specific Pod. When a DaemonSet is created, it automatically schedules a Pod on every node that matches its nodeSelector or tolerates its tolerations. If a new node joins the cluster, the DaemonSet controller immediately deploys a Pod to it. Conversely, if a node is removed, its Pod is garbage collected. This ensures a uniform, node-level presence for services like:

Logging agents (e.g., Fluentd, Logstash)
Monitoring agents (e.g., Prometheus Node Exporter, Datadog agent)
Storage daemons (e.g., Glusterd, Ceph)
Network plugins (e.g., Calico, Weave Net)

Selective Deployment with Node Selectors

DaemonSets can target specific subsets of nodes using nodeSelectors and node affinity/anti-affinity rules. This allows for heterogeneous cluster management. For example:

Deploy a GPU monitoring agent only to nodes with GPU hardware (nodeSelector: { accelerator: nvidia-tesla-k80 }).
Run a specialized storage daemon only on nodes with high-performance SSDs.
Exclude master/control-plane nodes from running certain workloads by default (though they can be targeted with tolerations). This granular control is essential for managing infrastructure services across mixed-role or zoned clusters.

Tolerations for Tainted Nodes

DaemonSets are the primary mechanism for running Pods on tainted nodes. Nodes can be tainted (e.g., node-role.kubernetes.io/master:NoSchedule) to repel normal Pods. A DaemonPod must declare a matching toleration in its spec to be scheduled onto such a node. This is critical for deploying cluster services that must run on every node, including the control plane. For instance, a network plugin Pod must run on masters to enable pod-to-pod networking across the entire cluster. The DaemonSet's tolerations are a key differentiator from Deployments or StatefulSets.

Update Strategies: RollingUpdate & OnDelete

DaemonSets support two update strategies, controlled by spec.updateStrategy.type:

RollingUpdate (Default): Pods are automatically updated in a controlled, rolling fashion. You can configure maxUnavailable (e.g., 1) to control how many Pods can be unavailable during the update. The controller terminates old Pods on each node and creates new ones, respecting Pod lifecycle hooks.
OnDelete: The DaemonSet Pod template is updated, but existing Pods are not automatically replaced. New Pods are only created when you manually delete the old Pods on each node. This strategy provides manual control for critical stateful daemons where automated rollouts are risky. Updates are tied directly to node lifecycle, not just DaemonSet spec changes.

Static, Node-Bound Identity

Unlike Pods from a Deployment, DaemonSet Pods have a strong, static identity bound to their node. Each Pod's name is deterministic, derived from the DaemonSet name and the node name (e.g., fluentd-elasticsearch-master-node-1). The Pod is always scheduled to that specific node and uses a hostPort or hostNetwork more commonly than cluster-internal Services. This makes them ideal for:

Collecting node-specific metrics (using the node's filesystem or network namespace).
Providing a node-local cache or proxy.
Implementing network policy enforcement that requires host-level visibility. Their lifecycle is managed per-node, not by a desired replica count.

Use Case: Agent Deployment Observability

In the context of Agent Deployment Observability, DaemonSets are the standard pattern for deploying observability agents that must see the node's true state. This is foundational for monitoring agent deployments themselves (like canaries or A/B tests). For example:

A telemetry agent deployed via DaemonSet collects metrics from the node's kernel, container runtime (Docker/containerd), and Kubelet. It provides the ground-truth data for monitoring the health and resource usage of other agent Pods scheduled on that node.
This allows SREs to correlate agent Pod failures with node-level issues (e.g., disk pressure, network congestion).
The DaemonSet ensures the monitoring infrastructure is present before any application Pods (including agents) start, and persists after they terminate, providing continuous coverage.

EXPLORE

CONTROLLER COMPARISON

DaemonSet vs. Other Kubernetes Controllers

A comparison of core Kubernetes controllers based on their primary use case, scaling behavior, and suitability for different workloads.

Feature / Behavior	DaemonSet	Deployment (ReplicaSet)	StatefulSet	Job / CronJob
Primary Use Case	Cluster-level services (e.g., logging, monitoring, storage daemons)	Stateless web applications, microservices	Stateful applications requiring stable identity (e.g., databases)	Batch processing, one-off, or scheduled tasks
Pod Placement Guarantee	One pod per node (or subset via nodeSelector)	No guarantee; pods scheduled based on resource availability	Stable, ordered pod identity and placement	No guarantee; pods run to completion
Scaling Mechanism	Automatically scales with cluster nodes	Manually or via HPA based on metrics	Manually scaled; preserves pod identity order	Defined by completions/parallelism spec; non-scalable post-creation
Pod Identity & Naming	Non-unique; pods named <daemonset-name>-<random-hash>	Non-unique; pods named <deployment-name>-<replicaset-hash>-<random-hash>	Stable, ordinal naming (e.g., <statefulset-name>-0, -1)	Non-unique; pods named <job-name>-<random-hash>
Storage & Networking	Typically uses HostPath volumes or node network; no stable storage	Uses ephemeral storage or generic PersistentVolumeClaims	Uses stable, pod-specific PersistentVolumeClaims	Uses ephemeral storage or generic PersistentVolumeClaims
Update Strategy	RollingUpdate or OnDelete	RollingUpdate or Recreate	Partitioned RollingUpdate	Not applicable; new Job/CronJob instance created
Suitable for Agentic Observability
Ensures Long-Running Service

DAEMONSET

Frequently Asked Questions

A DaemonSet is a core Kubernetes controller used to deploy cluster-level services. This FAQ addresses its core function, use cases, and operational specifics for DevOps and SRE teams.

A DaemonSet is a Kubernetes controller that ensures a copy of a specified Pod runs on all (or a subset of) Nodes in the cluster. It automatically creates and manages Pods, placing one on each eligible node as nodes are added to the cluster and garbage-collecting Pods as nodes are removed. Its primary purpose is to deploy cluster-level daemons like logging agents, monitoring collectors, or network plugins that must run on every node to provide a fundamental platform service.

Unlike a Deployment or ReplicaSet, which manages a desired number of Pods that can run on any node, a DaemonSet's scheduling is intrinsically tied to the node lifecycle. It is defined using a standard Pod template and uses a nodeSelector or affinity/anti-affinity rules to target specific nodes, such as those with a particular label (e.g., disk=ssd).

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.

Talk to Us

DaemonSet

What is DaemonSet?