Node Affinity

Node affinity ensures AI inference Pods are scheduled exclusively onto nodes equipped with specialized hardware accelerators like NPUs (Neural Processing Units), GPUs, or Google Edge TPUs. This is a hard requirement (requiredDuringSchedulingIgnoredDuringExecution) to guarantee the model can execute with the necessary low latency and power efficiency.

• Example Rule: A Pod for a computer vision model requires a node with the label accelerator: nvidia-t4.
• Benefit: Maximizes throughput and ensures deterministic performance by preventing scheduling onto CPU-only nodes, which would be unable to meet the inference Service Level Agreement.

In distributed edge deployments, node affinity pins workloads to nodes in specific geographic regions, data centers, or network zones to comply with data sovereignty laws (e.g., GDPR) or to minimize latency for local users.

• Example Rule: A Pod processing sensitive EU citizen data has an affinity for nodes labeled region: eu-west-1.
• Use Case: A retail analytics model must run in the same store location (zone: store-1234) where video data is captured to avoid cross-border data transfer and ensure real-time processing.
• Implementation: Often combined with topology spread constraints to also ensure high availability within the designated zone.

Node affinity is often used in conjunction with taints and tolerations for advanced scheduling. Taints repel all Pods except those with a matching toleration, while affinity attracts specific Pods to labeled nodes.

• Common Pattern:
  1. Taint dedicated GPU nodes with gpu=true:NoSchedule.
  2. Add a matching toleration to AI inference Pods.
  3. Use node affinity to further specify which GPU node type (e.g., gpu-vendor: nvidia) the Pod prefers.
• Result: This two-pronged approach ensures GPU nodes are reserved for GPU workloads (via taints) and that workloads get the most appropriate GPU type (via affinity), enabling precise heterogeneous fleet orchestration.

Taints and tolerations are the inverse mechanism to node affinity. A taint is applied to a node to repel all Pods that do not explicitly tolerate it. This is used to:

• Reserve nodes for specific workloads (e.g., GPU nodes).
• Evict Pods from nodes that are experiencing problems.
• Implement node anti-affinity by repelling Pods that shouldn't run together. While node affinity attracts Pods to nodes, taints repel them, providing a push-based model for scheduling constraints.

Pod topology spread constraints control how Pods are spread across your cluster among failure-domains like regions, zones, nodes, and other user-defined topology domains. This is critical for:

• High availability by ensuring replicas are distributed across different hardware.
• Balanced resource utilization to avoid overloading a single node or zone.
• Complementing affinity rules; you can use node affinity to select a pool of nodes, then spread constraints to distribute Pods evenly within that pool. They express soft (whenUnsatisfiable: ScheduleAnyway) or hard (whenUnsatisfiable: DoNotSchedule) spreading goals.

Resource requests and limits are the primary mechanism for informing the Kubernetes scheduler about a Pod's compute needs.

• Requests specify the minimum amount of CPU/memory a Pod needs. The scheduler uses this to select a node with sufficient allocatable resources.
• Limits specify the maximum amount a container can use, enforced by the kernel. Node affinity rules are evaluated after the scheduler filters nodes based on resource availability. A Pod with a requiredDuringScheduling affinity for a node that lacks resources will remain pending.

A custom scheduler is a program that implements the Kubernetes scheduler interface, allowing you to replace or run alongside the default scheduler (kube-scheduler). This is used when:

• Default scheduling policies (affinity, resources) are insufficient.
• You need domain-specific scheduling logic (e.g., for AI/ML workloads, batch processing).
• Implementing complex multi-dimensional bin packing. You might use node affinity rules defined in Pod specs, but the custom scheduler interprets them with its own scoring and filtering algorithms.

Node selector is a simpler, predecessor field to node affinity. It specifies a map of key-value pairs that must match labels on a node for the Pod to be schedulable.

• It expresses only hard requirements (like requiredDuringSchedulingIgnoredDuringExecution).
• It does not support soft preferences, anti-affinity, or complex logical operators. For most new configurations, node affinity is preferred due to its expressive power. Node selector is essentially a basic subset of node affinity syntax.

Inter-Pod affinity and anti-affinity constrain which nodes your Pod can be scheduled on based on the labels of Pods already running on that node, not the node labels themselves.

• Pod Affinity: Schedule this Pod onto a node if it is already running a Pod with specific labels (e.g., co-locate a web server with its cache).
• Pod Anti-Affinity: Schedule this Pod onto a node only if it is not running a Pod with specific labels (e.g., spread replicas across nodes for HA). These rules can be requiredDuringScheduling or preferredDuringScheduling and are computationally more expensive for the scheduler to evaluate than node affinity.