Data Plane

The low-level software responsible for executing and managing containerized applications. It pulls container images, creates the container's filesystem, manages cgroups for resource isolation, and runs the application process.

• Examples: containerd, CRI-O, Docker Engine.
• In Edge AI, the runtime must be lightweight and support hardware acceleration for model inference (e.g., accessing GPU or NPU resources).

A local agent that handles all inbound and outbound network traffic for workloads on the node. It implements core data plane functions like:

• Load Balancing: Distributing requests across local service endpoints.
• Service Discovery: Resolving service names to local Pod IPs.
• TLS Termination: Handling encrypted traffic.
• In a Service Mesh architecture, this is typically a sidecar proxy (e.g., Envoy) injected alongside each application container.

Extensions that enable containers to access persistent storage volumes. They abstract the underlying storage infrastructure (local SSD, network-attached storage, cloud block storage).

• Key Functions: Mounting/unmounting volumes, formatting filesystems, handling quotas.
• For Edge AI, plugins often manage local high-performance storage for time-series sensor data or model artifacts, and must handle intermittent connectivity to central storage systems.

Kubernetes agents that advertise specialized node hardware to the kubelet, making it available for scheduling. Critical for Edge AI to expose accelerators.

• Manages: GPUs, Neural Processing Units (NPUs), Vision Processing Units (VPUs), and FPGAs.
• The plugin performs health checks and handles the device lifecycle, allowing Pods to request resources like nvidia.com/gpu: 1 in their specifications.

The primary node agent that runs on each machine in the cluster. It is the central data plane component that:

• Registers the node with the API server.
• Pods: Receives PodSpecs and ensures the described containers are running and healthy.
• Resources: Manages container resources (CPU, memory) and reports node status.
• Probes: Executes liveness and readiness probes defined for containers.

A specialized data plane service for executing machine learning models. It is a critical Edge AI component that provides:

• Model Serving: Loading one or more models and exposing them via APIs (gRPC/REST).
• Hardware Optimization: Utilizing device plugins to run inference on GPUs/NPUs.
• Dynamic Batching: Combining multiple inference requests to improve throughput.
• Examples: NVIDIA Triton Inference Server, TensorFlow Serving, TorchServe.

The core software responsible for pulling container images, creating isolated execution environments (namespaces, cgroups), and managing the lifecycle of containers on a host. In Edge AI, the runtime must be lightweight and support hardware acceleration pass-through.

• Key Function: Executes the containerized application or AI model workload.
• Edge Consideration: Often a stripped-down version like containerd or cri-o is used instead of the full Docker daemon to reduce overhead.

A network-level data plane component that intercepts and manages all inbound/outbound traffic for a service. It enforces policies set by the control plane for load balancing, routing, security (mTLS), and observability.

• Sidecar Pattern: Typically deployed as a separate container alongside the application container in a Pod.
• Edge Function: Handles east-west traffic between microservices on the edge node and manages north-south communication with the cloud, often in bandwidth-constrained environments.

Implements the Container Storage Interface (CSI) to provide dynamic provisioning, attachment, and mounting of persistent storage volumes to containers. The data plane action is the actual mounting of the block device or filesystem.

• Edge Challenge: Must support local storage types (NVMe, SD cards) and network-attached storage optimized for high latency or intermittent connectivity.
• Example: A CSI driver for a local path provisioner that creates PersistentVolumes using the node's local disk for temporary model or sensor data.

Implements the Container Network Interface (CNI) specification to configure network interfaces, IP addresses, and routing rules for Pods when they are created or destroyed.

• Data Plane Actions: Creating virtual Ethernet pairs (veth), placing one end in the Pod's network namespace, applying bandwidth limits, and setting up iptables/ebpf rules for traffic shaping.
• Edge Examples: Calico for overlay networking, Cilium for eBPF-powered networking and security, or Flannel for a simple layer 3 network fabric.

A Kubernetes mechanism that advertises specialized node hardware (e.g., GPUs, NPUs, FPGAs) to the kubelet, allowing the scheduler to place Pods on nodes with those resources. The data plane is responsible for preparing and allocating the device to the container.

• Edge AI Critical: Enables AI inference workloads to request and exclusively access hardware accelerators like the NVIDIA Jetson's GPU or Intel's Neural Compute Stick.
• Operation: The plugin runs as a DaemonSet, discovers devices, and publishes them as allocatable resources. Upon Pod scheduling, it performs the device-specific operations (e.g., loading drivers, setting GPU modes) before container start.

The specialized software library that executes a trained machine learning model. It performs the core tensor operations, often heavily optimized for the underlying CPU, GPU, or NPU architecture.

• Data Plane Core: This is the primary AI workload execution component on an edge node.
• Optimizations: Includes graph optimizations, kernel fusion, and leveraging low-precision arithmetic (INT8) for maximum throughput and minimal latency.
• Integration: Runs inside the application container, consuming input from sensors or network, and producing predictions or classifications.

The control plane is the centralized decision-making layer in a distributed system. It defines the desired state—such as which models to deploy, where, and with what resources—and issues commands to the data plane. In Kubernetes, the control plane components (API server, scheduler, controller manager) make global scheduling and health decisions.

• Key Function: Policy and intent declaration.
• Analogy: The air traffic control tower issuing flight plans.
• Contrast with Data Plane: The control plane decides what should happen; the data plane executes how it happens on each node.

The orchestration plane is a higher-level abstraction that often encompasses both control and data plane functions for a specific domain, like AI workload management. It translates high-level application goals (e.g., "run object detection on all warehouse cameras") into the specific resource declarations and policies consumed by the underlying cluster's control plane.

• Scope: Manages the full lifecycle of complex, stateful applications.
• Example: An edge AI platform that handles model versioning, A/B testing, and drift detection across a fleet, then uses Kubernetes as its execution substrate.

A sidecar proxy is a secondary container deployed alongside the main application container in a Pod. It is a fundamental data plane component for implementing a service mesh, handling all network traffic to and from the application. For edge AI, sidecars can manage:

• Secure Model Serving: Intercepting inference requests and applying authentication.
• Local Telemetry: Collecting metrics on model latency and power consumption.
• Traffic Routing: Directing requests between different model versions for canary testing.

This pattern offloads cross-cutting concerns from the main AI workload.

A node is a worker machine in a Kubernetes cluster, which can be a physical server or a virtual machine. It is the physical host where the data plane operates. Each node runs the necessary services to execute Pods:

• Container Runtime: Software like containerd that runs containers.
• Kubelet: The primary node agent that communicates with the control plane and manages the Pods and containers on its host.
• Kube-proxy: Maintains network rules on the node for Pod networking.

In edge AI, a node is often a constrained device (like a gateway or robot) running the data plane components to execute AI workloads locally.

A Pod is the smallest deployable unit in Kubernetes, representing a single instance of a running process. It is the core execution unit of the data plane. A Pod encapsulates:

• One or more Containers: Such as a containerized ML model and a sidecar for logging.
• Shared Storage Volumes: For accessing model files or temporary data.
• Unique Network IP: All containers in a Pod share the same network namespace.

For edge AI, a Pod typically contains the inference engine serving a specific model version. The data plane's job is to keep these Pods running and healthy on their assigned nodes.

The Container Runtime Interface is a plugin interface that enables Kubernetes to use different container runtimes. It is a critical low-level data plane component on each node. The Kubelet communicates with the CRI to:

• Manage Container Lifecycle: Start, stop, and pause containers.
• Execute Commands: Run commands inside a running container.
• Pull Container Images: Fetch images from a registry.

Common CRI implementations include containerd and CRI-O. This abstraction allows the Kubernetes data plane to be portable across various underlying container technologies, which is essential for heterogeneous edge hardware.