NVIDIA MIG

Unlike software-based virtualization, MIG partitions the GPU at the hardware level using the NVIDIA Ampere architecture and later. Each instance receives dedicated Streaming Multiprocessors (SMs), L2 cache slices, and memory bandwidth. This prevents noisy-neighbor interference where one workload's memory thrashing degrades another's performance. The isolation extends to error containment—a fault in one instance cannot corrupt or crash another, making MIG suitable for multi-tenant Kubernetes clusters where different teams or customers share a single GPU.

MIG exposes configurable GPU Instance (GI) sizes that administrators can combine to match workload requirements. On an NVIDIA A100 80GB, available profiles range from a 1g.5gb slice (1/7th of compute, 5GB memory) to a full 7g.80gb instance. Each GI can be further subdivided into Compute Instances (CIs) that share the GI's memory but isolate compute streams. This granularity allows platform engineers to right-size GPU resources for inference serving, where a small language model might only need a 1g.10gb slice rather than monopolizing an entire GPU.

Each MIG instance receives dedicated memory bandwidth and guaranteed throughput independent of other instances. The GPU's on-chip crossbar switch and memory controllers enforce strict partitioning. This delivers predictable latency for real-time inference workloads where tail latency matters. In air-gapped sovereign deployments, deterministic QoS ensures that a mission-critical inference service running on one instance cannot be starved by a batch training job on another—a critical requirement for defense and healthcare environments where Service Level Agreements must be met regardless of co-located workloads.

MIG enables a single physical GPU to simultaneously serve heterogeneous workloads without context switching overhead. A typical configuration on an A100 might run:

• Two 2g.20gb instances for moderate-throughput LLM inference
• One 3g.40gb instance for a larger model or fine-tuning job Each instance can run a different framework—Triton Inference Server on one, vLLM on another, and a PyTorch training loop on a third—all on the same physical die. This consolidation dramatically improves GPU utilization in air-gapped data centers where hardware procurement is constrained and every accelerator must be maximized.

MIG differs fundamentally from GPU time-slicing and Multi-Process Service (MPS). Time-slicing rapidly context-switches between processes, introducing jitter and cache thrashing. MPS shares the GPU context but provides no memory isolation—a single process can exhaust all memory. MIG provides physical partitioning with:

• Dedicated memory: No overcommitment or OOM cross-contamination
• Dedicated cache: No L2 cache eviction from co-located workloads
• Fault isolation: Errors are contained to the instance For sovereign AI infrastructure requiring strict multi-tenancy guarantees, MIG is the only option that delivers hardware-enforced boundaries.

A node taint is a Kubernetes property that repels pods from being scheduled onto a node unless the pod has a matching toleration. For MIG-enabled GPUs, administrators apply custom taints like nvidia.com/mig-1g.5gb to nodes with specific MIG profiles. This ensures that only workloads requesting that exact GPU slice are scheduled there. This mechanism provides hard multi-tenancy at the scheduler level, preventing a generic GPU workload from consuming a carefully partitioned MIG device and enforcing strict resource isolation.

A Custom Resource Definition (CRD) extends the Kubernetes API to manage domain-specific objects. The NVIDIA GPU Operator uses CRDs like ClusterPolicy to declare the desired state of GPU software across the cluster. For MIG, the ClusterPolicy CRD specifies the MIG strategy (single, mixed, or none) and the exact MIG geometry to apply. This declarative approach aligns with GitOps principles, allowing the entire GPU configuration to be version-controlled in a Git repository and automatically reconciled in air-gapped clusters.

GPUDirect RDMA enables direct data transfer between a GPU's memory and a network interface card, bypassing system memory and the CPU. When combined with MIG, each GPU instance gets a dedicated slice of the high-speed interconnect. This is critical for multi-node inference in sovereign clouds where latency must be minimized. The technology allows a model sharded across multiple MIG instances on different nodes to communicate without host-side copies, maintaining near-linear scaling even in disconnected, high-security environments.