▸ saaro:~/blog ▊ Notizen & Artikel DE
← back

📷 "Breaking a Nvidia GeForce 4 Ti : Sawing the thing! 2/3" by qubodup is marked with CC0 1.0. To view the terms, visit https://creativecommons.org/publicdomain/zero/1.0/.

GPU Scheduling on Kubernetes: Comparing Kueue, Volcano, and KAI Scheduler in 2026

09 September 2026 · 4 min · Martin Jochum #Kubernetes#GPU-Scheduling#Kueue#Volcano#KAI Scheduler#KI#DevOps#GPU

GPUs are the most expensive resource in modern Kubernetes clusters – and at the same time the worst managed. An H100 accelerator costs more per hour than an entire mid-range server, yet Kubernetes’ standard scheduler treats GPUs like CPUs: pod by pod, without any concept of job atomicity, quotas, or actual utilization. The result is clusters where expensive hardware is reserved but not used, while other teams wait for freed-up capacity.

Since KubeCon Europe 2026 in Amsterdam, it’s clear: GPU scheduling is the dominant topic in the cloud-native ecosystem. The question is no longer whether to run AI workloads on Kubernetes, but how to distribute scarce GPU resources fairly and efficiently. Three open-source projects have emerged as the relevant solutions: Kueue, Volcano, and the NVIDIA KAI Scheduler.

Kueue: The Slim Admin Layer

Kueue is a SIG-Scheduling project from Kubernetes itself and is at version 0.19 as of September 2026. Its approach is deliberately non-invasive: Kueue does not replace the scheduler, but places an admin layer in front of it. A job is created in a suspended state and is only released by Kueue when the defined quota in the ClusterQueue is available. Pod placement is still handled by the kube-scheduler.

The model is simple: platform teams define ClusterQueues with nominal quotas and optional BorrowingLimits. Multiple queues can be grouped into a Cohort to share unused capacity among themselves. Teams get visibility via LocalQueues in their namespace without needing cluster rights.

Kueue’s strength is its low operational complexity. For managed Kubernetes offerings like EKS, AKS, or GKE, it is the first choice because no changes to the scheduler are required. The current version natively supports Jobs, JobSets, Kubeflow Training, and Ray workloads. With MultiKueue, jobs can even be dispatched across cluster boundaries.

Volcano: The Batch Scheduler from the HPC World

Volcano is a CNCF incubating project (v1.15 since May 2026) and takes a fundamentally different approach. Instead of an admin layer, it completely replaces the scheduler for pods that set schedulerName: volcano. This enables Gang Scheduling at the placement level – the critical difference from Kueue.

In distributed training with 64 parallel workers, a single pending pod brings the entire job to a halt. Volcano ensures that all pods of a job are started simultaneously and can also preempt if needed. In addition, it offers GPU sharing mechanisms such as software-based vGPU, dynamic MIG partitioning, and integration with HAMi for hard-isolated fractions of a GPU.

For dedicated training clusters with large, communication-intensive jobs, Volcano is the most mature solution. Network topology awareness reduces cross-switch traffic during parallel training – an advantage that Kueue does not offer.

KAI Scheduler: GPU-Centric from NVIDIA’s Forge

The KAI Scheduler (v0.17, September 2026) comes from the same codebase as NVIDIA’s commercial Run:ai platform and was released as an open-source project. Since December 2025, it has been a CNCF sandbox project.

KAI runs as an optional secondary scheduler alongside kube-scheduler and only takes over pods that explicitly set schedulerName: kai-scheduler. Its specific strengths lie in GPU allocation: Fractional GPUs allow multiple inference workloads on one card, hierarchical queues with fair sharing and reclamation mechanisms inherit years of production experience from Run:ai, and NVLink domain awareness places multi-GPU jobs within the same fabric island.

Especially for mixed clusters with training and inference workloads, KAI is a strong argument. The finely graduated fair-share mechanisms with time-based reweighting are the most mature multi-tenancy model for GPU clusters among the three projects.

Decision Matrix for Practice

The choice depends on the cluster type:

Scenario Recommendation
Managed Cloud (EKS, AKS, GKE), mixed workloads Kueue
Dedicated training cluster with large, distributed jobs Volcano
GPU-dense cluster, high utilization as goal KAI Scheduler
Large multi-team platform Kueue + Volcano (Kueue controls admission, Volcano places)

The combination of Kueue and Volcano is not a contradiction: Kueue decides on admission to the queue, Volcano handles the actual pod placement – a documented enterprise pattern that combines the strengths of both projects.

Conclusion

GPU scheduling on Kubernetes is no longer a niche topic in 2026, but an operational necessity. Kueue, Volcano, and KAI Scheduler address different levels of the problem: Kueue handles quota management and job admission, Volcano handles batch-optimized placement, and KAI handles GPU-centric fairness. Anyone running an AI-powered Kubernetes cluster today should choose at least one of the three projects – the time of “just set nvidia.com/gpu and hope” is over. The hardware is too expensive, and the teams are too many.

Sources

🌐 Machine-translated from the German original, editorially reviewed. 🤖 Written with AI assistance.