Skip to content

GPU VM

NUMA-Aware GPU VMs: How NVIDIA's Reference Architecture Actually Fixes It

In Part 1 of the blog series, we walked through the basics: NUMA nodes, remote-access penalties, and why a GPU VM that doesn't know its own topology can quietly sabotage itself by pinning a hot thread to one vCPU while its data lands on a memory bank a socket away.

We ended with a question we didn't answer: what does the fix actually look like in practice?

NVIDIA has published a prescriptive recommendation called Performance Reference Architecture. It's worth walking through this in some detail, because it's the closest thing to an industry-standard answer for how do I make a GPU VM behave like bare metal?.

Why is your GPU VM slower Than Its Twin?

Picture two GPU VMs on the same bare-metal host, provisioned identically: same GPU model, same vCPU count, same memory allocation, same container image. One of them runs your inference workload 20% slower than the other — every time, consistently, with no other tenant contention and nothing wrong in the logs.

The difference isn't the GPU. It's the distance between the CPU cores feeding that GPU and the memory those cores are reading from. That distance has a name Non-Uniform Memory Access (NUMA). On modern multi-socket, multi-GPU hosts, it's one of the most common, least visible sources of inconsistent performance in GPU cloud environments.

At Rafay, we spend a lot of time thinking about how to make GPU infrastructure predictable because "predictable" is the whole point of a platform. So, we thought it is worth taking a step back and explaining what NUMA actually is, why it matters esp. once GPUs enter the picture, and what a platform needs to do about it.

This is the first part of a blog series on NUMA, how it impacts VMs esp. GPU VMs and approaches that are required to deliver great performance to GPU VMs on moden platforms.

NVIDIA Performance Reference Architecture: An Introduction

Artificial intelligence (AI) and high-performance computing (HPC) workloads are evolving at unprecedented speed. Enterprises today require infrastructure that can scale elastically, provide consistent performance, and ensure secure multi-tenant operation. NVIDIA’s Performance Reference Architecture (PRA), built on HGX platforms with Shared NVSwitch GPU Passthrough Virtualization, delivers precisely this capability.

This is the introductory blog in a multi part series. In this blog, we explain why PRA is critical for modern enterprises and service providers, highlight the benefits of adoption, and outline the key steps required to successfully deploy and support the PRA design/architecture.