Skip to content

NVIDIA-Certified Hypervisors: Validating Single-Node Performance on NVIDIA GB200

Part 2 of our three-part series on NVIDIA-Certified Hypervisors

In Part 1, we introduced the NVIDIA-Certified Hypervisors program and the certification tracks NVIDIA provides for Arm and x86 platforms. In this second post, we go deeper into the Arm platform certification, currently scoped to NVIDIA GB200 NVL systems. The certification validates a single 4-GPU, 2-CPU passthrough virtual machine within one GB200 NVL compute tray.

A GPU VM can boot successfully. All four GPUs can appear correctly in the guest operating system. The NVIDIA software stack can install successfully. Yet the system can still suffer from incorrect NUMA placement, degraded CPU-to-GPU bandwidth, an underperforming GPU, or other configuration issues that significantly affect application performance.

For a GPU cloud provider, the GPU is visible is not the same as "the GPU infrastructure is ready for a tenant.

In this post, we'll look at the types of tests used to validate a single NVIDIA GB200 system and what they tell us about the performance and readiness of a virtualized GPU environment.

Single Node Hypervisor Certification

The Test Environment: Four Virtualized NVIDIA GB200 GPUs

The single-node environment consisted of a VM with:

  • 4 × NVIDIA GB200 GPUs, each with approximately 189 GiB of HBM
  • 2 × NVIDIA Grace CPU sockets / NUMA domains
  • Two GPUs physically local to each CPU socket
  • NVLink/NVSwitch for high-speed intra-node GPU communication
  • GPU passthrough into the VM

The environment correctly exposed all four GPUs to the workload. But simply detecting the GPUs is only the beginning of certification. A production-ready GPU system needs to be tested across multiple dimensions.

Multiple Performance Dimensions

No single benchmark can establish whether a GPU VM is correctly configured and performing as expected.

The certification exercise validates the system across GPU compute, memory bandwidth, CPU-to-GPU locality, GPU-to-GPU communication, CPU performance, and finally a real multi-GPU LLM inference workload. Together, these tests provide a much more complete picture of the system than GPU discovery alone.

Test What It Validates
NCCL All-Reduce GPU-to-GPU communication and NVLink/NVSwitch
NVIDIA nvbandwidth GPU memory, P2P and CPU↔GPU bandwidth
cuBLAS SGEMM Per-GPU compute consistency
HPCG Memory- and latency-sensitive HPC performance
HPL (CPU) NVIDIA Grace CPU performance
HPL (GPU) Multi-GPU FP64 performance
HPL-MxP Tensor Core / mixed-precision performance
GPU STREAM Sustained GPU memory bandwidth
TensorRT-LLM Real-world multi-GPU LLM inference

Multi-GPU AI workloads depend heavily on fast GPU-to-GPU communication, making NCCL All-Reduce one of the most important tests.

All four GPUs were placed into a single NCCL communicator, with the benchmark exercising message sizes from 8 bytes through 16 GB across the NVLink/NVSwitch fabric. The system achieved approximately ~470 GB/s average NCCL bus bandwidth with zero incorrect values.

The two signals are equally important. Bandwidth demonstrates that the GPU communication fabric is performing as expected, while zero incorrect values validates the correctness of that communication.

For a GPU cloud provider, four GPUs appearing in nvidia-smi isn't enough. The GPUs must also be able to communicate correctly and at the expected performance level.

2. NUMA Topology Matters

NUMA topology is an important consideration for GPU VM performance, particularly on systems such as GB200 that contain multiple Grace CPU sockets with GPUs physically associated with each socket. For optimal performance, the VM should accurately reflect the underlying CPU, GPU, PCIe, and NUMA topology so that each GPU can access host memory through the appropriate CPU socket.

This is important because conventional infrastructure health checks may still report a completely healthy environment. However, these checks alone do not confirm that the system is delivering its expected performance.

In certification testing, properly aligned NUMA topology delivered host-to-device bandwidth of up to: 211 GB/s per GPU with aggregate host-to-device bandwidth reaching: 845 GB/s

The key takeaway for operators is that GPU health alone is not sufficient to validate a GPU VM. CPU, GPU, PCIe, and NUMA topology should also be validated to ensure workloads receive the performance the underlying hardware is capable of delivering.

This is one of the important areas that GPU infrastructure certification can validate before the environment is handed over to a tenant or production workload.

3. Validating the Complete GPU Fabric

nvbandwidth was used to examine the complete GPU and host-memory data paths.

Metric Result
Device-to-device read 9.2 TB/s
Device-to-device write ~9.3 TB/s
Bidirectional D2D 18.3 TB/s
D2D per GPU pair ~768 GB/s
Aggregate host → device 845 GB/s
Aggregate device → host 774 GB/s

GPU-to-GPU NVLink performance remained consistent at approximately 767–768 GB/s per GPU pair, demonstrating strong and stable inter-GPU connectivity.

Info

There is value in using multiple complementary benchmarks during certification. Different tests validate different parts of the system, helping operators confirm performance across GPU-to-GPU, CPU-to-GPU, memory, and interconnect paths rather than relying on a single metric. Together, these results provide a more complete view of system performance and help validate that the infrastructure is operating as expected.

4. Checking for an Underperforming GPU

A node can also appear healthy while one GPU quietly performs below the others. A per-GPU cuBLAS SGEMM test provides a straightforward consistency check:

GPU FP32 Performance
GPU 0 72.689 TFLOP/s
GPU 1 72.358 TFLOP/s
GPU 2 72.480 TFLOP/s
GPU 3 72.554 TFLOP/s

In the certification, all four GPUs performed within approximately 0.5% of each other. This consistency is particularly valuable because it provides a mechanism for detecting a throttling or otherwise underperforming GPU before the infrastructure reaches a customer.

5. Validating HPC and Tensor Core Performance

The certification also exercised both traditional HPC and AI-oriented compute patterns.

  • HPCG produced 1,207 GFLOP/s with a VALID result, exercising workloads sensitive to memory bandwidth and latency.
  • Grace CPU HPL reached 2.26 TFLOP/s FP64, validating the CPU side of the GB200 environment.
  • Four-GPU HPL produced 147.6 TFLOP/s aggregate FP64 performance, with the residual check passing.

For AI-oriented mixed-precision compute, HPL-MxP achieved 1.69 PFLOP/s reportable performance and 3.39 PFLOP/s during the LU factorization phase.

GPU STREAM provided another dimension of validation, measuring sustained GPU memory bandwidth. The system achieved approximately 7.5 TB/s on STREAM Triad, with almost identical results across separate runs.

6. From Infrastructure Benchmarks to a Real AI Workload

Infrastructure benchmarks are useful, but ultimately customers are deploying applications, not benchmarks. The final validation therefore deployed GPT-OSS 120B using TensorRT-LLM across all four GB200 GPUs, configured with Tensor Parallelism = 4 and Expert Parallelism = 4.

Concurrency was increased from 10 to 500 using a 1,000-token input and 1,000-token output workload. At concurrency 500, the environment reached approximately: 38,000 output tokens/second

A subsequent full-suite run reproduced almost the same result, demonstrating highly consistent inference performance.

This test ties the individual infrastructure validations together. It exercises workload scheduling, GPU allocation, multi-GPU model placement, NVLink communication, TensorRT-LLM, and ultimately token generation as one end-to-end AI workload.

Putting the Results Together

The single-node GB200 environment produced the following representative results:

Certification Area Result
NCCL All-Reduce ~470 GB/s, zero errors
Full-mesh GPU D2D read ~9.2 TB/s
Bidirectional GPU D2D ~18.3 TB/s
NUMA-local host → GPU ~211 GB/s per GPU
FP32 cuBLAS ~72.5 TFLOP/s per GPU
HPCG 1,207 GFLOP/s, VALID
Grace CPU HPL 2.26 TFLOP/s
Four-GPU FP64 HPL 147.6 TFLOP/s
Four-GPU HPL-MxP 1.69 PFLOP/s
GPU STREAM Triad ~7.5 TB/s
GPT-OSS 120B ~38K output tokens/sec

Important

These numbers shouldn't be treated as universal performance thresholds. Results will vary based on hardware, firmware, software versions, workload configuration, and virtualization environment. Rather, they demonstrate the value of establishing a validated baseline against which GPU infrastructure can be evaluated.

From GPU-as-a-Service to Certified GPU-as-a-Service

For GPU clouds and AI factories, perhaps the most important takeaway isn't any individual benchmark result. It's where certification fits into the infrastructure lifecycle.

Certification on GB200 illustrates why this distinction matters.

  • NCCL validated the GPU communication fabric.
  • nvbandwidth validated the NUMA topology.
  • cuBLAS demonstrated consistency across GPUs.
  • HPL, HPCG, and STREAM validated compute and memory behavior.
  • TensorRT-LLM demonstrated that the complete environment could translate those capabilities into real LLM inference performance.

For GPU cloud providers, that represents an evolution from simply providing access to accelerators toward delivering validated, production-ready GPU infrastructure with a measurable performance baseline.

Part-3: Multi-Node Certification

Single-node certification establishes that the GPUs, CPUs, memory paths, topology, and high-speed GPU fabric within a system are behaving as expected. But modern AI workloads rarely stop at one node.

In Part 3, we'll describe how we extended the validation tests across multiple GPU systems and examine multi-node communication, scale-out NCCL performance, high-speed networking, and the additional challenges introduced when distributed AI workloads cross physical node and VM boundaries.