The New Paradigm: MLPerf Inference 5.1 Confirms VCF is the Future of AI/ML Performance

MLPerf Inference 5.1 Performance with VCF on SuperMicro server with NVIDIA 8xB200 MLPerf Inference 5.1 Performance with VCF on Dell server with NVIDIA 8xH200 MLPerf Inference 5.1 Performance in VCF with Intel Xeon 6 Processor Conclusion Discover more from VMware Cloud Foundation (VCF) Blog Related Articles Deploy VCF Private AI Services in Minimal VMware Cloud Foundation Environments Using Harbor as a Proxy Cache for Cloud-Based Registries The New Paradigm: MLPerf Inference 5.1 Confirms VCF is the Future of AI/ML Performance Broadcom collaborated with Dell, Intel, NVIDIA, and SuperMicro to highlight the advantages of virtualization, delivering standout MLPerf Inference v5.1 results. VMware Cloud Foundation (VCF) 9.0 achieved performance on par with bare-metal environments across key AI benchmarks—including Speech-to-Text (Whisper), Text-to-Video (Stable Diffusion XL), LLMs (Llama 3.1-405B and Llama 2-70B), Graph Neural Networks (R-GAT), and Computer Vision (RetinaNet). These results were achieved across GPU and CPU solutions with NVIDIA’s virtualized 8x H200 GPUs, passthrough/DirectPath I/O 8x B200 GPUs, and Intel’s virtualized dual-socket Xeon 6787P processors. Refer to the official MLCommons Inference 5.1 results for the raw comparison of relevant metrics. With these results, Broadcom once again demonstrates that VCF virtualized environments perform on par with bare metal, allowing customers to benefit from the increased Agility, Availability, and Flexibility that VCF provides while leveraging excellent performance. VMware Private AI is an architectural approach that balances the business gains from AI with the privacy and compliance needs of the organization. Built on top of the industry-leading private cloud platform, VMware Cloud Foundation (VCF), this approach ensures privacy and control of their data, choice of open-source and commercial AI solutions, optimum cost, performance, and compliance. Broadcom aims to democratize AI and ignite business innovation for all organizations. VMware Private AI enables enterprises to use a range of AI solutions for their environment—NVIDIA, AMD, Intel, open–source community repositories, and independent software vendors. With VMware Private AI enterprises can deploy confidently, knowing that Broadcom has built partnerships with the leading AI providers. Broadcom brings the power of its partners Dell, Intel, NVIDIA, and SuperMicro to VCF to simplify management of AI accelerated data centers and enable efficient application development and execution for demanding AI/ML workloads. We showcase three configurations in VCF: SuperMicro GPU SuperServer AS-4126GS-NBR-LCC with NVLinked 8xB200s in DirectPath I/O, Dell PowerEdge XE9680 with NVlinked 8xH200s in vGPU mode, and 1-node-2S-GNR_86C_ESXi_172VCPU-VM with INTEL(R) XEON(R) 6787P 86-core CPUs.