Unleash the Power of Intel Gaudi 2 AI Accelerator and DDN Storage

Table of Contents

1. Introduction
2. The Rise of AI in HPC
3. The Intel Gaudi 2 AI Accelerator
4. Proven Performance of the Gaudi 2
   • MLPerf Benchmark Results
   • Price-Performance Comparison
   • Linear Scalability
   • Ease of Use with SYNAPSE Software Stack
5. Exploring the Architecture of Gaudi 2
   • Heterogeneous Computer Engine
   • Matrix Multiplication Engines
   • Soft Managed Memory Architecture
   • Media Engine for Visual Workloads
   • Scalability with Rocky Connectivity
6. Unveiling the Gaudi 2 Accelerator Server
   • Super Micro Server Configuration
   • Scale-Up and Scale-Out Capabilities
   • Integration with DDN Storage Systems
7. The Power of DDN Storage Servers
   • Scalable Architecture for AI and Large Models
   • Provisioning Based on GPU Requirements
   • High-Speed Read and Write Capabilities
8. The Synapse AI Software Stack
   • Embedded and Hardware-Agnostic Tools
   • Collaboration with PyTorch and Hugging Face
   • Easy Porting of Existing Models to Gaudi 2
9. Progress and Milestones with Gaudi 2
   • Rapid Development and Porting of Models
   • Inference and Training on Large Language Models
10. Explore and Experience Gaudi 2
    • Workshops and Credits on the Intel Developer Cloud
    • Super Micro Server Display and Contact Information
11. Conclusion

The Rise of AI in HPC

Artificial Intelligence (AI) has become an integral part of High-Performance Computing (HPC) with its ability to generate simulation data, analyze outputs, and enhance various computational processes. As AI continues to evolve, there is a growing demand for increased computing power and storage capacity. To meet these requirements, Intel has collaborated with DDN, a leading storage solutions provider, to create an innovative storage appliance by combining the power of the Gaudi 2 AI accelerator servers with DDN storage systems.

The Intel Gaudi 2 AI Accelerator

The Intel Gaudi 2 AI accelerator is specifically designed for deep learning acceleration at scale. With its proven performance, it offers a compelling alternative to the h100 for machine learning workloads. The Gaudi 2 has demonstrated impressive results in MLPerf benchmarks, doubling the training speed of GPT-3 models compared to h100 and delivering fast inference with only 9% of the h100's performance. Furthermore, in terms of price-performance, the Gaudi 2 outperforms both h100 and a100 GPUs.

Proven Performance of the Gaudi 2

The performance of the Gaudi 2 is its key strength, showcasing its capabilities in handling large-scale language models and complex AI workloads. With 24 Tensor Processing Units (TPUs) and dual matrix multiplication engines, it excels in deep learning training and inference tasks. The Gaudi 2 also boasts a soft managed memory architecture, combining high-bandwidth memory (HBM2) and SRAM to optimize data access and processing. The inclusion of a media engine further enhances performance for visual workloads.

Exploring the Architecture of Gaudi 2

The heterogeneous computer engine of Gaudi 2, coupled with its matrix multiplication engines, enables efficient handling of complex matrix operations required in deep learning processes. Its memory architecture, consisting of HBM2 and SRAM, provides fast and scalable memory access for improved performance. The media engine adds versatility by enabling the execution of visual workloads alongside AI tasks. The integration of Rocky connectivity ensures seamless scaling capabilities and enhanced connectivity.

Unveiling the Gaudi 2 Accelerator Server

The Gaudi 2 accelerator server, developed in collaboration with Super Micro, offers a powerful AI training platform. With eight Gaudi 2 accelerators and multiple high-speed Ethernet ports, this server provides exceptional performance and scalability. The configuration enables both scale-up and scale-out approaches to meet diverse computational needs. In combination with DDN storage systems, the Gaudi 2 server transforms into a storage appliance, capable of handling large datasets and models.

The Power of DDN Storage Servers

DDN storage servers offer unmatched storage capabilities for AI and large models. These servers are designed to provide high-speed read and write access, enabling efficient data handling by Gaudi 2 accelerators. DDN's scalable architecture ensures adequate storage provisioning based on the number of GPUs, allowing seamless expansion as computational requirements grow. With global support from data experts, DDN storage servers offer exceptional performance and reliability.

The Synapse AI Software Stack

The Synapse AI software stack is specifically tailored for the Gaudi family of AI accelerators. It includes embedded tools, programming toolkits, and hardware-agnostic components that optimize deep learning training and inference. Intel has collaborated with prominent AI frameworks such as PyTorch and Hugging Face to ensure seamless compatibility and performance. Porting existing models to the Gaudi 2 is made easy with just a few lines of code modification, allowing users to leverage the power of Gaudi 2 without significant code changes.

Progress and Milestones with Gaudi 2

Intel's rapid development and porting of models on the Gaudi 2 have resulted in significant milestones. In less than six months, inference on large language models with over 100 billion parameters was achieved. Within twelve months, large-scale training on these models became a reality. The Gaudi 2's performance has been tested and validated on various models, including those available on the Hugging Face platform. The progress made showcases the immense potential of Gaudi 2 in the AI and HPC domains.

Explore and Experience Gaudi 2

To experience the power of Gaudi 2, Intel offers workshops and credits on the Intel Developer Cloud. These resources provide an opportunity to learn more about Gaudi 2, its capabilities, and its integration with the Intel ecosystem. Additionally, Super Micro has Gaudi 2 servers on display, allowing interested individuals to explore the hardware in person. The combination of Gaudi 2 accelerator servers with DDN storage systems is a Game-changer in the AI and HPC landscapes.

Conclusion

The collaboration between Intel and DDN has resulted in a storage appliance that combines the Gaudi 2 AI accelerator servers with powerful storage capabilities. The Gaudi 2 delivers exceptional performance, scalability, and ease of use. The integration of DDN storage systems ensures efficient handling of large datasets and models. With the Synapse AI software stack and collaboration with key AI frameworks, porting existing models to Gaudi 2 is Simplified. The future of AI in HPC is brighter than ever, thanks to innovative solutions like the Gaudi 2 storage appliance.

Frequently Asked Questions (FAQ):

Q: What is the Gaudi 2 AI accelerator designed for? A: The Gaudi 2 AI accelerator is purpose-built for deep learning acceleration at scale.

Q: How does the Gaudi 2 compare to the h100 in terms of performance? A: The Gaudi 2 has proven to deliver faster training speed and better price-performance compared to the h100.

Q: Can the Gaudi 2 handle large language models? A: Yes, the Gaudi 2 has demonstrated its ability to handle large language models of over 100 billion parameters.

Q: Is it possible to port existing models to the Gaudi 2? A: Yes, Intel has made it easy to port existing models to the Gaudi 2 with just a few lines of code modification.