Unlocking Performance: Revolutionizing HPC Storage with Deos
Table of Contents:
- Introduction
- Background on Lustre Scalability
- Limitations of File System Performance Scalability
- Evolution of Storage Media and Software Stacks
- Challenges with the POSIX Interface and Block-Based I/O
- Introducing Deos: A Revolutionary Solution
- Leveraging Intel Optane Persistent Memory
- Improving Performance with Gen Over Gen Enhancements
- Key Features of Intel Optane Persistent Memory 200 Series
- Performance Improvements with Third-Generation Intel Xeon Scalable Processors
- Benchmarking Performance with IOR and MD Test
- Performance Analysis of Deos with Persistent Memory
- Results and Findings: Second Generation vs. Third Generation Intel Xeon Scalable Architecture
- Conclusion
Introduction
In this article, we will explore the performance of the Deos software stack running on the latest generation of Intel Xeon Scalable processors and Intel Optane Persistent Memory 200 series. Over the years, the HPC team at Intel has been working on tackling the challenges faced in Lustre scalability. While advancements in storage media, such as SSD technologies, have taken place, the limitations in file system performance scalability have become evident due to the legacy storage software stacks. These limitations are primarily caused by the POSIX interface and block-based I/O. To address these limitations, Intel has developed Deos, a performance-focused solution that leverages techniques from the database space to provide data consistency and unlocks the file system for future scalability.
Background on Lustre Scalability
The HPC team at Intel recognized the limitations of Lustre scalability in mass-Scale deployments. As storage media evolved with technologies like SSDs, the bottlenecks in the legacy software stacks became more prominent. The most significant bottlenecks were identified as the POSIX interface and block-based I/O. The POSIX interface, to ensure data consistency, required pessimistic locking, which led to serialization issues when multiple clients were writing to the same file. This pessimistic approach hindered scalability, making it necessary to explore alternative ways to provide data consistency and unlock the file system for future scalability.
Limitations of File System Performance Scalability
The limitations in file system performance scalability became apparent as storage media technologies advanced. While SSD technologies revolutionized storage by offering faster access times, the performance scalability was still limited by the file system software stack. The bottlenecks in the legacy software stacks, such as the POSIX interface and block-based I/O, hindered the utilization of the full potential of the evolving storage media technologies. To overcome these limitations, a more innovative and efficient software stack was required.
Evolution of Storage Media and Software Stacks
As storage media technologies evolved, Intel recognized the need for a cooperative change in both software and storage media to overcome the limitations of the legacy software stacks. Intel Optane Persistent Memory emerged as a fundamental technology to build upon. Deos, designed from the ground up for performance, combines persistent memory and NVMe SSDs with direct access through the PMDK and SPDK user space libraries to communicate directly with the media. The high-efficiency software stack built on top of this foundation provides rich functionality over RDNA-enabled Fabric.
Challenges with the POSIX Interface and Block-Based I/O
The POSIX interface, a fundamental component of the legacy storage software stack, posed challenges for scalability. To ensure data consistency, the POSIX interface required pessimistic locking, which resulted in serialization and reduced parallelism. When multiple clients were writing to the same file, the amount of serialization caused by locking on every Relevant operation hindered scalability. However, with the advancements in storage media technologies, a more scalable solution was necessary to provide data consistency without sacrificing performance.
Introducing Deos: A Revolutionary Solution
To address the limitations of the legacy storage software stack, Intel introduced Deos. This revolutionary solution utilizes techniques borrowed from the database space to provide optimistic concurrency control, eliminating the need for excessive locking. By adopting optimistic concurrency control, Deos delivers higher performance and unlocks the file system for future scalability. The innovative approach of Deos breaks free from the constraints imposed by the legacy storage software stack, allowing for improved performance and scalability.
Leveraging Intel Optane Persistent Memory
Intel Optane Persistent Memory is a fundamental technology that Deos builds upon. By leveraging the unique properties of Intel Optane Persistent Memory, Deos is able to redefine how data is accessed and stored. Unlike traditional storage media, Intel Optane Persistent Memory offers low latency and granular access to small pieces of data in Parallel. This allows applications to access data without the overhead of locking required by block-based I/O. Deos takes advantage of this by storing metadata and small I/Os in persistent memory, optimizing both data writes to NVMe SSDs for better performance.
Improving Performance with Gen Over Gen Enhancements
With the introduction of the Intel Optane Persistent Memory 200 series and the third-generation Intel Xeon Scalable processors, Deos has seen significant performance improvements compared to the previous generations. The new server platform design, based on the third-generation Intel Xeon Scalable processors, provides enhanced memory bandwidth with eight memory channels, compared to six in the previous generation. This increased memory bandwidth, coupled with other improvements in the PMEM module design, contributes to the overall performance of Deos.
Key Features of Intel Optane Persistent Memory 200 Series
The Intel Optane Persistent Memory 200 series offers several key features that drive improved performance. The module speed has been increased, providing higher transfer rates compared to the previous generation. With full two DIMMs per Channel population, the module speed reaches up to 3200 mega transfers per second. Additionally, the introduction of the EADR (Energy Store and Data Removal) capability allows the platform to Take Care of flushing data from the processor's caches, eliminating the need for synchronization points in the software stack. Finally, the third-generation server platform supports PCIe Gen4, enabling better I/O subsystems and improved overall performance.
Performance Improvements with Third-Generation Intel Xeon Scalable Processors
The third-generation Intel Xeon Scalable processors deliver significant performance improvements for Deos. With more cores per CPU (36 cores each compared to 24 cores in the second generation), the third-generation platform can handle higher workloads and deliver improved performance. The increased memory channels and higher bandwidth per DIMM in the Intel Optane Persistent Memory 200 series contribute to the significant performance gains achieved by Deos.
Benchmarking Performance with IOR and MD Test
To assess the performance of Deos with persistent memory, benchmarking tests were conducted using IOR and MD Test. These benchmarks stress various aspects of the HPC storage, such as metadata performance, chunk size, and overall scalability. MD Test measures the speed at which metadata operations can be served, including file creation, stating, unlinking, and attribute updates. On the other HAND, IOR measures the throughput performance by performing simultaneous reads and writes across multiple files using many clients.
Performance Analysis of Deos with Persistent Memory
Using both IOR and MD Test benchmarks, the performance of Deos with persistent memory was analyzed. The results showcased the impressive performance gains achieved by Deos compared to previous generations. The improved memory bandwidth, higher module speed, and support for PCIe Gen4 in the third-generation platform contributed to the significant performance improvements observed in the benchmark results. Deos with persistent memory showcased superior performance in both easy and hard workloads, outperforming traditional storage systems.
Results and Findings: Second Generation vs. Third Generation Intel Xeon Scalable Architecture
Comparing the performance of Deos running on second-generation Intel Xeon Scalable architecture with Intel Optane PM100 series to the performance on the third-generation platform with Intel Optane PM200 series, substantial improvements were observed. The third-generation platform demonstrated an average improvement of 32 percent in random 70/30 mix operations. Additionally, Deos achieved comparable performance for both easy and hard workloads, with a degradation of less than 10 percent in the hard workload. The third-generation platform delivered significant improvements in file creation and file stat rates, showing its superior performance over the second-generation platform.
Conclusion
The performance improvements achieved by Deos running on the latest generation of Intel Xeon Scalable processors and Intel Optane Persistent Memory 200 series are remarkable. The combination of software and hardware innovations, including advancements in storage media, evolution of the software stack, and the introduction of new interfaces, has revolutionized the field of HPC storage. With Deos, users can experience higher performance, improved scalability, and optimized data access, ultimately unlocking the full potential of their storage systems.
Highlights:
- Deos addresses the limitations of Lustre scalability and file system performance scalability.
- The legacy storage software stacks faced bottlenecks, primarily due to the POSIX interface and block-based I/O.
- Intel Optane Persistent Memory and Deos offer a revolutionary solution for improved performance and scalability.
- Deos leverages techniques from the database space to provide optimistic concurrency control and unlock the file system for future scalability.
- Intel Optane Persistent Memory offers low latency and granular access, optimizing data writes and improving performance.
- The third-generation Intel Xeon Scalable processors and Intel Optane Persistent Memory 200 series deliver significant performance improvements for Deos.
- Performance benchmarks, including IOR and MD Test, demonstrate the superior performance of Deos with persistent memory.
- Results show substantial performance gains compared to previous generations, with improved file creation and file stat rates.
- Deos running on the third-generation platform showcases its ability to handle both easy and hard workloads efficiently.
- The combination of software and hardware innovations has revolutionized HPC storage, unlocking the full potential of storage systems.
FAQ
Q: What are the limitations of file system performance scalability?
A: The limitations of file system performance scalability are primarily caused by the POSIX interface and block-based I/O. The POSIX interface requires pessimistic locking, leading to serialization and reduced parallelism. Block-based I/O poses a challenge for smaller I/Os, including file system metadata, as it results in more locking and serialization of activity.
Q: How does Deos address the limitations of the legacy storage software stack?
A: Deos addresses the limitations of the legacy storage software stack by introducing optimistic concurrency control and leveraging Intel Optane Persistent Memory. It borrows techniques from the database space to provide data consistency without excessive locking. Intel Optane Persistent Memory offers low latency and granular access, allowing Deos to optimize data writes and improve overall performance.
Q: What are the key features of Intel Optane Persistent Memory 200 series?
A: The key features of Intel Optane Persistent Memory 200 series include increased module speed, higher transfer rates, support for the Energy Store and Data Removal capability (EADR), and compatibility with PCIe Gen4. These features contribute to improved performance and overall efficiency in accessing and storing data.
Q: How does the third-generation Intel Xeon Scalable processors enhance performance in Deos?
A: The third-generation Intel Xeon Scalable processors offer several enhancements for performance in Deos. With more cores per CPU, higher memory bandwidth, and support for PCIe Gen4, the third-generation platform can handle higher workloads and deliver superior performance compared to earlier generations.
Q: What are the results of benchmark tests for Deos with persistent memory?
A: Benchmark tests, including IOR and MD Test, demonstrate significant performance improvements for Deos with persistent memory. The third-generation platform showcases an average improvement of 32 percent in random 70/30 mix operations. Additionally, Deos achieves comparable performance for both easy and hard workloads, with a degradation of less than 10 percent in the hard workload.
Resources: