Unveiling the Secrets of Intel Omni-Path: A Journey to Open Source

Table of Contents

1. Introduction
2. Background of Omni Path
3. Open Sourcing Omni Path
4. The Three Phases of Open Sourcing a Project
5. The Ecosystem of Omni Path
6. Components of the Omni Path Ecosystem
   1. Fabric Manager
   2. User Space Tools
   3. Address Resolution
   4. MPI Benchmarks
7. Enhancements for Omni Path
   1. OpenMPI
   2. lib IBU mad
   3. GASNet
   4. IBA CM
   5. Linux Firmware
   6. Core Patches in the RDMA stack
   7. IP over IB enhancements
8. Recent Additions to Omni Path
   1. RDMA VT
   2. Kernel Changes
9. Summary of GitHub Repositories
10. Conclusion

Introduction

Welcome to the world of Omni Path! In this article, we will delve into the fascinating technology behind Omni Path, its origin, and its journey towards open sourcing. As we explore the various aspects of this cutting-edge technology, we will also discuss the ecosystem surrounding Omni Path and the enhancements made to optimize its performance. So, let's get started and uncover the secrets of Omni Path.

Background of Omni Path

To truly understand Omni Path, it is essential to delve into its origins. Omni Path was borne out of Intel's acquisition of Cue Logic and part of Cray, bringing together a core team dedicated to its development. The team's vision was to create a fabric that would revolutionize data transmission and deliver innovative features, such as new Quality of Service (QoS) capabilities. By leveraging their expertise and collaboration with the community, they set out to make Omni Path a Game-changer.

Open Sourcing Omni Path

One of the key milestones in the journey of Omni Path was its transition to an open-source project. Open sourcing Omni Path was a strategic move to foster collaboration, expand its reach, and ensure that the entire community could contribute to its development. This decision was guided by the realization that open source projects thrive in an environment where the community actively participates, shares ideas, and contributes back to the project. By open sourcing their code and working with other organizations, Omni Path aimed to create a thriving ecosystem where everyone can work together towards a common goal.

The Three Phases of Open Sourcing a Project

Open sourcing a project involves three distinct phases. The first phase is finding the project and hosting the code in a suitable environment. Omni Path's code was hosted on well-known platforms like GitHub and kernel.org, ensuring easy accessibility for developers and enthusiasts alike. The Second phase revolves around packaging the project and delivering it to customers. Omni Path collaborated with industry leaders like RedHat and SUSE to integrate their software into mainstream distributions, ensuring a seamless and widespread adoption of their technology. The final phase, which can be deemed as "parole if eration," is the formation of a community that actively uses the open-source code and contributes back to its development. Omni Path sought to create a sharing ecosystem where diverse minds collaborate, incorporating user feedback and continuously improving the technology.

The Ecosystem of Omni Path

Similar to the InfiniBand ecosystem, Omni Path has built its own ecosystem of elements and enhancements to support its fabric technology. By creating an ecosystem, Omni Path aligns itself with the existing infrastructure, leveraging the benefits and compatibility offered by the InfiniBand stack. This ensures seamless integration and allows users to harness the power of both InfiniBand and Omni Path.

Components of the Omni Path Ecosystem

To understand the intricacies of the Omni Path ecosystem, let's explore its various components:

1. Fabric Manager

At the heart of the Omni Path ecosystem lies the fabric manager. Similar to the SM (Subnet Manager) in the InfiniBand ecosystem, Omni Path's fabric manager extends and enhances the functionality to support the Omni Path fabric. It comprises five essential elements: SM, SA, PM, PA, and FE. The Fabric Executive (FE) enables secure SSH access to the manager for statistical analysis and management. The code for the fabric manager is hosted on a dedicated repository, allowing developers and users to explore its capabilities.

2. User Space Tools

To facilitate the management and debugging of the fabric, Omni Path provides a set of user space tools. These tools assist in bringing up the fabric and diagnosing any issues that may arise. The tools are divided into two categories: one for the compute nodes and the other for the management node. RPM packages are created for easy installation and usage. These tools provide essential insights and aid in the smooth operation of the fabric.

3. Address Resolution

Address resolution plays a crucial role in ensuring efficient communication within the fabric. Omni Path has enhanced the existing IB CM (InfiniBand Communication Manager) by providing a plugin that enables the sharing of path Record data between IB CM and their PSM (Performance Scaled Messaging) library. This enhancement optimizes data transfer and ensures that applications can utilize the fabric's capabilities seamlessly.

4. MPI Benchmarks

To verify the fabric's performance and benchmark its capabilities, Omni Path offers an MPI (Message Passing Interface) benchmarks Package. Leveraging the power of open MPI and ADVA (Advanced Network Devices Architecture), users can thoroughly test and validate the fabric's performance. This package serves as a valuable asset for developers and organizations looking to harness the power of Omni Path.

Stay tuned for the next sections as we dive deeper into the enhancements made to Omni Path and the recent additions that further strengthen its position as a leading fabric technology.

（Continued in the next section...)