Unveiling the Power of Super Micro Intel Omni Path 48x 100Gbps Switch

Table of Contents

1. Introduction
2. What is Intel Omni Path?
3. Deployment of Omni Path
   • 3.1 PCIe Integration
   • 3.2 On-Chip Integration
4. Advantages of Omni Path
   • 4.1 Cost-effectiveness
   • 4.2 Dedicated PCIe Lanes
5. Challenges and Limitations
   • 5.1 Cable Availability
   • 5.2 Non-Inclusion in Cascade Lake
6. The Super Micro Intel Omni Path Switch
   • 6.1 Model: Super Micro SSHCee 48q
   • 6.2 External Ports and Connectivity
   • 6.3 Internal Components
7. Cooling and Power
8. Management Card
9. Use Cases and Demo Setup
10. Conclusion

🖥️ Intel Omni Path: High-Speed Networking for Server Hardware

In today's digital landscape, where data processing and communication play a crucial role, innovations in server hardware platforms are continuously pushing the boundaries of performance and efficiency. One such technology that showcases significant advancements is the Intel Omni Path (OPA) switch. Designed to compete with established high-performance computing (HPC) fabrics like InfiniBand and Ethernet, OPA offers lightning-fast connectivity and seamless integration. In this article, we delve into the world of Intel Omni Path, exploring its deployment strategies, advantages, limitations, and examining the Super Micro Intel Omni Path 48-port switch.

What is Intel Omni Path?

Intel Omni Path originated as a high-speed Fabric capable of delivering 100 Gbps (gigabits per Second) transfer rates. Its primary aim was to cater to the demanding computing requirements of the HPC domain. Intel's intention was clear - to provide a viable alternative to existing technologies like InfiniBand and Ethernet. Two deployment methods were employed for Omni Path - PCIe integration and on-chip integration.

Deployment of Omni Path

3.1 PCIe Integration

The initial deployment of Omni Path took the form of a standard PCIe x16 Gen3 card. Functioning similar to popular Ethernet NICs or Mellanox InfiniBand adapters, this approach ensured compatibility with existing server infrastructure. However, it was the second deployment method that stood out as intriguing.

3.2 On-Chip Integration

In a bid to stimulate adoption, Intel integrated Omni Path fabric adapters directly onto chip packages. This integration was not limited to a single generation of chips, as it spanned across the Xeon Phi Knights Landing and Knights Mill, as well as the Cascade Lake predecessors. The incorporation of the fabric adapter onto the chip package brought remarkable benefits. Not only did it save PCIe lanes dedicated to networking, but it also significantly reduced the cost overheads associated with purchasing separate adapters. With a mere $155 addition to the cost of a standard skew, organizations could enjoy the advantages of a hundred gigabit fabric connector.

However, despite the cost-effectiveness and integration benefits, Omni Path faced some hurdles along the way.

Advantages of Omni Path

4.1 Cost-effectiveness

One of the major advantages of Omni Path's design was its affordability. By integrating the fabric adapter directly onto chip packages, Intel offered a cost-effective solution to organizations. With a modest uplift in price, the convenience of built-in hundred gigabit fabric connectors became accessible to a broader user base.

4.2 Dedicated PCIe Lanes

Another notable advantage stemmed from the integration approach's technical aspect. With the fabric adapter Present on the chip Package, organizations no longer needed to allocate additional PCIe lanes for high-speed networking. The dedicated PCIe lanes on the Intel chips facilitated seamless communication, eliminating potential bottlenecks and enhancing overall system efficiency.

Despite its advantages, Intel Omni Path encountered a few challenges during deployment.

Challenges and Limitations

5.1 Cable Availability

One of the minor setbacks during Omni Path's deployment was the unavailability of suitable cables. Vendors faced difficulties sourcing cables that could connect CPUs to the chassis. While this issue did not impede functionality, it posed a slight inconvenience to system integrators.

5.2 Non-Inclusion in Cascade Lake

Another surprising turn of events came when Intel's subsequent chip generation, Cascade Lake, did not include Omni Path integration. This omission hinted at a reset for Omni Path's next iteration, OPA 200, which was announced but not widely seen or adopted. Despite this setback, Intel remains committed to further developing and refining the Omni Path technology.

Now, let's take a closer look at the Super Micro Intel Omni Path 48-port switch.

The Super Micro Intel Omni Path Switch

6.1 Model: Super Micro SSHCee 48q

Among the various designs available in the market, the Super Micro SSHCee 48q stands as an excellent representation of Intel's Omni Path ecosystem. With its 48 qSSP28 ports on the front panel, this switch closely resembles Ethernet and InfiniBand connectors. While several other manufacturers produce similar designs based on Intel's reference model, we chose the Super Micro switch for a detailed examination.

6.2 External Ports and Connectivity

The Super Micro SSHCee 48q switch features a total of 48 qSSP28 ports on its front panel. These ports, alongside a manageable B port, ensure seamless connectivity, offering flexible data transfer options for high-performance computing environments.

6.3 Internal Components

Upon opening the switch, we discover a remarkably neat and uncomplicated internal design. The components are strategically placed, resulting in simplistic functionality. The rear of the chassis houses standard 1U fans, which employ four-Pin PWM power connectors commonly found in white-box servers. While these fans are not hot-swappable, they are easily replaceable, thanks to their convenient extension cables. Additionally, the switch utilizes 750-watt redundant power supplies, further enhancing its reliability in critical operations.

Cooling and Power

The cooling system primarily revolves around the fans and power distribution board within the switch. The 1U fans ensure adequate airflow, preventing excessive heat build-up. Combined with the redundant power supplies, this design guarantees consistent performance even under high-stress conditions. Notably, the cooling and power requirements of the Super Micro Intel Omni Path switch are relatively modest compared to more advanced switches available today.

Management Card

A noteworthy feature of the Super Micro Intel Omni Path switch is the presence of a management card. Found inside the switch, this card houses a CPU and memory, enabling comprehensive control and management functions. This additional computational power enhances the switch's capabilities beyond its primary role of Speedy data transfer. While management cards of this nature are common in high-end switches, the Super Micro SSHCee 48q provides a compelling glimpse into the control plane operations.

Use Cases and Demo Setup

During its tenure in our lab, the Super Micro Intel Omni Path switch demonstrated its versatility. It seamlessly integrated into our Muller fabric setup and showcased excellent performance. In fact, we even tested an NVMe over Fabric demo with this switch, further highlighting its capabilities for high-speed data transfers. Such demonstrations underline the practical usability and potential applications of Omni Path technology.

Conclusion

The Intel Omni Path switch, exemplified by the Super Micro SSHCee 48q, represents a significant innovation in high-speed networking for server hardware. By providing cost-effective integration options and efficient data transfers, Omni Path has gained recognition in the HPC domain. Though faced with minor availability and adoption issues, its potential remains promising. As Intel progresses towards OPA 200 and beyond, the advancements in Omni Path technology continue to Shape the future of server hardware and high-performance computing.

Resources:

• Intel Omni Path
• Super Micro