Revolutionizing AI SoCs with Die-to-Die Connectivity

Revolutionizing AI SoCs with Die-to-Die Connectivity

Table of Contents

1. Introduction
2. Scaling AI SoCs
3. Trends in IP for AI SoCs
4. Die-to-Die Connectivity
   • 4.1 SRAM Intensive Chips
   • 4.2 Die Splitting for Improved Yields
   • 4.3 High Bandwidth and Low Power Requirements
   • 4.4 Achieving Low Latency and Zero BER
5. Markets Served by Die-to-Die Solutions
   • 5.1 Organic Substrate
   • 5.2 Interposer Technology
   • 5.3 InFO Technology
6. Proprietary Controllers and Fragmented Market
7. Synopsys' Approach to Interconnect Solutions
8. Collaboration and Coordination with Ecosystem
9. Conclusion
10. FAQs

💡 Highlights

• The complexity of AI algorithms has led to the scaling of AI SoCs with multiple systems and high-performance requirements.
• Die-to-die connectivity plays a crucial role in improving yields, achieving low latency, and minimizing bit error rate (BER).
• Markets are served by both organic substrate and interposer technology, each offering different speeds and density options.
• The market for die-to-die solutions is currently fragmented, with proprietary controllers being widely used.
• Synopsys addresses the challenges with optimized IP solutions, collaboration with ecosystem partners, and customization options for customers.

🧩 Introduction

As AI algorithms continue to evolve and become more complex, the demand for high-performance AI Systems-on-Chip (SoCs) is on the rise. These AI SoCs consist of integrated circuits that combine processing capabilities and SRAM (static random access memory) to handle the computational requirements. However, the increasing complexity often necessitates scaling these systems to multiple dies. In this article, we will explore the trends, challenges, and solutions in the field of AI SoCs and die-to-die connectivity.

🔬 Scaling AI SoCs

Scaling AI SoCs has become a necessity to meet the requirements of modern AI algorithms. The ever-increasing demand for processing power and SRAM capacity has led to the integration of more components into a single chip. As a result, the size of these chips is approaching the maximum reticle size and, in some cases, even exceeding it. This trend poses challenges in terms of manufacturing yields and performance.

📈 Trends in IP for AI SoCs

Synopsys, a leading provider of IP solutions, delivers specialized technology for AI SoCs. One prominent trend in these systems is the need for SRAM-intensive chips with localized SRAMs in close proximity to the intelligent processing units (IPUs). This approach helps optimize performance and power efficiency. As AI SoCs continue to Scale, the requirement for high bandwidth, low power, low latency, and zero bit error rate (BER) becomes essential.

🧱 Die-to-Die Connectivity

Die-to-die connectivity refers to the interconnection between multiple dies in an AI SoC system. It plays a vital role in improving yields, reducing latency, and ensuring reliable data transfer between different components. Let's explore the key aspects of die-to-die connectivity in the context of AI SoCs.

📚 4.1 SRAM Intensive Chips

AI SoCs often require a significant amount of SRAM for efficient data storage and retrieval. As the chips become larger, accommodating a massive amount of SRAM on a single die becomes challenging. To address this, the industry is now adopting an approach known as die splitting.

📚 4.2 Die Splitting for Improved Yields

Die splitting involves dividing a large chip into multiple dies, each with localized SRAMs close to the IPUs. This technique helps improve manufacturing yields by reducing the impact of defects and increasing the overall performance of the system. However, to maintain seamless connectivity among the split dies, robust die-to-die interfaces are required.

📚 4.3 High Bandwidth and Low Power Requirements

Die-to-die interfaces must provide high bandwidth to facilitate efficient data transfer between the multiple dies. Simultaneously, power consumption needs to be minimized to ensure optimal energy efficiency. The industry is constantly pushing for lower power consumption while still achieving high performance.

📚 4.4 Achieving Low Latency and Zero BER

In addition to high bandwidth and low power, die-to-die interfaces must also offer low latency and a minimal bit error rate. This is particularly crucial when splitting the die right in the center since it enables different CPUs in different dies to function as a single unified entity. Achieving nearly zero BER is essential to ensure reliable data transfer and preserve the integrity of AI algorithms.

🌍 Markets Served by Die-to-Die Solutions

Die-to-die solutions cater to two distinct market segments: organic substrates and interposers. Let's delve into each of these markets:

🗺️ 5.1 Organic Substrate

Organic substrate technology provides an economical solution for die-to-die connectivity. It utilizes ultra-short reach and extra-short reach serializer/deserializer (SerDes) technologies that offer speeds ranging from 56 Gigabits per Second (NRZ) to 112 PAM-4. This technology is suitable for applications that don't require high-density interconnections.

🗺️ 5.2 Interposer Technology

Interposer technology is mainly used in applications that demand high-density interconnections, such as AI accelerators incorporating high-bandwidth memory (HBM). Interposers offer speeds aligned with HBM, starting from 3.2 gigabits per second and expected to reach over six gigabits per second in the future.

🗺️ 5.3 InFO Technology

InFO (integrated fanout) technology serves as an intermediate solution between organic substrates and interposers. It promises higher density than organic substrates while being less complex and cheaper than interposers. TSMC (Taiwan Semiconductor Manufacturing Company) has developed InFO technology, which offers enhanced performance and cost-effectiveness.

🔌 Proprietary Controllers and Fragmented Market

The current market for die-to-die connectivity is fragmented, with various proprietary controllers and solutions available. Given that many customers own both sides of the link, they often develop proprietary controllers tailored to their specific needs. These controllers range from light transport layers to interlocking controllers, incorporating technologies like forward error correction and replay.

While standardization efforts, such as Open Compute platform and PIPE 5.1.1 interface, are gaining traction, the market still heavily relies on proprietary solutions. The lack of standardization presents both challenges and opportunities for IP providers like Synopsys to collaborate with customers and deliver optimized interconnect solutions.

💡 Synopsys' Approach to Interconnect Solutions

Synopsys addresses the complex challenges of die-to-die connectivity with a comprehensive portfolio of IP solutions. The company offers ultra-short reach and extra-short reach SerDes IP, compliant with the latest industry standards. These IPs are optimized for the five key requirements - bandwidth, power, latency, bit error rate, and reach - offering customers a performance edge in their AI SoC designs.

Synopsys collaborates closely with foundries, packaging vendors, and bump houses to stay abreast of the latest packaging technologies. The company also focuses on the digital aspects of interconnect solutions, aiming to provide customers with customizable options and seamless integration with their existing digital architectures.

🤝 Collaboration and Coordination with Ecosystem

Synopsys recognizes the importance of collaboration and coordination within the ecosystem. The company actively engages with customers, foundries, and other ecosystem partners to ensure interoperability and compatibility of their IP solutions. This collaborative approach allows customers to leverage Synopsys' expertise and seamlessly integrate the interconnect solutions into their Fabric bus designs.

🎯 Conclusion

The field of AI SoCs and die-to-die connectivity is constantly evolving to meet the demands of ever-advancing AI algorithms. The scaling of AI SoCs, die splitting, die-to-die interfaces, and interconnect technologies pose significant engineering challenges. However, companies like Synopsys are at the forefront, addressing these challenges by delivering optimized IP solutions, fostering collaboration, and striving for standardization. As the AI SoC market continues to grow, exciting times lie ahead with advancements in die-to-die connectivity and the broader AI ecosystem.

🙋‍♀️ FAQs

Q1: What are the key requirements for die-to-die interfaces in AI SoCs?

• Bandwidth, power efficiency, latency, bit error rate (BER), and reach are the key requirements for die-to-die interfaces in AI SoCs. Achieving high performance while minimizing power consumption and maintaining low latency and a near-zero BER is crucial for seamless operation and reliable data transfer.

Q2: What are the different markets served by die-to-die solutions?

• Die-to-die solutions cater to two main market segments: those that require an organic substrate and those that utilize interposer technology. Organic substrates are suitable for applications that don't necessitate high-density interconnections, while interposers provide high-density interconnections for demanding applications such as AI accelerators incorporating high-bandwidth memory (HBM).

Q3: How does Synopsys address the fragmented market and proprietary controllers?

• Synopsys offers a comprehensive portfolio of IP solutions, including ultra-short reach and extra-short reach SerDes IP. While the market still relies on proprietary controllers, Synopsys collaborates closely with customers to understand their specific requirements and deliver optimized interconnect solutions. The company also strives towards standardization in the industry through initiatives like the Open Compute platform and PIPE interfaces.

Q4: How does Synopsys ensure compatibility and seamless integration of its interconnect solutions with the customer's fabric bus?

• Synopsys actively collaborates with ecosystem partners, including foundries, packaging vendors, and bump houses, to ensure compatibility and interoperability of its IP solutions. This collaboration enables customers to leverage Synopsys' expertise and integrate the interconnect solutions smoothly into their fabric bus designs.

Resources:

• Learn more at Synopsys.com