Revolutionize AI Workloads with Untether's At Memory Computation

Table of Contents

1. Introduction
2. About Untether
3. The Energy Challenge in AI Workloads
4. The Untether Solution: At Memory Computation
5. The Architecture of the Run AI 200 Chip
6. The Tsunami Accelerator Card
7. Efficiency and Performance Optimization
8. Untether's Software Tools and Optimization Techniques
9. The Future of Untether's Products
10. Conclusion

Introduction

In this article, we will delve into the innovative technology and products developed by Untether, a Toronto-based company focused on accelerating AI workloads. With the rapidly increasing demand for AI computations, Untether aims to help companies execute these workloads faster, cooler, and cheaper. We will explore the energy challenges faced by traditional processing approaches and how Untether's at memory computation solution addresses these challenges. Furthermore, we will examine the architecture of their Run AI 200 chip, the benefits of their Tsunami Accelerator Card, and the software tools and optimization techniques available to users. Finally, we will discuss Untether's future plans and the impact their products will have on AI computing.

About Untether

Untether, founded in 2018 and based in Toronto, is a company dedicated to enhancing the execution of AI workloads. With close ties to the University of Toronto and the University of Waterloo, Untether operates at the forefront of AI research and development. Backed by venture capital firms Radical Ventures and Intel Capital, Untether is well-equipped to bring their innovative solutions to the AI market. Their mission is to help companies execute AI workloads faster, cooler, and cheaper through the use of at memory computation technology.

The Energy Challenge in AI Workloads

As neural networks and AI computations continue to advance, the demands on processing power and energy consumption have reached unprecedented levels. The energy spent on performing multiply-accumulate operations, a fundamental component of AI workloads, has become a significant bottleneck. Traditional processing approaches, such as the Von Neumann architecture, allocate a large portion of energy to data movement instead of computation. This inefficiency is no longer sustainable in the face of rapidly growing AI workloads.

The Untether Solution: At Memory Computation

Untether has developed a disruptive solution to the energy challenge in AI workloads called at memory computation. By reimagining the traditional processing approach, Untether focuses on inference first and eliminates the need for excessive data movement. Their innovative approach involves storing coefficients directly on the chip itself, minimizing data movement and maximizing concurrency. With hundreds of thousands of processing elements located directly in the memory array, Untether achieves both massive parallelization and high memory bandwidth. This results in a significant reduction in energy consumption and improved computation efficiency.

The Architecture of the Run AI 200 Chip

At the heart of Untether's solution is the Run AI 200 chip. This chip has 200 megabytes of on-chip SRAM, distributed across 511 memory banks. Each memory bank contains 385 kilobytes of SRAM and 512 processors, creating a highly scalable architecture. The processing elements are located directly in the memory array, with coefficients stored in the SRAM array. This proximity minimizes data transfer distances and ensures efficient computation. The Run AI 200 chip offers a tremendous memory bandwidth of 262 terabytes per Second, enabling high-performance AI workloads.

The Tsunami Accelerator Card

Untether's Tsunami Accelerator Card is a powerful solution that combines four Run AI 200 chips on a single PCIe card. With two peta operations per second, this card delivers unmatched compute density for AI inference workloads. The chips are interconnected using a PCIe switch chip, providing efficient communication between the chips. The Tsunami Accelerator Card can operate in both eco and sport modes, offering scalability and flexibility to suit various computing needs. With exceptional compute density, this accelerator card outperforms any other product currently on the market.

Efficiency and Performance Optimization

Untether's emphasis on efficiency and performance optimization sets them apart from conventional AI hardware solutions. Their software tools and optimization techniques streamline the development and deployment processes, making it easier for users to harness the full potential of their products. Untether offers an untether-aware training process for quantization, allowing users to regain accuracy lost during the quantization process. Additionally, users can specify their optimization constraints, such as maximizing efficiency or performance, and Untether's optimization techniques automatically handle multi-chip partitioning and duplication of layers to achieve the desired results.

Untether's Software Tools and Optimization Techniques

Untether provides users with powerful software tools to maximize the efficiency and performance of their AI workloads. These tools automate the optimization process, including quantization, physical allocations, and graph-level optimizations. The user-friendly interface allows users to Visualize and analyze their network graphs, enabling them to make informed decisions. Untether's customized risk processor and runtime API facilitate efficient communication and coordination between the hardware and software components. Through these tools and techniques, Untether empowers users to push the boundaries of AI computation.

The Future of Untether's Products

Untether's commitment to innovation and performance leadership positions them as a key player in the AI hardware market. The pet-ops era has just begun with Untether's Tsunami Accelerator Card delivering unrivaled compute density. As they continue to refine their products and expand their offerings, Untether aims to provide even greater advancements in AI computing. With ongoing research and development, Untether will meet the evolving needs of the AI industry and contribute to groundbreaking advancements in artificial intelligence.

Conclusion

Untether's revolutionary approach to AI computation offers a Game-changing solution for companies seeking to maximize the efficiency and performance of their AI workloads. By focusing on at memory computation and minimizing data movement, Untether's products offer significant energy savings and improved compute density. With their Run AI 200 chip, Tsunami Accelerator Card, and powerful software tools, Untether provides the infrastructure necessary for efficient and high-performance AI computing. As the demand for AI workloads continues to grow, Untether is poised to lead the way in advancing AI hardware and unlocking new possibilities in artificial intelligence.

Highlights

• Untether's at memory computation technology revolutionizes AI workloads, reducing energy consumption and improving computation efficiency.
• The Run AI 200 chip combines hundreds of thousands of processing elements with high memory bandwidth, enabling high-performance AI computations.
• The Tsunami Accelerator Card offers unparalleled compute density, outperforming any other product on the market.
• Untether's software tools and optimization techniques automate the optimization process, making it easier for users to achieve maximum efficiency and performance in AI workloads.
• Untether's commitment to innovation positions them as a key player in the AI hardware market, with future advancements on the horizon.

FAQ

Q: How does Untether's at memory computation technology work?

A: Untether's at memory computation technology eliminates the need for excessive data movement in AI workloads by storing coefficients directly on the chip itself. This reduces energy consumption and improves computation efficiency.

Q: What are the benefits of Untether's Tsunami Accelerator Card?

A: Untether's Tsunami Accelerator Card combines four Run AI 200 chips, delivering unmatched compute density for AI inference workloads. It offers exceptional performance and scalability in a compact form factor.

Q: How does Untether optimize efficiency and performance?

A: Untether provides software tools and optimization techniques that automate the optimization process, such as quantization and multi-chip partitioning. Users can specify their optimization constraints, and Untether's techniques handle the rest to maximize efficiency and performance.

Q: What is the future of Untether's products?

A: Untether aims to continue advancing AI hardware and meet the evolving needs of the industry. With ongoing research and development, they will contribute to groundbreaking advancements in artificial intelligence.