As GPU Counts Rise, the Bottleneck Grows: Cisco Redesigns the Backend Network
IT DAILY ·
✦ AI Summary
As AI infrastructure grows, the importance of networks that handle high-speed data exchange between GPUs is increasing, not just GPU performance itself.
Cisco Korea introduced its network strategy for AI infrastructure and key product lines at a media tech session on the 20th.
Cisco provides switches based on its own ASICs and NVIDIA Spectrum-X ASICs, and is pushing ahead with eliminating GPU communication bottlenecks through a Dual-plane architecture.
As AI infrastructure expands, the importance of high-speed data exchange networks between GPUs is growing alongside GPU computing performance. Against this backdrop, a trend has emerged in which the network that handles communication between GPUs is becoming as important as GPU performance itself.
In this context, eliminating bottlenecks in GPU-to-GPU communication has emerged as a challenge. A typical GPU server network is divided into a management network, a front-end network for service delivery, a network for storage connections based on an IP network, and a backend network for GPU connections.
Cisco Korea held a media tech session on the 20th. The session introduced the company’s network strategy for AI infrastructure and key product lines.
The presentation was delivered by Choi Soo-young, managing director at Cisco Korea TSAG. At the event, Choi explained the company’s AI infrastructure network strategy and products.
Cisco's strategy is to combine the introduction of high-performance switches with improved network design. Cisco also said it aims to address large-scale AI infrastructure demand with an Ethernet-based integrated management framework.
Choi Soo-young, managing director at Cisco Korea TSAG, said the backend network is the key factor determining the performance of large-scale AI infrastructure and also the bottleneck. He said that in environments where multiple GPUs perform joint computing, fast data exchange between GPUs is necessary. He added that a single server uses 8 or 16 network ports.
Choi said that such environments require a large-scale network configuration capable of supporting fast transfer speeds and a large number of connections. To implement this, he said high-performance switching ASICs for handling heavy traffic, switches equipped with those ASICs, and management solutions for operating large-scale networks are needed.
Cisco said it provides switches based on its own in-house ASICs as well as switches based on NVIDIA Spectrum-X ASICs. It also said it applies Cisco's operating system, NX-OS, to products using NVIDIA ASICs. The goal is to allow customers to use the operating environment they are already familiar with.
Choi said the two product lines differ in the ASICs they use and the features they support, even if their per-port transmission speeds and number of ports are the same. He said customers can choose products by considering the functions and operating environment they have used in their existing networks, as well as additional features for NVIDIA-based AI infrastructure.
Cisco also said it is pushing ahead with changes to network connectivity structures alongside improvements in ASIC performance. In the existing structure, one NIC was mapped for every 8 GPU ports.
However, the existing structure had a limitation in that the number of switch ports was fixed, restricting the scale of GPUs that could be connected. In response, Cisco proposed a Dual-plane architecture.
Cisco explained that the Dual-plane method splits NIC ports into two separate links of 400Gbps or 800Gbps, with the core idea being that the two ports of the NIC are connected to different planes. The company said this design eases the port-count constraints of the existing structure, supports more GPUs, improves network scalability, and reduces the number of switches data must traverse during GPU communication.
Cisco is seeking to expand its ability to supply AI infrastructure in partnership with Supermicro, and plans to broaden its AI infrastructure offering by combining its network, optical communications, and security solutions, along with Splunk-based observability solutions, with Supermicro's diverse GPU server lineup. Through this, Cisco is pushing ahead with a vision of providing AI infrastructure that encompasses computing, networking, and operations management.
Source: IT DAILY · Seong Won-young
Original: https://www.itdaily.kr/news/articleView.html?idxno=242328
References
This article was produced with the help of an automated content generation algorithm.
Source: IT DAILY
View originalThis article was summarized and organized by BizCrush based on the original article from IT DAILY. For exact quotations and full details, please refer to the original article.