"Cloud Alone Can't Handle 86% of Inference Demand": On-Device and Edge Strategies Gain Ground

South Korea's AI infrastructure has been shown to be running into physical limits, including data center power shortages and a lack of available floor space. Access Partnership published the report, "From Cloud to Edge: The Value of Distributed AI in Korea," commissioned by Qualcomm.

The report projected that domestic AI inference computing demand will increase 30-fold by 2030 from current levels. It said AI inference demand is expected to surge in the years ahead amid the constraints facing South Korea's AI infrastructure.

The report found that meeting this demand using the existing cloud model alone would require at least a 5-fold expansion in domestic data center capacity. It also said that a centralized cloud alone would not be able to handle 86% of AI inference computing demand in 2030.

It further projected that computing processing capacity would fall short by 86% if current investment and expansion trends continue. Access Partnership said in the report that a cloud-centric response would face an extreme supply cliff.

It has been pointed out that limits on power connections for data centers in the greater Seoul area, as well as difficulty securing sites for such facilities there, make it difficult to meet demand by expanding central servers alone. In addition, there is a computing bottleneck, and analysis suggests that rising demand cannot be handled by the existing approach alone.

Against this backdrop, the focus of AI is shifting from text generation to "Physical AI." The applications of "Physical AI" include factory equipment, robots, and mobility, and its defining feature is real-time control. As a result, the growing impact of computing bottlenecks could lead to even more critical failures in "Physical AI."

In particular, industrial robotic arms and autonomous inspection systems at high-risk worksites require real-time deterministic control of less than 10 ms. The reason such less-than-10 ms control is needed is worker safety and precision processes. However, because of the round-trip communication latency structure of centralized cloud systems, the centralized cloud model cannot meet the less-than-10 ms standard.

Security issues and technological sovereignty issues also act as constraints on a cloud-only model. Core process data from industrial sites could be routed to external clouds, and such routing would generate data exhaust along the network path. That data exhaust could create risks of exposure for factory communications architecture and key know-how, further increasing the limitations of a cloud-only model.

The report noted that, to protect core secrets and meet sovereign regulatory requirements, an air-gapped environment physically isolated from external networks is necessary. It also said that local infrastructure processing that operates independently in an Air-gapped environment is essential.

Based on these premises, the report proposed that large-scale model training be handled by the central cloud. Real-time inference and sensitive data processing would be distributed to on-site edge servers and devices, creating a hybrid architecture that combines the central cloud with distributed on-site processing.

Against this backdrop, the AI infrastructure paradigm is shifting from competition to expand central data centers to distributed computing at the point of use. Accordingly, securing an infrastructure system that integrates cloud and edge has emerged as a challenge for South Korean industry, and it is defined as an essential survival task.

Source: IT DAILY · Kwon Young-seok
Original: https://www.itdaily.kr/news/articleView.html?idxno=241999

References

• https://www.itdaily.kr/news/articleView.html?idxno=241999