“Inference Will Account for 60% by 2030”...An Open Ecosystem Is Needed
IT DAILY ·
✦ AI Summary
Experts from industry and government said the center of AI is shifting from training to inference.
Lee Do-gyu, head of the Ministry of Science and ICT, projected a 2030 workload mix of 60% inference and 40% training.
The panelists said an open infrastructure ecosystem is needed to break the monopoly of a specific global vendor.
As the center of AI services shifts from large-scale model training to inference for massive commercial services, experts from industry and government said an open ecosystem is needed to break the monopoly of a specific global vendor, citing a 60% inference share by 2030. Their shared view was that building an open infrastructure ecosystem is urgent.
The discussion took place in the main session of AMD's “Advancing AI World Tour & AI DevDay Seoul 2026,” held on the 4th at the Westin Seoul Parnas in Gangnam District, Seoul. The panel discussion, which lasted about 40 minutes, was organized around the themes of building Korea's AI infrastructure and expanding an open ecosystem.
Lee Do-gyu, head of the Ministry of Science and ICT's Office of Information and Communications Policy, Cho Gang-won, CEO of Moreh, and Kim Jang-woo, CEO of MangoBoost, attended the event and exchanged views. In the photo, from left, are Jang Jeong-hee, head of AMD Korea's commercial sales, Kim Jang-woo, CEO of MangoBoost, Cho Gang-won, CEO of Moreh, and Lee Do-gyu, head of the Ministry of Science and ICT's Office of Information and Communications Policy. The photo credit is Kwon Young-seok.
The discussion began with a question about the optimal location for running AI models and data. The company executives said the industry's concerns over infrastructure placement are clearly split in two.
Kim Jang-woo, CEO of MangoBoost, said customer groups are divided into two. He explained that startups aiming for a fast service launch prefer the cloud.
On the other hand, Kim Jang-woo, CEO of MangoBoost, said large companies are reluctant to move their data outside the company and prefer to deploy their entire systems in-house. He added that even if they want to build their own systems, the high price of hardware is a problem, and excessive hardware costs create a TCO barrier.
Cho Gang-won, CEO of Moreh, said there is demand not to rely on external cloud services, especially in sensitive data areas such as defense, security and healthcare. He also pointed to the certainty of demand for sovereign AI infrastructure that does not depend on external clouds such as OpenAI and Anthropic.
The earlier remarks made clear that the choice among security level, input costs and productivity is not simply a technical judgment for IT departments or development teams. He then said a CEO's direct decision is needed and that this should be considered a business trade-off. He stressed that this is a strategic issue.
Lee Do-gyu, the government panelist, pointed to a structural shift in the infrastructure paradigm. He said GPU infrastructure is currently at the center and its use is model training. He then projected a 2030 workload mix of 60% inference and 40% training, saying inference will overtake training.
Lee said that once the inference market takes off, the industry will move away from an all-purpose GPU-only approach. He explained that general-purpose GPUs consume enormous amounts of power and cost, and said it is important to deploy specialized accelerators such as domestic NPUs that offer better power efficiency and cost-effectiveness. He also said that deploying specialized accelerators is a factor that will determine national and corporate competitiveness, and mentioned large-scale demonstrations based on a domestic NPU and the 'K-Perf' in collaboration with AMD.
The panelists agreed that simply staying within NVIDIA's monopoly software platform, CUDA, ecosystem would not solve global supply chain bottlenecks and cost issues. As an alternative, a cooperative model for an open computing system being pursued by the government and AMD gained attention.
Lee said the first clause of the MOU signed with AMD in the U.S. last year was 'linkage and cooperation with domestic NPUs.' He explained that specifying domestic chip linkage in an agreement with a global big tech company was an unusual achievement. He added that AMD's CPU and GPU are being combined with technologies from domestic NPU companies. He also said the government's policy direction is to build a global open ecosystem that is not tied to any specific company.
The event also discussed practical solutions to overcome developers' heavy reliance on CUDA. Cho said that good semiconductors alone are not enough and that software optimization capabilities are needed. He added that Korea needs to accumulate full-stack software capabilities for data centers that can respond to any accelerator without relying on a specific chip.
Kim said AI infrastructure should not be viewed by separating GPU, NPU, memory and storage individually, but rather from an integrated system perspective. He also stressed the need for a testbed that can prove cost-effectiveness with numbers.
Lee then introduced specific support measures to secure trust in domestic infrastructure in both the public and private markets. He pointed to objective indicators and references as key tools for helping developers move to familiar environments, and mentioned 'K-Perf,' a benchmark jointly developed by 15 organizations. He also said a company recently completed performance verification for introducing NPUs worth KRW 10 billion using 'K-Perf.' In addition, he said large-scale data-center-level demonstration projects are planned to be launched in earnest next year, with next year's project direction focused on expanding the verification stage from small chip units, and the goal being to lower market entry barriers.
In their closing remarks, the panelists agreed that the next year will be a decisive golden time that will shape the global AI competitive landscape.
The three speakers each laid out their own standards for AI infrastructure and semiconductor selection. Kim said he would bet on integrated system solutions and proving service cost-effectiveness, while Cho urged data center owners to consider alternative semiconductors beyond NVIDIA.
Cho went on to urge data center owners to test their ability to operate alternative semiconductors as well. Lee said the inference market is entering an explosive phase and that the right decision must be made at the right time from a TCO perspective, adding that the government will continue to support efforts to secure leadership within the open platform of Korea's AI ecosystem.
Source: IT DAILY · Kwon Young-seok
Original: https://www.itdaily.kr/news/articleView.html?idxno=242692
References
This article was produced with the help of an automated content generation algorithm.
Source: IT DAILY
View originalThis article was summarized and organized by BizCrush based on the original article from IT DAILY. For exact quotations and full details, please refer to the original article.