“Tapping Idle GPUs”: ETRI Unveils 'OpenK-Cloud' to Break Through AIDC Bottlenecks
IT DAILY ·
✦ AI Summary
ETRI has unveiled 'OpenK-Cloud' to address AIDC bottlenecks.
The platform integrates heterogeneous AI semiconductors in a single cloud and applies uninterrupted rolling updates and computation separation technology.
ETRI plans to intensify optimization for LLM services and operator operations starting next year, and to deploy domestic NPU chips in real-inference services such as MRI-based disease prediction.
BizCrushWant to follow Korean talks and live streams in real time?Try CapCrush on the web
The Electronics and Telecommunications Research Institute (ETRI) has unveiled 'OpenK-Cloud' to ease AIDC bottlenecks. ETRI disclosed platform technology that efficiently integrates heterogeneous AI semiconductors in a single cloud environment and enables uninterrupted services, and on the 10th it held 'openKcloud DevDay 2026' at the Korea Science and Technology Center.
The keynote presentation was delivered by Choi Hyun-hwa, ETRI's technology chief. Choi pointed out the structural limitations of AIDC and outlined the development progress, key achievements, and blueprint for 'OpenK-Cloud' to overcome those constraints.
Choi said that a key challenge for AI infrastructure these days, after securing accelerators, is the ability to manage heterogeneous chips in a single cloud environment and operate them in connection with services. He added that cloud platform technology is essential to support smooth real-world service operations in enterprise environments.
The photo shows Choi Hyun-hwa, ETRI's technology chief, speaking at the event. It was taken by reporter Kwon Young-seok. Choi identified resource inefficiency as the biggest weak point in AIDC infrastructure operations. Amid a continuing shortage of accelerators, he said the data center GPU allocation rate is about 75%, but the actual GPU utilization rate remains at 50% to 60%.
Choi said the reason resources are used so little in AIDC operations, even when they are sufficiently deployed, lies in changes to AI service architecture. As AI services evolve into agentic systems, he said demand is rising for processing preprocessing logic beyond model serving. Accordingly, he explained that CPU bottlenecks and scheduling constraints are emerging as major causes of wasted resources.
He said each accelerator requires at least 8 to 11 CPU cores. He also explained that even if there are remaining GPU resources in a server, GPU isolation occurs when there are not enough matching CPUs. He pointed out that as the pairing between GPU and CPU breaks down, it becomes difficult to use leftover resources in actual services.
He also explained that in high-density GPU architectures, 8 GPUs are bundled and operated per server. However, since some service allocation cases involve 4 or 6 GPUs, he said it is difficult to efficiently split and use the 1 or 2 GPUs left over after allocating 4 or 6. He explained that this server-level GPU bundling structure is a factor that constrains the use of leftover GPUs.
Choi said there are also limits to splitting and using leftover GPUs by combining them with resources from other servers. He said this is because latency increases by about 20% each time a network switch is traversed once. He explained that the difficulty of splitting GPU usage and the physical limits of network latency are factors that hinder resource utilization.
Global big tech companies Google, AWS, and Microsoft are pushing ahead with a shift to agentic enterprises. To address infrastructure imbalances, these companies are pursuing a combination of Kubernetes orchestration and the introduction of their own chips.
ETRI developed the Kubernetes-based platform 'OpenK-Cloud' as a means to solve the same challenge. Along with it, ETRI also disclosed demonstration data.
'OpenK-Cloud' automatically detects accelerators installed in physical servers. The accelerators subject to automatic detection include FuriosaAI, Rebellions, and Tenstorrent. In addition, 'OpenK-Cloud' unattendedly provisions dedicated drivers and SDKs.
The core technology of 'OpenK-Cloud' is 'uninterrupted rolling updates.' This technology works by applying changes sequentially without service interruption. When upgrading chip drivers, it automatically migrates running tasks to another server (node), safely drains that server, and then reboots and restores it. ETRI said this has enabled a management system for driver upgrades without service interruption.
The computation separation technology was applied so that when local server AI accelerator resources are insufficient, tasks can be transferred to remote server accelerators, and it was designed so developers do not need to modify code directly. The technology focuses on limiting performance degradation by analyzing network response speed and cache status in real time to select the optimal server. In actual tests on Rebellions' 'AtomPlus' and Tenstorrent's 'P150,' it suppressed the increase in network latency to around 10%, demonstrating the effectiveness of remote distributed acceleration.
The chief said the focus so far has been on proof-of-concept, or PoC, for each function. He added that starting next year, the plan is to intensify optimization technology for LLM services and operator operations overall, and to move beyond simple benchmarking. He also said the plan is to deploy domestic NPU chips in demanding medical and bio real-inference services such as MRI-based disease prediction, while also pursuing commercial references.
Source: IT DAILY · Kwon Young-seok
Original: https://www.itdaily.kr/news/articleView.html?idxno=242814
References
This article was produced with the help of an automated content generation algorithm.
Source: IT DAILY
View originalThis article was summarized and organized by BizCrush based on the original article from IT DAILY. For exact quotations and full details, please refer to the original article.
Keep reading
BizCrush