Morai to Unveil 'Token Factory OS' at 'AI Infrastructure Summit 2026'

Moreh said it will participate in the 'AI Infrastructure Summit 2026,' which runs from the 15th to the 17th local time at the Santa Clara Convention Center in the United States, and unveil 'Token Factory OS' at the event. Moreh will also introduce its heterogeneous accelerator-based distributed inference technology. On the first day of the event, Moreh CEO Cho Gang-won will speak during the AI computing session on the topic of 'LLM Serving Framework for Heterogeneous Accelerator Data Centers.'

The 'Token Factory OS' that Moreh will present is an OS that integrates and uses different AI accelerators as if they were a single resource. The heterogeneous accelerator-based distributed inference technology to be shown alongside it is designed for a model in which multiple types of AI accelerators are used together.

Moreh's concept is to shift AIDC competitiveness criteria from a GPU inventory-centered measure to token production efficiency. Moreh CEO Cho Gang-won said the focus of AI data center importance is moving from providing GPU capacity to efficiently producing tokens for API consumption.

Moreh said it sees the number of data centers jointly operating heterogeneous and next-generation accelerators such as NVIDIA and AMD as increasing. It also said the importance of software that optimizes the overall inference path and resource allocation, compared with the performance of individual GPUs, is expanding.

Moreh introduced Token Factory OS as the software suited to this direction. Token Factory OS is AI infrastructure software that is not tied to a specific GPU or inference engine. It operates by integrating multiple accelerators in a data center into a single resource pool, and it builds inference paths based on the model, request characteristics, and computational performance of each accelerator. It also connects computation and data processed across different accelerators.

As a core technology supporting this, Moreh presented the 'MoAI Inference Framework.' The 'MoAI Inference Framework' groups heterogeneous accelerators from NVIDIA, AMD, and Tenstorrent into a single cluster. It also distributes inference tasks according to the characteristics of each accelerator.

Moreh also disclosed a combined inference case using heterogeneous accelerators as a real-world example. In this case, the NVIDIA H100 handled prompt processing and the AMD MI300X handled response generation. Moreh's distributed inference software was used to implement the case, enabling data linkage between the two accelerators.

Moreh said in its presentation that the throughput of the configuration was up to 1.67 times that of a system based on the same vendor. Moreh attributed the improvement in throughput to a processing method that assigns the appropriate accelerator to each computational stage instead of having a single GPU handle the entire inference process, and said this improved overall token production efficiency.

Moreh said it plans to expand its use of heterogeneous accelerators from individual configurations to the entire data center. To that end, it proposed combining technology that operates GPUs and NPUs as a single logical cluster with distributed inference software.

Moreh said it aims to build an AI infrastructure operating system that is not dependent on specific hardware through this approach. CEO Cho Gang-won said AI data centers are evolving from spaces that provide GPUs into token factories that produce tokens, and added that Moreh will support the building of hardware-agnostic token factories through distributed inference technology that uses various accelerators in a single infrastructure form.

Source: TECHWORLD · Kim Seung-gi
Original: https://www.epnc.co.kr/news/articleView.html?idxno=407034

References

• https://www.epnc.co.kr/news/articleView.html?idxno=407034