Kakao Cuts AI Training Costs and Makes Models Lighter

According to AI TIMES, Kakao unveiled at COLM 2026 on the 8th a method for improving the training efficiency of large mixture-of-experts models, as well as a lightweighting technology that makes models lighter while also improving performance. In particular, it said its byte-based BBT architecture reduced model size by up to 40.4% while boosting test performance by up to 6.6%. The key takeaway from the announcement is that it reduces the costly search for training configurations in large-model training and presents a methodology that can be applied stably in actual training processes. Kakao explained that it improved pretraining efficiency by refining a way to scale up the optimal settings found in small models to large models in a way suited to mixture-of-experts architectures. In another study, it presented an approach that uses bytes to reduce storage burden while preserving the advantages of existing compression methods, making models lighter without performance degradation. The company said this research could broaden the applicability of real-world services, including on-device and multilingual environments, and is expected to further reduce the development and operational burden of large language models going forward.