DeepBrain AI Develops XR-Ready 'AI Human NPC' SDK for Real-Time Conversations, Customized Responses
TECHWORLD ·
✦ AI Summary
DeepBrain AI said on the 15th that it had completed development of 'Persona XR AI Human SDK,' a 3D AI human NPC interface development tool for XR-based metaverse services.
The tool can converse with users in real time and respond to context, using TTS, a phoneme-level lip-sync engine, and LLM-based speech generation technology.
It is provided as a Unity-based plugin, and the company plans to roll it out sequentially to XR content production sectors including exhibitions, museums, education, tourism, and public information.
DeepBrain AI has unveiled a 3D AI human NPC development tool for XR spaces. On the 15th, the company said it had completed development of 'Persona XR AI Human SDK,' an interface development tool for 3D AI human NPCs for XR-based metaverse services.
The tool can hold real-time conversations with users and respond according to the situation. 'Persona XR AI Human SDK' incorporates DeepBrain AI's high-speed text-to-speech (TTS), phoneme-level lip-sync engine, and large language model (LLM)-based speech generation technology.
It works by analyzing the user's question before the AI human NPC responds in real time. This enables real-time question-and-answer interactions.
The tool is offered as a plugin based on Unity, a game and XR content development engine. Developers can apply AI human NPCs with voice chat functions to existing XR content, and can configure gender, appearance, voice, gestures, and persona.
The technology is designed so that the AI model for generating responses can be changed according to the purpose of the content. It supports mobile XR devices based on Windows, macOS, and Android.
In NPC implementation, the system uses the role and personality set by the developer to render the NPC's facial expressions, gestures, and voice. It uses LLM for response content and speaking style, and employs a scenario-based learning standard model.
If DeepBrain AI's emotion-recognition technology is linked, it can analyze the user's question, facial expressions, and voice. Based on the analysis results, the technology is designed to reflect emotionally appropriate expressions in the response and to generate speech suited to the configured NPC persona even as the conversation changes.
DeepBrain AI also conducted performance verification of its video synthesis technology for AI human implementation. Its in-house synthesis model became subject to evaluation by a KOLAS-accredited testing laboratory in July last year, with assessment items including video quality, lip-sync accuracy, and pronunciation accuracy.
The company plans to roll out the SDK sequentially for XR content production in fields such as exhibitions, museums, education, tourism, and public information. It also plans to create NPC and user interface (UI) components and sector-specific templates. In addition, it plans to provide NPC and UI components through an integrated platform and support user experience (UX) evaluation services for XR content developers.
The SDK was developed as part of a cultural technology R&D project supported by the Ministry of Culture, Sports and Tourism and the Korea Creative Content Agency. The project is titled 'Development of Multimodal UX Evaluation Platform Technology for Optimizing Responsive Content in XR Spaces.' It is led by the Industry-Academic Cooperation Foundation of Yonsei University, with Hanyang University, Hankuk University of Foreign Studies, DeepBrain AI, Nine Five, and Stoic Entertainment participating as co-research and development institutions.
The goal of the project is to evaluate the user experience of XR content using multimodal data such as biosignals and behavior. It also aims to build an integrated platform that optimizes NPC and UI based on the evaluation results. A final demonstration and evaluation is scheduled for the second half of this year.
DeepBrain AI CEO Jang Se-young said the significance of the achievement lies in completing a development tool that can be used immediately in XR content production sites for AI human technology. He added that the company will continue R&D to turn AI human into a practical solution for diverse fields including exhibitions, education, and public services.
Source: TECHWORLD · Kim Seung-gi
Original: https://www.epnc.co.kr/news/articleView.html?idxno=406960
References
This article was produced with the help of an automated content generation algorithm.
Source: TECHWORLD
View originalThis article was summarized and organized by BizCrush based on the original article from TECHWORLD. For exact quotations and full details, please refer to the original article.