WiseStone to Train ‘AI Test Architect’ Experts for LLM and Generative AI Verification
IT DAILY ·
✦ AI Summary
WiseStone held the first class of the 'AI Test Architect Lab' to help ensure the quality of generative AI- and LLM-based systems.
The course aims to train AI Test Architects, and participants carried out the full verification process from target analysis to test execution and results reporting.
The program was conducted based on the AI Test Framework and LLM/RAG Test Playbook, and WiseStone plans to use it as the basis for finalizing the draft of the AI Test Framework.
According to IT Daily, AI SW quality testing and certification company WiseStone held the first class of the 'AI Test Architect Lab.' The program was operated to help ensure the quality of generative AI- and LLM-based systems.
The course aims to train 'AI Test Architects.' An AI Test Architect analyzes the target system in the early stages of an AI verification project, identifies quality risks, and develops a testing strategy.
Recently, the adoption of enterprise-centered LLM and RAG (retrieval-augmented generation) technologies has surged. As a result, the need to manage the unique quality risks of AI systems has also been growing.
Examples of AI-specific quality risks include Hallucination, evidence mismatch, data security, and domain-specific errors. The need for specialized architects is emerging to preemptively control these risks.
WiseStone's operating program, the first class of the 'AI Test Architect Lab,' was structured not simply to deliver theory and train testers. It was run as a hybrid hands-on training program in which participants directly carried out the full verification process on actual AI conversational systems and other targets. Participants were responsible for the full scope of verification, from target analysis and risk analysis to test planning, test design, test execution, and results reporting, and directly handled the entire verification process.
The training was conducted based on WiseStone's internally built 'AI Test Framework,' 'LLM/RAG Test Playbook,' and standard verification document templates, and participants designed and validated sophisticated test cases (TC) and rubrics (evaluation criteria). In this process, test design and verification incorporated integrated review items covering general SW quality characteristics such as functionality and security, as well as AI-specific quality factors such as accuracy, transparency, and robustness, along with domain and operational risks in areas such as finance and regulation.
WiseStone said it plans to use the practical feedback and deliverables obtained from this program to push ahead with finalizing the draft of the 'AI Test Framework.' The deadline for finalizing the draft of the 'AI Test Framework' is by the end of August, and the final draft is intended to be immediately applicable in the field. WiseStone also plans to develop a more advanced package of AI verification methodologies.
Lee Young-seok, CEO of WiseStone, said that in the period of upheaval brought on by AX (AI transformation), securing the reliability and safety of AI systems is a key factor in determining whether a company succeeds or fails. He added that the company plans to continue cultivating specialized Advisory and AI Test Architects based on a systematic framework and verification playbook, and expressed its commitment to leading the improvement of domestic AI SW quality.
Source: IT DAILY · Kim Ho
Original: https://www.itdaily.kr/news/articleView.html?idxno=241111
References
This article was produced with the help of an automated content generation algorithm.
Source: IT DAILY
View originalThis article was summarized and organized by BizCrush based on the original article from IT DAILY. For exact quotations and full details, please refer to the original article.