AI Agents’ Validation Becomes the Bottleneck as Samsung Focuses on Control and Memory

On the 30th, 'Samsung AI Forum 2026' was held at Samsung Electronics' Seocho headquarters. Samsung Electronics provided an overview photo of the venue. As the use of AI agents in real-world development work expands, issues that had been overshadowed by model performance are coming to the fore, and such concerns were repeatedly raised at 'Samsung AI Forum 2026'.

At the forum, the need for human involvement as agents take on more work was emphasized. The point was that people must decide the scope of tasks to assign, set permitted boundaries, and also verify the accuracy of the results produced by agents. OpenAI introduced an example in which it deployed agents in the design of its own inference semiconductor while applying existing verification tools in parallel.

Samsung Electronics' Device Solutions (DS) Division said it has established a separate system for managing AI user permissions and another separate system for managing AI user usage. Samsung Research was said to be studying long-term memory for agents' past tasks and users, as well as next-generation AI architectures for operation on-device.

Samsung Electronics held the Samsung AI Forum for the 10th time. This year's forum theme was 'The Agentic Shift: From Intelligence to Impact.' The forum was attended by 100 professors and researchers from domestic AI graduate schools and 300 Samsung Electronics employees in the AI field.

Vice Chairman and CEO of Samsung Electronics Jun Young-hyun said AI's status has shifted into a core driver of everyday innovation. He pointed out that as AI spreads, security, trust, and alignment with the business environment are emerging as challenges. He also said it is necessary to examine the social ripple effects of technological convergence beyond technological progress itself.

Under this sense of urgency, Samsung Electronics addressed as key agenda items how to connect agentic AI and physical AI to actual work and how to link agentic AI and physical AI to outcomes. Discussions continued at the forum on how to apply these technologies in real-world settings.

The subtitle was 'Assigning Work Has Become Harder Than Giving Answers.' Richard Ho, vice president in charge of hardware at OpenAI, pointed to the extent to which work is handed over as the criterion that distinguishes chatbots from agents.

Ho explained the difference between chatbots and agents as a structural difference between question-and-answer-centered systems and systems centered on goal delegation and execution. He said chatbots respond sequentially to questions, whereas agents receive a goal, use tools to carry out multi-step tasks, and then verify and adjust the results. He added that the shift is moving from obtaining a single answer to delegating work to agents, with people setting goals and success criteria while agents handle execution between those goals and criteria.

Ho then said enterprise use of agents is increasing. Based on OpenAI's enterprise customer data, he said Codex accounted for 64% of combined output tokens from ChatGPT and Codex. He also said the gap in output tokens per user is widening between the top 10% of companies by agent use and mid-level companies, explaining that the per-user output token gap grew from 2.6 times to more than 8.3 times in June.

Ho stressed, however, that results should matter more than usage itself. He said output tokens alone cannot explain the whole picture, and explained that the benchmark for performance should be the connection to finished deliverables such as changes that pass real tests or analyses that have been fully verified.

At OpenAI's research site, parallel execution by agents has pushed total runtime to levels far beyond a human workday. Based on OpenAI's research organization, by mid-August the total runtime of agents per 8-hour workday for one researcher was estimated at about 3.1 days. The benchmark time unit was 8 hours.

This figure was calculated by summing the parallel execution time of multiple agents. OpenAI explained that as the amount of agent execution increased, the human role became clearer as well.

The human role was structured around selecting which problems to delegate. Humans were also presented as responsible for setting required materials and defining completion conditions. Reviewing results was likewise included among human responsibilities.

Richard Ho, vice president in charge of hardware at OpenAI, said that as agent activity increases, the importance of human judgment grows as well. He said investments are needed in experimental design capabilities alongside agent execution, and that investment is also needed in review capabilities.

Richard Ho gave this keynote on the 30th at 'Samsung AI Forum 2026,' held at Samsung Electronics' Seocho headquarters in Seocho-gu, Seoul. The photo was provided by Samsung Electronics.

OpenAI's first in-house inference semiconductor was 'Jalapeño.' The development process of 'Jalapeño' was presented as an example of this division of roles being applied to actual semiconductor design.

Ho said it took about 9 months from drafting the design language to tape-out for Jalapeño. During the development of Jalapeño, AI generated candidate hardware design changes, and existing EDA tools verified the candidates' PPA, or power, performance, and area. The verification results were then fed back into AI, and the next design proposal was generated on that basis, repeatedly.

Ho identified the key elements of this process as the design problem, repeatable exploration, and concrete testing. He also explained that the method of adopting designs was not a structure in which AI outputs were adopted directly. AI handled candidate exploration, while existing engineering tools handled selection of the results.

This approach led to performance gains in individual design blocks. Compared with a human-optimized design, the matrix operation unit area was reduced by 10%, and compared with a human-optimized design, the SIMD (Single Instruction Multiple Data) functional unit area was reduced by 8%.

The same approach was applied to software as well. The optimization target was the DeepSeek MLA (Multi-head Latent Attention) kernel, and optimization was carried out using agents. As a result, software performance improved by about 287 times compared with the initial implementation that worked correctly, and the initial implementation level was 0.31% of the hardware's theoretical performance ceiling, while the final implementation level rose to 88.94% of the hardware's theoretical performance ceiling.

Ho explained that AI agents' exploration capabilities and the criteria for accepting results must be distinguished. He said the scope of exploration can be broad, but accepting results requires very specific conditions.

He pointed out that each field has its own objective verification standards for accepting results. In mathematics, the criterion is proof; in software, it is regression testing; and in engineering, it is measurement and simulation. He said that, separate from an agent's own judgment of correctness, verification methods are needed.

He then said AI usage and productivity are not necessarily proportional. As an example, he cited METR's 2025 randomized controlled study of open-source developers. In that study, developers felt that using AI improved work speed.

However, actual completion time increased by 19%. Ho stressed that end-to-end results are what matter. He also said the scope of work measurement should include the entire task, including review and rework time.

In enterprise environments, since agents do more than generate answers and directly handle internal data and development tools, authorization controls are added to verification of results within the company. On this point, David Green, AI technology leader for AWS Asia Pacific and Japan, explained that boundaries need to be set around the agent rather than inside it.

He said that rather than relying on prompts to make agents obey rules on their own, access should be limited only to data and tools within the user's permission scope, and system-level access restrictions are necessary. This approach was presented as a move to apply access control at the system level rather than through prompts.

Samsung Electronics' DS Division and AWS Korea co-developed 'Awesome Gateway' for that purpose. 'Awesome Gateway' handles AI usage management functions, and when a user logs in with a company account, the gateway identifies the user, the tools used, and the budget, and performs team-by-team budget settings, team-by-team usage limit settings, allowed model designation, token usage logging, and cost logging.

David Green, AI technology leader for AWS Asia Pacific and Japan, said the number of developers at Samsung Electronics' DS Division using AI development environments based on Amazon Bedrock had surpassed 10,000. Adoption began with a small pilot operation and expanded to actual development teams.

As agents began handling actual systems, the scope of security shifted from AI answers to include AI permissions and actions. Accordingly, a principle blocking access to confidential documents was presented, along with the idea that documents not opened to users should also not be accessible to agents.

On the security operations side, limits on tool use were also presented. Agents must use only authorized tools, and in high-risk operational procedures, final human approval should be possible.

At the same time, Samsung Research raised memory as the next challenge. Long-running task agents face an additional challenge of how to remember past tasks and user information, and long-interaction agents must continue accepting new information while remembering past experiences.

To address these challenges, Kim Yoon-hyung, executive vice president at Samsung Research, introduced a next-generation Agentic Intelligence architecture. The purpose of the introduction is to expand agent capabilities and areas of application.

Behind this is the fact that mainstream AI today is based on transformers. Transformer-based AI has a limitation in that computation and memory burdens increase as context length grows, and this burden works against long-interaction agents.

Kim outlined research directions for model architectures to handle long context and implement Continual Learning. He introduced an existing approach that combines attention and Linear Attention, and also introduced an approach that compresses long context into smaller latent representations. The purpose of the research, he said, is to implement Continual Learning that preserves past information while accepting new information.

Samsung Electronics is reviewing extending this issue to edge devices. The target devices for agents include user-near devices such as smartphones and home appliances. Running agents on edge devices requires resolving memory and power constraints. In addition, Samsung identified minimizing repeated transmission of personal data to servers and shortening response times as challenges for edge devices.

In this process, Kim said it is not unrealistic to envision smartphones and other edge devices implementing functions at the level of today's high-performance AI models within the next few years. He then suggested that rather than simply shrinking or porting cloud models, an 'on-device native' architecture that incorporates device memory and compute characteristics from the outset is needed.

In the afternoon, Samsung Electronics introduced 'Personalizing AI' during a DX Division session. 'Personalizing AI' aims to simultaneously preserve prior knowledge, learn new information, and revise memory.

Samsung Electronics also introduced 'Self-Evolving AI.' 'Self-Evolving AI' is a method that improves performance by analyzing real-world usage experiences and inconveniences after product release.

Samsung Electronics identified efficient AI, physical AI, and Continual Learning as key research themes in the DX Division. In physical AI, there is a verification problem, and this becomes more difficult in connection with AI's robot control.

Professor Ranjay Krishna of the University of Washington in the U.S. pointed out that there are limits to applying language models' reasoning methods directly to the physical world. He proposed a visual reasoning approach for space and motion using sketches instead of language-only reasoning, and said the approach aims to improve robotics AI performance in small models.

In a panel discussion, Professor Krishna introduced an example of 'environment hacking,' saying unexpected behavior can occur during simulation-based robot training. 'Environment hacking' refers to cases in which AI exploits loopholes in a simulator to obtain high rewards rather than solving the intended problem.

As an example, he cited a robot given the task of moving an object across a wall. The wall in the simulation did not have the same depth as in reality, and instead of transporting it normally, the robot threw the object hard enough to pass through the wall. This method met the given evaluation criteria but could not be used in the real world.

Based on this, Professor Krishna proposed a layered structure divided by role rather than assigning all physical AI decisions to one massive model. In this structure, the upper model handles planning and task decomposition, while smaller, faster lower-level models handle responses to changes in the real environment. The layered structure uses the upper model's reasoning capabilities while separating the execution layer for fast on-site robot response.

Source: IT DAILY · Kim Byung-joo
Original: https://www.itdaily.kr/news/articleView.html?idxno=241922

References

• https://www.itdaily.kr/news/articleView.html?idxno=241922