Professor Lee Byung-young Says Prompt Injection Is Unavoidable, Calls for Dual LLM Architecture to Reduce Attack Risk
TECHWORLD ·
✦ AI Summary
At the AI Risk Conference on the 7th, Lee Byung-young, a professor in the Department of Electrical and Computer Engineering at Seoul National University, gave a presentation on "Protecting Agents Using Dual LLM Technology."
Lee explained that prompt injection has become more important to address because AI agents have external data access and real-world action capabilities.
He said guardrails alone have limits and that external data and user commands should be separated through trust-based isolation and a "dual LLM" architecture.
On the 7th, the AI Risk Conference was held, and Lee Byung-young, a professor in the Department of Electrical and Computer Engineering at Seoul National University, gave a presentation on "Protecting Agents Using Dual LLM Technology." Lee emphasized that as the use of AI agents grows, the importance of responding to prompt injection is also increasing.
Lee explained that AI agents are characterized by their ability to obtain external data and carry out real actions according to user requests. This differs from conventional LLMs in that they can connect to external data and perform execution functions. Examples of external data included web pages, email, and local files. He said that, because of this structure, malicious code can also be introduced along with external data, which may cause behavior that differs from the user's intent and ultimately increase the risks faced by AI agent users.
Lee said the prompt injection problem in AI agents has limitations when addressed only with guardrails. As an alternative, he said that trust-based isolation and the introduction of a "dual LLM" architecture are needed, and explained that the dual LLM structure can minimize attack possibilities.
He cited as real-world attack examples the manipulation of external data such as an attacker's web pages and email, as well as attack chaining, and pointed to web agents, GitHub Copilot, Microsoft 365 Copilot, and Perplexity Comet as cases. He noted that one problem with AI agents is the mixed processing of user prompts and LLM context containing data created by attackers, and explained that the reason LLMs are vulnerable lies in that mixed-processing structure.
He said a response to this vulnerability is needed and that external data and user commands must be structurally separated. He then explained two ways to address prompt injection in AI agents.
The two approaches are guardrails and dual LLM, and he said both have advantages and disadvantages. He argued that applying the dual LLM approach instead of guardrails would make it possible to build a safer environment.
Guardrails determine whether to block content by judging whether it aligns with user intent based on input and output content. However, this approach has the limitation of making it difficult to respond to new attack patterns that differ from existing ones, and the speaker questioned whether guardrails can adequately handle attack patterns.
As an alternative, dual LLM was presented as a comparison model. Dual LLM divides LLMs into trusted and untrusted categories, and does not grant resource permissions to the untrusted LLM. The untrusted LLM is placed in a sandbox, and as a result of that sandboxing, access to system resources is restricted.
In this structure, even if external data contains malicious commands, they are blocked so that they cannot be carried out on the actual system. Accordingly, dual LLM is effective in protecting against situations such as prompt injection. In addition, only the necessary results or references are passed to the trusted LLM, while the original text of data that may contain malicious commands is not transmitted.
The speaker explained that the key to dual LLM lies in preventing attacker data from being exposed. He then said that several companies are researching dual LLM architectures, citing Google CaMeL and Microsoft FIDES as examples.
The speaker started from the premise that prompt injection is an unavoidable problem. He also said that the basis for LLM operation is external data, and that an LLM without external data is meaningless, explaining that the core challenge lies in how external data is managed. He then emphasized the need for a safe AI agent environment based on dual LLM.
Source: TECHWORLD · Kim Hye-jin
Original: https://www.epnc.co.kr/news/articleView.html?idxno=406631
References
This article was produced with the help of an automated content generation algorithm.
Source: TECHWORLD
View originalThis article was summarized and organized by BizCrush based on the original article from TECHWORLD. For exact quotations and full details, please refer to the original article.