OpenAI Co-Founder: Gap in Perceived AI Capability Widens as User Experience Shapes Judgments
IT DAILY ·
✦ AI Summary
Andrej Karpathy said AI performance evaluations vary because users differ in model access levels and fields of use.
He divided people into four stages, from those who barely use LLMs to free chatbot users, professionals using cutting-edge LLMs, and those with access to internal state-of-the-art systems.
Karpathy said the sense of AI progress differs by group, with most experts feeling only a slight improvement in work efficiency while some feel it as a vertical surge.
IT Daily examined why user assessments of AI performance diverge, diagnosing a widening perception gap among AI users.
Andrej Karpathy, known as an OpenAI co-founder, said in a recent post on SNS X that the gap in people's perceptions of LLM capabilities is widening. He is currently in charge of AI research at Anthropic.
Karpathy pointed to differences in experience by model used and by field of use as the reason AI evaluations are becoming more divided. He said the scale of problems users assign to AI varies, the fields in which they use it differ, and their level of access to models also differs.
He explained that the more complex the task, the more clearly AI capabilities are felt, and that AI's strengths are especially evident in math and coding. He also said whether users have access to free services, paid services, or internal systems affects that perception.
Karpathy said the effect of questions that can be answered in one paragraph on testing system capability is minimal, likening the current situation to a funnel narrowing rapidly. He then divided people into four stages depending on their level of LLM use and access rights.
He cited as Stage 1 people who have barely encountered LLMs, estimating that population at about 6 billion. Stage 2 consists of people who use free chatbots occasionally, a group he estimated at about 1 billion to 2 billion. Karpathy said these occasional users of free chatbots see AI as a tool for search and writing assistance.
Stage 3, he said, consists of people who use state-of-the-art LLMs at work in math and coding, and he estimated the size of that group at about 20 million. He said this group uses AI agents, which dramatically reduce the time needed for projects that previously took weeks or months. Stage 4 consists of people who can access internal state-of-the-art systems, and he said that population is about 5,000.
He explained that the sense of AI progress is not the same across groups. He said most experts feel only a slight improvement in work efficiency. By contrast, he said some very small number of experts experience it as a dizzying vertical ascent on the growth curve. He added that these phenomena are all unfolding at the same time.
Earlier, in April, Karpathy posted a message saying that AI evaluations differ depending on user experience. He pointed out that there is a difference in how AI progress is felt between ordinary users of free chatbots and professionals who use the latest agentic models in technical work.
An industry source said AI usage has changed dramatically compared with the early days after ChatGPT's release. The source said that what matters now is what AI can do and what outcomes can be achieved.
Source: IT DAILY · Yang Seung-gap
Original: https://www.itdaily.kr/news/articleView.html?idxno=242231
References
This article was produced with the help of an automated content generation algorithm.
Source: IT DAILY
View originalThis article was summarized and organized by BizCrush based on the original article from IT DAILY. For exact quotations and full details, please refer to the original article.