Arm to Redefine CPU Performance Metrics for AI Agents, Doubling Rack-Level Throughput

Arm has introduced a new standard for measuring CPU performance in agentic AI environments. On the 15th, Arm unveiled a CPU performance evaluation framework for agentic AI. The framework was developed with support from Fujitsu, Meta, and OpenAI.

Using this standard, Arm also released verification results comparing its Arm AGI CPU with AMD EPYC Turin. Signal65 published performance verification results comparing the Arm AGI CPU and AMD EPYC Turin. Arm said its self-designed Arm AGI CPU delivered 1.99 times the average rack-level workload throughput of AMD server CPUs. The performance result was based on independent verification.

The new framework was designed to assess the processing performance of completing real AI agent tasks end to end, rather than partial performance metrics. The methodology avoids evaluations centered on individual compute performance and also avoids assessments focused on the speed of specific components. Instead, it focuses on evaluating CPU processing efficiency and completion capability for real AI agent tasks.

The framework's core principles are representative agentic AI use cases, repeatable CPU performance measurement, and end-to-end outcome validation. Arm unveiled a new CPU performance evaluation methodology for agentic AI.

AI agents are evolving beyond merely responding to user requests, toward maintaining context, calling external tools and services, searching data, and evaluating task results. As a result, the CPU's role is expanding to include agent execution, task scheduling, data preparation, data movement, and cloud-native service handling. Arm cited data introduced by a16z to explain this trend.

As agent usage expands, consumption is rising as well. Agent token usage at OpenRouter has increased roughly 14-fold since February 2026. As this spread continues, demand for high concurrency and sustained task processing is growing across computing, memory, storage, and input/output (I/O).

With these changes, in AI data centers, the role of the CPU is emerging as an important issue alongside the performance of accelerators such as GPUs. Because the CPU handles task scheduling and data supply, insufficient performance in these functions makes it difficult to fully utilize high-performance accelerators. In the end, AI data centers are placing importance not only on accelerator performance but also on the CPU operating capabilities that support them.

As agent autonomy rises, security is becoming more important, and infrastructure control is also becoming more important. In this trend, the host CPU handles agent execution, while DPU and other infrastructure processors take on roles supporting networking, storage, policy enforcement, telemetry, and isolation.

Arm supports an architecture spanning CPU and DPU domains. Arm is also pushing ahead with the spread of open rack-scale AI infrastructure, contributing to the Open Compute Project (OCP) with AGI CPU reference servers, and its Arm AGI CPU has been adopted in Meta's Yosemite YV5 rack-scale design.

In this context, Signal65 evaluated seven agentic AI workloads on the Arm AGI CPU and AMD EPYC Turin, including tooling, orchestration, inference, and planning. For comparison, it applied the same benchmark scripts to both platforms, and for performance estimation it used rack-level performance modeling based on measured server-level performance.

In the verification, the Arm AGI CPU was compared with the AMD EPYC 9755 and AMD EPYC 9965 to measure rack-level modeled performance and sandbox creation latency. The results showed that the Arm AGI CPU achieved 1.99 times the average rack-level modeled performance across the 7 workloads versus the AMD EPYC 9755. Arm said the result was helped by high compute density and power efficiency.

By workload, performance was reported at 1.7 times for tooling, 2 times for agentic workflows and orchestration, and 2.2 times for inference and planning. In addition, the Arm AGI CPU showed up to a 45% reduction in sandbox creation latency compared with the AMD EPYC 9965.

A sandbox is an isolated execution environment used when agents perform tasks. Shorter sandbox creation times reduce the overhead of preparing agent tasks.

However, these figures are rack-level modeling results calculated by Signal65 based on specific workloads and system configurations. Actual data center performance may vary depending on server configuration, software, power and cooling conditions, and workload characteristics.

Arm said traditional CPU performance metrics alone are insufficient to fully assess efficiency in agentic AI infrastructure, and proposed a new benchmark methodology. Arm concluded that traditional CPU metrics alone make it difficult to gauge the efficiency of agentic AI infrastructure.

In agentic AI environments, the response speed of individual tasks is important, and so is the ability to maintain responsiveness while handling multiple tasks simultaneously. Throughput must also remain stable even at high utilization, and maintaining that throughput stability is directly tied to increasing the number of tasks actually completed.

At the same time, data center operations face constraints in power, heat, and physical space. Accordingly, evaluation standards are shifting so that, beyond CPU performance itself, the amount of work that can be processed continuously within limited rack space and power is becoming more important. The Arm AGI CPU is designed to address these needs with high core density, dedicated memory per core, and high-bandwidth I/O.

Arm proposed a method that uses component-level measurement to isolate CPU performance from other variables. It also proposed end-to-end evaluation to verify whether performance improvements translate into actual task completion results.

Arm stressed that performance evaluation methodologies must also evolve in line with changes in agent architectures and software. Arm explained that if the industry establishes common measurement standards, it will make system comparisons and bottleneck analysis easier and also help optimize next-generation AI infrastructure.

Fujitsu said that transparency, reproducibility, and representative performance measurement were important in the development of FUJITSU-MONAKA. Toshio Yoshida, a senior vice president and head of the Advanced Computing Development Division at Fujitsu, said he expects Arm's initiative to promote industry collaboration and support the optimization of next-generation AI infrastructure.

Abhishek Danotia, a software engineer at Meta, said that as agentic AI enters real-world operations, it is necessary to clearly understand how diverse workloads use the underlying infrastructure. He said Arm's efforts to establish a consistent comparison standard will help define the direction of future system development.

Colin Jarvis, global forward deployed engineering lead at OpenAI, described the evolution of AI systems as moving toward completing complex tasks based on inference, tool use, and interaction with software environments, and said the importance of understanding the performance of infrastructure that supports agentic workflows such as orchestration and tool execution has grown accordingly. He added that industry collaboration is needed for representative benchmarking.

Source: TECHWORLD · Park Gyu-chan
Original: https://www.epnc.co.kr/news/articleView.html?idxno=408230

References

• https://www.epnc.co.kr/news/articleView.html?idxno=408230