Intel makes the case for measuring agentic AI infrastructure by work completed, not just core count
In an August 20 opinion piece, Intel’s Karin Eibschitz Segal argues the industry measures AI infrastructure incorrectly. As agentic AI dominates enterprise workloads, legacy benchmarks like single-thread performance, core density, and rack throughput fall short. Intel contends the true metric is reliable, useful agent output per rack within strict latency, cost, and power budgets.
The concurrency problem
A single-agent loop makes performance look simple. Faster cores complete steps faster. But production systems run many agents simultaneously, competing for CPU, memory, I/O, and accelerator resources. Under that load, the bottleneck shifts from computation to scheduling and queueing.
Intel cites a public analysis of 739 anonymised Claude Code conversations, multi-turn coding agent workflows. At a single concurrent request, CPU-side wait time accounted for less than 1% of total latency. At 32 concurrent requests, that rose to 15%, driven mostly by queueing, not computation. A faster core cannot make a queue disappear.
The problem extends to GPUs. For code-execution agents, the same research estimates GPUs perform useful work during only 50 to 60% of wall-clock time. When an agent waits for a database or retrieval system, the GPU waits too. This turns the most expensive component in the rack into an idle cost.
“The most expensive accelerator is the one waiting for the rest of the system.”
Delivered agents per rack
Intel proposes a new metric, “delivered agents per rack”. It is defined as the agent workflows a rack can reliably complete at the required throughput, quality, and latency. All of this within a defined power and cost envelope. The formula multiplies theoretical capacity by routing efficiency, resource balance, and offload efficiency. This should capture the gap between what a rack can theoretically handle and what it actually delivers under real workload conditions.
Intel is addressing that gap through three mechanisms in its Xeon platform. Xeon 6 Flat Memory Mode combines CXL-attached memory and native DDR5 in a single address space. It delivers 96% of all-DDR5 performance at roughly 25% lower memory cost in a joint Intel and SAP evaluation. It supposedly keeps more agent sessions warm at lower cost.
Fixed-function accelerators including Intel QuickAssist Technology, Data Streaming Accelerator, and In-Memory Analytics Accelerator offload infrastructure tasks from general-purpose cores. This process supposedly frees up processing power. Ultimately, it should return that capacity directly to agent execution. And Xeon’s P-core and E-core architecture allows workload placement to be matched to compute profile rather than defaulting everything to the same core type.
Intel’s argument is as much about industry accountability as hardware. The piece explicitly states that every agentic AI performance claim, including Intel’s own, should state:
- the concurrency level
- the latency target
- the losses between theoretical and delivered capacity
Intel has committed to reporting agents per rack at a stated p99 latency target on real agentic traces. Alongside routing efficiency, offload gains, and agents per watt under a defined rack power constraint. Whether competitors follow with equivalent transparency will be worth watching.
