Embodied Agent Arena
Are frontier VLM agents ready to be robot generalists?
An empirical study across perception, reasoning, and action.
1 HKUST (Guangzhou)2 The Chinese University of Hong Kong3 Knowin AI
01 The arena
From seeing a scene
to completing a task.
One arena tests what agents can perceive, reason about, and physically achieve across five robotic capabilities.
1,000evaluation cases
32 + 1sources + GeoProbe
5capability domains
7frontier VLM agents
Introducing GeoProbe
Separate object motion
from camera motion.
168 new cases isolate camera motion, object motion, depth, and scale.
02 A deep dive of Astra
Where Astra leads. Where it falls short.
Compare all seven agents
| Agent | Geometry | Spatial reasoning | Affordance | Planning | Manipulation | |||
|---|---|---|---|---|---|---|---|---|
| Rotation ↓degrees | Translation ↓cm | Pass@1 ↑% | AbsRel ↓relative error | Mask IoU ↑overlap | Contact ↑% | Pass@1 ↑% | Task success ↑% | |
Bold marks the best score; arrows show the better direction. ± is the standard deviation across runs. Manipulation uses the baseline setting.
03 From scores to behavior