Benchmarks

Simulation

LIBERO-Plus

Metric
Zero-shot success on 10,030 perturbed LIBERO instances, mean of 7 perturbation dimensions
Robot
Franka Panda
Simulator
MuJoCo (robosuite)
Tasks
10,030 instances
  1. 1InternW0-Δ93.2%, self-reported by the authors, not a leaderboard entry
  2. 2RoboHarness92.4%, self-reported by the authors, not a leaderboard entry
  3. 3Qwen-RobotManip-Context92.0%, self-reported by the authors, not a leaderboard entry

LIBERO-PRO

Metric
Mean success over 20 cells (4 suites × 5 perturbations: original, object, position, semantic, task)
Robot
Franka Panda
Simulator
MuJoCo (robosuite)
Tasks
20 perturbation cells
  1. 1QuoVLA69.8%, self-reported by the authors, not a leaderboard entry
  2. 2π0.562.0%, self-reported by the authors, not a leaderboard entry
  3. 3SimVLA58.6%, self-reported by the authors, not a leaderboard entry

CALVIN ABC→D

Near saturation
Metric
Average completed chain length (0–5) over 1,000 five-step instruction chains, ABC→D split
Robot
Franka Panda
Simulator
PyBullet
Tasks
34 tasks, 4 environments
  1. 1MMaDA-VLA4.78 / 5, self-reported by the authors, not a leaderboard entry
  2. 2Xiaomi-Robotics-04.75 / 5, self-reported by the authors, not a leaderboard entry
  3. 3VITA4.73 / 5, second-hand number, not confirmed at the source

LIBERO

Saturated
Metric
Mean success over 4 suites (Spatial, Object, Goal, Long), 50 rollouts per task
Robot
Franka Panda
Simulator
MuJoCo (robosuite)
Tasks
40 tasks (4 suites)
  1. 1LaST-R199.9%, self-reported by the authors, not a leaderboard entry
  2. 2QuoVLA99.6%, self-reported by the authors, not a leaderboard entry
  3. 3ABot-M0.599.4%, self-reported by the authors, not a leaderboard entry

SimplerEnv WidowX

Saturated
Metric
Visual-matching success, mean of 4 Bridge tasks
Robot
WidowX 250 (Bridge)
Simulator
SAPIEN (ManiSkill2)
Tasks
4 tasks
  1. 1StARe-VLA98.0%, self-reported by the authors, not a leaderboard entry
  2. 2CORAL97.9%, self-reported by the authors, not a leaderboard entry
  3. 3X-VLA95.8%, self-reported by the authors, not a leaderboard entry

Real robot

Low-cost arm

SO-101 Bench

Metric
Mean success over 15 task-family × object-split cells (916 episodes, 95% bootstrap CI)
Robot
SO-101
Simulator
Isaac Lab (digital twin)
Tasks
5 task families, 15 cells
  1. 1GPT-6.1 Sol (PyRUA) + Galaxea G0.583.2%, self-reported by the authors, not a leaderboard entry
  2. 2DeepSeek V4.1 Flash (PyRUA) + Galaxea G0.566.8%, self-reported by the authors, not a leaderboard entry
  3. 3InternW0-Δ61.4%, self-reported by the authors, not a leaderboard entry