RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild

Researchers introduced RoboColosseum, a distributed evaluation framework for testing general-purpose robot policies across diverse robots, tasks, and environments. The framework evaluated five vision-language-action policies and three AI agents across four robot types in blinded pairwise trials and fine-tuning tasks, finding that frontier agents rival specialized policies but all systems struggle with open-world tasks. Rankings showed weak agreement across robot-specific task suites.