Scaling in Games: Continued Pre-Training For Embodied Agents In Diverse Virtual Worlds

A study investigates continued pre-training for embodied agents across virtual worlds using 120 training configurations spanning four model sizes from 0.8B to 27B, six data budgets from 50 to 1,600 hours, and five data mixtures. Performance and generalization are assessed using three-level validation losses and gameplay benchmarks from in-corpus to unseen worlds.