Kavli Affiliate: Cheng Peng
| First 5 Authors: Jiahan Zhang, Jiahan Zhang, , ,
| Summary:
Generative world models (WMs) can now simulate worlds with striking visual
realism, which naturally raises the question of whether they can endow embodied
agents with predictive perception for decision making. Progress on this
question has been limited by fragmented evaluation: most existing benchmarks
adopt open-loop protocols that emphasize visual quality in isolation, leaving
the core issue of embodied utility unresolved, i.e., do WMs actually help
agents succeed at embodied tasks? To address this gap, we introduce
World-in-World, the first open platform that benchmarks WMs in a closed-loop
world that mirrors real agent-environment interactions. World-in-World provides
a unified online planning strategy and a standardized action API, enabling
heterogeneous WMs for decision making. We curate four closed-loop environments
that rigorously evaluate diverse WMs, prioritize task success as the primary
metric, and move beyond the common focus on visual quality; we also present the
first data scaling law for world models in embodied settings. Our study
uncovers three surprises: (1) visual quality alone does not guarantee task
success, controllability matters more; (2) scaling post-training with
action-observation data is more effective than upgrading the pretrained video
generators; and (3) allocating more inference-time compute allows WMs to
substantially improve closed-loop performance.
| Search Query: ArXiv Query: search_query=au:”Cheng Peng”&id_list=&start=0&max_results=3