WOVEN: Weaving Visual World Modeling into Multimodal LLMs
Presents novel training methodology and benchmark for improving visual world modeling in large multimodal models
AI Summary
Researchers introduce WOVEN, a and method that improves LLMs' visual transition reasoning capabilities by up to 27.3 percentage points on external benchmarks.
Excerpt
Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled
