What Matters in Quality-Aware VLA Pretraining: A Benchmark and Empirical Study
Provides empirical evidence and benchmarking framework for optimizing VLA pretraining strategies, crucial for robotics and AI researchers.
AI Summary
Researchers introduce AXIS-Bench for evaluating quality-aware vision-language-action pretraining methods, showing that episode conditioning improves robot success rates from 9.6% to 31.7% in real-world experiments.
Excerpt
How should demonstration quality be used during vision-language-action (VLA) pretraining to improve performance after target-task fine-tuning? We compare weighting, filtering, and conditioning across reward definitions and label granularities using 5,000 naturally suboptimal human demonstrations and 24 target tasks. Within each VLA, we match demonstrations and training budgets. Additional comparisons examine pretraining data size and the roles of quality tags during fine-tuning and inference. Ac