Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models
Addresses critical robustness issues in VLA models through systematic analysis and zero-shot mitigation techniques.
AI Summary
Research paper identifies systematic language sensitivity in vision-language-action models where minor phrasing changes cause performance swings up to 61 points, and proposes rule-based rephrasing method that improves success rates by 16-27%.
Excerpt
Vision-language-action models (VLAs) are strikingly sensitive to instruction phrasing and do not inherit the language robustness of the vision-language models they are built on. A one-word edit can move success by tens of points: $π_{0.5}$ turns on a LIBERO stove 100% of the time for "switch on the stove" and 2% for "switch on the hot plate", and a $π_0$ checkpoint finetuned with rephrase augmentation still shows swings of up to 61 points. We characterize this sensitivity with statistically test
