A problem with three correct answers. Which, how, and when does a model choose?
Provides insight into how models resolve ambiguity and make internal choices, relevant for AI safety and interpretability research.
AI Summary
Researchers analyzed how Qwen 2.5 models handle Bertrand's paradox across 18 phrasings, tracking which of three correct answers they choose and when this choice emerges internally.
Excerpt
TL;DR: A look at how models (3 sizes of the Qwen 2.5 model family) behave when posed with Bertrand's paradox. 18 phrasings, up to 30 answers each. For each, we check which of the three famous responses it outputs, whether it explains the method or recalls it, and when the method choice shows up in the model's internals. When models are posed with an ambiguous question, it is very useful to know whether we can identify their choice through the chain of thought transcript alone. We asked whether