Can Language Models Play the Wikipedia Game?
Provides rigorous benchmarks for language model navigation capabilities in information retrieval contexts.
AI Summary
Researchers tested language models on Wikipedia Game navigation tasks across five languages and academic datasets, finding top models achieved human-comparable performance.
Excerpt
Language models increasingly support or automate information retrieval from document collections too large to index. This task is difficult when it requires specialist linguistic or scientific knowledge. We used the Wikipedia Game to test language models on an analogous information retrieval task. In the game, players must reach a target article using only article links. We compared sentence embedding models and LLMs across five Wikipedia languages and five academic-literature datasets. We const