Small LLMs perform no-CoT two-hop reasoning on synthetic facts with real bridges
Advances understanding of LLM reasoning capabilities and limitations for alignment research and monitoring.
AI Summary
Small LLMs like Qwen3.6-35B-A3B and Qwen3-8B can perform no-CoT two-hop reasoning when bridge entities are real, challenging prior limitations on synthetic-only bridges.
Excerpt
TL;DR A good understanding of where LLM no-chain of thought (no-CoT) two-hop reasoning succeeds and fails is important for monitorability as well as for designing belief instillation-based misalignment evaluations. Prior work suggests that LLMs are limited in their ability to perform no-CoT two-hop reasoning with pairs of fabricated, fine-tuned propositions. Here, I show on Qwen3.6-35B-A3B and Qwen3-8B that this limitation only holds if the bridge entity connecting the propositions is also fabri