Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Provides novel theoretical framework for analyzing and improving speculative execution techniques in LLM inference.
AI Summary
Researchers introduce an information floor concept to separate path information loss from modeling imperfections in block drafting systems, showing current drafters remain far above their theoretical limits.
Excerpt
Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and imperfect modelling of observable information. Accepted length cannot distinguish them. We separate the two with an information floor, the minimum expected rejection at a specified conditioning order; rejection above this floor is the model gap. Estimating both from target rollouts across four domains, four open-weight t