Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize
Provides mechanistic insight into generalization that complements existing theoretical frameworks for ML researchers.
AI Summary
Researchers propose Neural Expansion, a geometric framework explaining how deep neural networks generalize through expansion of directionally-selective ray coverage in input space.
Excerpt
Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the ex