Your Transformer Is Secretly Elastic
Provides novel architecture-level optimization technique for ML engineers working with transformer models.
AI Summary
Researchers introduce a retraining-free method to make transformers elastic, enabling on-the-fly adaptation across computational budgets by jointly ranking MLP neurons and attention heads.
Excerpt
Scaling up transformers has produced models with broad capabilities, deployed across heterogeneous hardware and fluctuating workloads. Yet, model families offer only a few fixed sizes (e.g., S, B, L), forcing users to commit to a model based on hardware and latency constraints. This rigidity is often suboptimal under varying query loads. We introduce a retraining-free method that elastifies pre-trained transformers into a single model that adapts on-the-fly across a continuum of computational bu