Large to Small Model Stitching Destroys What You Built It to Carry Along the Fit
Critical finding for researchers working on interpretability and safety techniques that rely on model stitching across architectures.
AI Summary
Model stitching, used in AI safety for transferring features between models, may destroy the very capabilities it's designed to preserve according to new research.
Excerpt
Independent research; early stage. Posting for feedback, particularly on whether the effect survives at realistic scale, and on prior work I've missed. Model stitching is extensively used in AI Safety infrastructure in reusing SAE & probes to make interpretability cheaper; transferring refusal, steering vectors so that small model can help align larger ones, cross architecture model diffing (1, 2). In this post, I expand on Chen's et al. work on Model Stitching to transfer linear features across