EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
Novel technique for updating factual knowledge in model architectures without retraining, crucial for researchers working on model editing and maintenance.
AI Summary
EngramEdit enables precise LLM knowledge updates via conditional memory, achieving 3x baseline accuracy in chain-of-thought reasoning while preserving unrelated knowledge.
Excerpt
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation. Beyond model scaling, this architecture has demonstrated the potential to decouple factual knowledge storage from general-purpose computation, offering a promising route to updating factual knowledge while keeping the Transformer backbone fixed. Realizing this potential is challenging because differen