Janas: Run LLMs larger than RAM by streaming experts from SSD
Enables developers to run massive LLMs on consumer hardware through efficient SSD streaming techniques
AI Summary
Janas-LLM enables running large language models (up to 80B parameters) on hardware with limited RAM by experts from SSD, achieving 37 tok/s on Qwen3-Next-80B with 32GB RAM.
