clem 🤗 (@ClementDelangue): llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K (on two mac studios). Well done @finkd & @johnternus! https://t.co/Kjl7qbPV52
Shows practical deployment of massive models on consumer hardware for builders working with local AI inference.
AI Summary
Hugging Face CEO demonstrates running an 816B parameter model on two Mac Studios using 'llama serve' with a single command.
Excerpt
llama serve -hf ggml-org/Muse-Spark-Open-816B-A42B-GGUF:Q2_K (on two mac studios). Well done @finkd & @johnternus! https://t.co/Kjl7qbPV52
