JustVugg / colibri

项目摘要

Colibri is a pure C, zero-dependency inference engine for running frontier Mixture-of-Experts (MoE) models on consumer hardware. It keeps only active experts in RAM while streaming the rest from disk, enabling models with hundreds of billions of parameters on machines with as little as 8GB RAM and no GPU.

为什么值得关注

Its disk-streaming approach for MoE models is a genuinely novel solution to the memory bottleneck problem, making frontier models accessible on modest hardware. With 41K+ stars, 141 contributors, support for cutting-edge models like DeepSeek V4, Kimi K3, and Qwen3.x, and a dead-simple one-step setup, it has rapidly become a go-to tool for CPU-based inference of massive MoE models.

优势

• Pure C with zero dependencies — extreme portability and minimal footprint
• Innovative disk-streaming of MoE experts enables 700B+ parameter models on 8GB RAM
• Dead-simple one-step setup with automatic hardware detection and model recommendation
• Supports 13 cutting-edge model engines including DeepSeek V4, Kimi K3, GLM-5.x, Qwen3.x
• 141 contributors and 16 releases in 6 months showing strong community momentum

局限性

• No automated tests, reducing confidence in reliability across platforms
• No Docker or containerization support for reproducible deployments
• CPU-only inference will be significantly slower than GPU-based alternatives

使用场景

• Running frontier MoE models on consumer laptops/desktops without GPU requirements
• Offline/edge inference for large language models in resource-constrained environments
• Experimenting with and benchmarking MoE models on readily available hardware
• Educational exploration of how MoE sparsity enables efficient large model inference