Colibrì: Innovative Approach to Running Giant AI Models on Standard Hardware

Italian engineer Vincenzo (known as JustVugg) has unveiled a proof-of-concept called Colibrì, enabling the advanced GLM-5.2 language model—measuring 1.5 terabytes—to run on a standard PC. The system requires just 25GB of RAM, a CPU, and a virtual NVMe drive with a speed of 1GB/s. This solution unlocks new possibilities for enthusiasts and developers seeking to leverage powerful AI models without investing in costly server hardware like Nvidia NVL72.

GLM-5.2 is a Mixture-of-Experts (MoE) model featuring 744 billion parameters, capable of competing in response quality with solutions from leading companies such as OpenAI and Anthropic. However, due to limited hardware resources, Colibrì's token generation speed is a mere 0.05–0.1 tokens per second, making it unsuitable for real-time applications. For comparison, professional systems deliver 20–30 tokens per second.

Colibrì operates by dynamically loading only the necessary parts of the model into RAM. In the MoE architecture, each query segment is processed by separate 'experts'—specialized sub-models. Instead of loading the entire model, Colibrì loads only the experts required for processing the current token. This reduces memory requirements but places significant strain on the system's I/O and CPU performance.

Currently, the project remains experimental and does not support GPUs, though even with GPU acceleration, data transfer speed would remain the primary bottleneck. The developer is actively collecting data for optimization and invites the community to participate in testing. In the future, Colibrì could serve as a foundation for running advanced AI models on high-performance consumer hardware.