#Model Compression

PrismML released Bonsai 27B, a 27-billion-parameter language model that can run on mobile devices through advanced quantization techniques, reducing its size from ~50GB to ~4GB. The model achieves competitive performance while enabling on-device inference. This represents a significant step for on-device AI, bringing large-scale model capabilities to smartphones and edge devices without cloud dependency. It could enable new privacy-preserving and offline applications, while sparking interest from major players like Apple. The model is likely based on the Qwen 2.5 architecture \(as per community hints\) and uses ternary quantization \(BitNet-like\) to achieve extreme compression. However, community benchmarks indicate tool-calling performance is notably degraded compared to similar-size models like Gemma 4 12B.