Achieving Near-Lossless Model Compression

PrismML is challenging the assumption that high-reasoning LLMs require massive cloud infrastructure. By focusing on aggressive model compression, the startup has developed the Bonsai 2 27B model, which compresses the Qwen3.8 27B model down to 5.9 GB—a 9x to 10x reduction in memory footprint.

Unlike traditional quantization methods that may significantly degrade performance, PrismML’s approach achieves 98% of the original model's aggregate benchmark scores. CEO Babak Hassibi notes that while perfect parity remains elusive, the 2% performance gap is largely academic, as benchmarks often fail to capture real-world task performance, and the surrounding software harness plays a critical role in final output accuracy.

The Mechanics of Ternary Weights

PrismML achieves this footprint reduction by simplifying the model's weights. While standard models typically store weights in 16-bit precision, PrismML utilizes a "ternary" approach, reducing each weight to one of three values: +1, -1, or 0. This drastic simplification allows for significant memory savings without sacrificing the model's underlying intelligence.

Looking ahead, the team plans to apply this compression technique to models in the several-hundred-billion-parameter range. Hassibi suggests that larger models are actually easier to compress while maintaining intelligence, as there is more "room" to optimize without losing the core reasoning capabilities of the original architecture.

The Shift to Edge-Native Intelligence

By enabling advanced models to run locally on PCs and high-end smartphones, PrismML aims to solve two primary friction points in current AI adoption: cost and privacy.

  • Cost: Running models on existing consumer hardware eliminates the need for expensive cloud inference, effectively making high-end AI "free" for the end user.
  • Privacy: Local execution ensures that sensitive data never leaves the user's device, providing a secure alternative to cloud-based API calls.

This shift toward edge-native AI is supported by industry veterans like Ion Stoica, who emphasizes that the ability to run powerful models locally will fundamentally change how users interact with intelligence at their fingertips.