Unsloth has been available as an open-source framework for some time. Previously, to use the app, you had to work with Python ...

The small size and accessible hardware requirements mean that enterprises, indie developers, and even curious consumers can easily deploy the model locally without worrying about their data leaving ...

Bonsai 27B is a multi-billion parameter model small enough to fit into a smartphone. It’s useful for dev and research work, ...

8GB is still enough for certain use cases.

Highlights of Python 3.15 include lazy imports, faster JIT compilation, better error messages, and smarter profiling. A release candidate is now available. Python 3. ...

Quantization. Let me try to explain this properly, then tell you why I think it ends up being one of the more important technical threads of the next few years, and then hedge that with the thing that ...

Gemma 4 models are now available for download with quantization-aware training (QAT), which reduces the size and memory footprint of the models. These open-source models retain quality better thanks ...

As large language models scale to longer context windows and serve more concurrent users, the key-value (KV) cache has emerged as a primary memory bottleneck in production inference systems. For a ...

Google Research unveiled TurboQuant, a novel quantization algorithm that compresses large language models’ Key-Value caches by up to 6x. With 3.5-bit compression, near-zero accuracy loss, and no ...

turboquant-py implements the TurboQuant and QJL vector quantization algorithms from Google Research (ICLR 2026 / AISTATS 2026). It compresses high-dimensional floating-point vectors to 1-4 bits per ...