Gemma 3 QAT
2025-04-18 · Publisher : Google · Domain : Artificial Intelligence
Google releases versions of Gemma 3 trained with quantization (QAT), which sharply reduce the video memory required: the 27B model goes from 54 GB to 14.1 GB in int4 and fits on a consumer graphics card.
What changes
- Quantization-aware training (QAT) applied to Gemma 3 over about 5,000 steps, to limit quality loss at low precision.
- Gemma 3 27B runs locally on a consumer GPU such as the NVIDIA RTX 3090.
- Several quantized variants per size, ready for common inference engines.
### Specifications - Memory to load the weights: 27B from 54 GB (BF16) to 14.1 GB (int4); 12B from 24 GB to 6.6 GB. - 4B from 8 GB to 2.6 GB; 1B from 2 GB to 0.5 GB. - Context and modalities identical to Gemma 3 (128,000 tokens, vision for 4B and above). - Availability: int4 and Q4_0 weights on Hugging Face and Kaggle; supported by Ollama, MLX and llama.cpp (GGUF format). ### Pricing Open-weights models, with no usage pricing. ### Benchmarks The post compares quality (Chatbot Arena Elo) and GPU requirements without including a score for each quantized variant; the quality loss is described as minimal. ### Key takeaways Makes a 27-billion-parameter multimodal model usable on a workstation or a modest server, a key point for local use cases.
Links
Tags : Open Source, LLM, Google, Gemma, Quantification