VaultGemma
2025-09-12 · Publisher : Google · Domain : Artificial Intelligence
Google Research and DeepMind release VaultGemma, an open model with one billion parameters, the largest pre-trained from scratch with differential privacy, along with associated scaling laws.
What changes
- Largest open model fully pre-trained with differential privacy (1 billion parameters, based on the Gemma 2 architecture).
- Sequence-level privacy guarantee: epsilon less than or equal to 2.0 and delta less than or equal to 1.1e-10 (sequences of 1,024 tokens).
- No detectable memorization of training data in tests (50-token prefix).
- Scaling laws to model the trade-offs between compute, privacy and utility.
Google Research and DeepMind release VaultGemma, an open model with one billion parameters, the largest pre-trained from scratch with differential privacy, along with associated scaling laws. ### What changes - Largest open model fully pre-trained with differential privacy (1 billion parameters, based on the Gemma 2 architecture). - Sequence-level privacy guarantee: epsilon less than or equal to 2.0 and delta less than or equal to 1.1e-10 (sequences of 1,024 tokens). - No detectable memorization of training data in tests (50-token prefix). - Scaling laws to model the trade-offs between compute, privacy and utility. ### Specifications - Size: 1 billion parameters. Weights published on Hugging Face. - Utility comparable to that of non-private models about five years older (comparison with GPT-2 1.5B). ### Pricing Open model, with no usage pricing of its own. ### Benchmarks Comparison with Gemma 3 1B (non-private) and GPT-2 1.5B on HellaSwag, BoolQ, PIQA, SocialIQA, TriviaQA, ARC-C and ARC-E (results shown in a chart, not quantified in the text). ### Key takeaways A reference point for privacy-sensitive organizations: protection comes at a measurable cost in quality.
Links
Tags : Open Source, LLM, Google, Gemma, Confidentialité