T5Gemma

2025-07-09 · Publisher : Google · Domain : Artificial Intelligence

Google releases T5Gemma, a collection of encoder-decoder models obtained by converting decoder-only Gemma 2 models, with unbalanced size combinations such as a 9B encoder and a 2B decoder.

What changes

Adaptation technique: the weights of a decoder-only model initialize an encoder-decoder model, which is then pre-trained (UL2 or PrefixLM).
Based on Gemma 2 (2B and 9B adapted) and on new T5-sized models (Small, Base, Large, XL).
Unbalanced combinations are possible, for example a 9B encoder and a 2B decoder, suited to summarization.
Performance comparable to or better than the decoder-only equivalents on several benchmarks (including SuperGLUE), with a good quality-to-inference ratio.

### Specifications - Pre-trained and instruction-tuned versions published. - Open models, available for research and developers. ### Pricing Open models, with no usage pricing of their own. ### Benchmarks The post indicates performance comparable or superior to decoder-only models without including any specific score in the text. ### Key takeaways Renews the encoder-decoder architecture, useful for summarization and translation, with smaller models.

T5Gemma

What changes

Links