New term: SLM
2024-04-23 · Publisher : Microsoft · Domain : Artificial Intelligence
An SLM (small language model) has at most a few billion parameters. It targets on-device, offline or low-cost uses where a large model would be oversized.
What changes
- Size is no longer the only race: training data quality becomes a core argument.
- Useful models run on a phone or laptop, without a cloud connection.
- Architectures pair small models for simple tasks with large models for the rest.
- Major vendors release compact versions of their models.
### Definition An **SLM** (*small language model*) is a compact model, from a few hundred million to about fifteen billion parameters. It trades some general knowledge for lower cost, higher speed and the ability to run locally. ### Origin Microsoft used the phrase as early as December 2023 for Phi-2, but the launch of Phi-3 on April 23, 2024 is what established the acronym. The Azure blog post was titled "Introducing Phi-3: Redefining what's possible with SLMs" and presented Phi-3 as "the most capable and cost-effective small language models (SLMs) available". ### What the term covers - Phi-3-mini (3.8 billion parameters), Phi-3-small (7 billion) and Phi-3-medium (14 billion); the technical report highlights a model that runs locally on a phone. - Microsoft lists three use cases: constrained environments (on-device, offline), cases where fast response times matter, and simpler tasks on a tight budget. - The recipe relies on heavily filtered and synthetic data rather than raw web volume. ### Criticism and limits The line between "small" and "large" is blurry and moves over time. SLMs shine on benchmarks but remain weaker on factual knowledge and on long or open-ended tasks. ### Key takeaway The SLM label turned model size into an architecture decision: pick the smallest model that is good enough for the task.
Links
Tags : Vocabulaire, Modèles, Embarqué, Coûts