Docker Desktop 4.40: Docker Model Runner runs LLMs locally

2025-04-01 · Publisher : Docker · Domain : Cloud & DevOps

On 1 April 2025, Docker Desktop 4.40 introduces Docker Model Runner in beta: local execution of language models packaged as OCI artifacts, accelerated by Apple Silicon GPUs, with an OpenAI-compatible API.

### Background On 1 April 2025, Docker releases **Docker Desktop 4.40**, which introduces **Docker Model Runner** in beta. The tool lets you run language models locally from Docker Desktop, with GPU acceleration on Apple Silicon. Models are packaged as OCI artifacts, so they can be stored and versioned using the same registries as container images. ### Key facts - llama.cpp inference engine, reachable through an OpenAI-compatible API; execution happens on the Mac host, to use the hardware directly. - Models distributed as OCI artifacts, with versioning and CI/CD integration. - Announced roadmap: Windows with GPU, publishable custom models and tighter integration into the development loop. - The Gordon agent (Docker AI Agent) supports the MCP protocol, as both client and server, and runs shell commands, Git operations and file management. - An AI tool catalog extension lets you browse and connect containerized MCP servers published on Docker Hub. ### Why it matters Docker puts model execution in the container workflow developers already master, which eases generative AI prototypes without an external service. For an IT services firm, the interest is reproducibility: models are versioned like images. The limits at the time are clear: beta, Apple Silicon Macs first, and a local usage that does not replace a production inference service. The release also adds, for Docker Business plans, real-time tracking of compliance with Docker Desktop settings policies. ### Takeaway Docker now treats a model as an OCI artifact: local AI enters the Docker workflow.

Tags : Conteneurs, IA, MCP, Cloud natif