LiteLLM

Ring : ASSESS · Domain : OPS_INFRA · Vendor : BerriAI (Berri AI)

Python SDK and self-hosted AI gateway (proxy) exposing 100+ LLM providers in the OpenAI format: same responses, errors mapped to OpenAI exceptions, a Router with retries, fallbacks and load balancing. The proxy adds virtual keys with budgets per key, team or user, spend tracking, centralized logging, guardrails (content filtering, PII masking), caching and an admin UI, plus a gateway for A2A agents and MCP servers with per-key access control. A litellm-core package (SDK only) exists. Latest release: v1.105.0 (11 October 2026); Docker images signed with cosign since v1.83.0-nightly.

Business value

Provides a single entry point to consume several language models, with per-team cost control and usage limits. Avoids rewriting applications whenever the provider changes and gives IT a centralized view of consumption. Caveat: a supply-chain incident hit the PyPI package in March 2026.

AI value

Unifies access to LLMs and controls their cost behind a single OpenAI-compatible API.

Tags : gateway, proxy, llm, multi-provider, finops, mcp, a2a, open-source, supply-chain

https://www.litellm.ai