Tool Highlight: LiteLLM — One OpenAI-Shaped Door to 100+ Models

Every founder who ships an AI feature eventually meets the same wall. You wrote against one provider's SDK because it was fastest to start. Now a cheaper model would halve your bill, or a customer needs their data to stay on a specific cloud, and you discover that "swap the model" means touching every call site, every retry, every streaming handler. The provider isn't just a config value — it's threaded through your code.

LiteLLM is the tool that pulls it back out into a config value. It's an open-source gateway that puts one OpenAI-shaped interface in front of 100+ model providers, so the thing you change when you change models is a string.

What it is#

LiteLLM comes in two shapes, and the distinction is the whole story.

The SDK is a Python library you import. You call one function — completion() — with an OpenAI-style messages array, and LiteLLM translates it to whatever provider the model name points at. OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex, and a long tail of others all answer through the same request and response shape.

uv add litellm

from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "..."
os.environ["ANTHROPIC_API_KEY"] = "..."

# Same call, different provider — the only change is the model string.
r1 = completion(model="openai/gpt-4o",            messages=[{"role": "user", "content": "Hello!"}])
r2 = completion(model="anthropic/claude-sonnet-5", messages=[{"role": "user", "content": "Hello!"}])

The proxy is the same idea for a team. It's a server you run that speaks the OpenAI API, so any app — or any teammate's OpenAI client, in any language — points at it and gets all 100+ providers behind one base URL.

uv tool install 'litellm[proxy]'
litellm --model gpt-4o

import openai
client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000")
client.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}])

Who it's for#

If you're one person shipping one app, the SDK is the win by itself: you stop hard-coding a vendor and buy yourself the option to route to a cheaper tier or a different cloud later without a refactor.

The proxy earns its keep the moment there's a second consumer — a second app, a teammate, a background worker. Instead of a root API key copied into five .env files, you issue virtual keys per project that you can revoke, and you get spend and cost tracking per key, load balancing across deployments, guardrails, request logging, and an admin dashboard. It exposes the full menu of endpoints too — not just chat, but embeddings, images, audio, batches, and rerank — so it can sit in front of most of what an AI app touches, not only text generation.

The SDK removes vendor lock-in from your code. The proxy removes it from your organization — one governed door instead of a drawer full of loose keys.

What it costs#

The core is genuinely free. The SDK and the proxy are MIT-licensed and self-hostable at no cost — this is the part you can verify by reading the repo. BerriAI, the company behind it, also runs a managed cloud and an enterprise tier for teams that would rather not host it; I couldn't independently confirm those prices for this piece, so treat the enterprise docs as the source of truth rather than any number you read secondhand.

The honest catch#

Two, and both are visible in the source.

First, the proxy is infrastructure, not an import. It's a component you deploy, monitor, and keep patched, and it adds a network hop between your app and the model. For a single app, the SDK avoids all of that; for a team, the hop usually pays for itself in key hygiene and spend visibility — but it is a real thing you now own. Go in knowing which of the two shapes you actually need.

Second, LiteLLM is not 100% MIT. The repository's enterprise/ directory sits under a separate commercial license, so a handful of team-grade features — SSO and certain governance controls among them — live behind the paid tier rather than the open core. That's a fair split for a company that has to fund the thing, but it's worth knowing before you assume the whole platform is yours to run.

None of that changes the core pitch, which is unusually clean for infrastructure: the day you want to move off a model, LiteLLM makes it a string change instead of a sprint. In a year when three frontier labs reprice each other every few weeks, that optionality is the point.