Model routers, unified APIs, and fast hosted inference for open + frontier models. Ranked by community traction, with live GitHub stars and what each is best at.
Short answer: the best llm gateways & inference for AI agents by community traction is Baseten (★ 0), followed by Cerebras Inference and Cloudflare AI Gateway / Workers AI.
Production inference platform: per-token Model APIs plus dedicated GPU deployments for custom and open models. Best for agent builders.
The fastest LLM inference in 2026 (1,800+ tokens/sec) on wafer-scale CS-3 chips, for open models like Llama and Qwen. Best for agent builders.
Free edge AI gateway (caching, rate/spend limits, observability) in front of any provider, plus Workers AI for serverless model inference. Best for agent builders.
Low-cost pay-per-use API for 50+ top open models via an OpenAI-compatible endpoint, no infra to manage. Best for agent builders.
Fast serverless inference for open models with serverless, on-demand, and reserved-capacity deployment tiers. Best for agent builders.
Ultra-fast inference on custom LPU chips (500+ tokens/sec) for open models like Llama, Qwen, and DeepSeek distills. Best for agent builders.
Open-access GPU and AI cloud with OpenAI-compatible serverless inference for 25+ open models plus on-demand GPU rental. Best for agent builders.
Open-source Python SDK + proxy server (AI gateway) that calls 100+ LLM APIs in OpenAI format with cost tracking, budgets, and virtual keys. Best for agent builders.
Serverless GPU/CPU cloud where you define container + hardware in Python code and run per-second-billed inference, batch, and training jobs. Best for agent builders.
OpenAI-compatible inference and fine-tuning for open models with base/fast checkpoint variants and transparent per-token pricing. Best for agent builders.
AI-native cloud: serverless access to 200+ open models, dedicated endpoints, GPU instances, and agent sandboxes through unified APIs. Best for agent builders.
A single OpenAI-compatible API that routes to 300+ LLMs from every major provider, with per-key budgets and programmatic key provisioning. Best for agent builders.
Production AI gateway routing to 1,600+ LLMs with observability, guardrails, governance, and an MCP gateway in one API. Best for agent builders.
Run thousands of community and official AI models (image, video, audio, LLM) via one HTTP API with no ML infra to manage. Best for agent builders.
Fast RDU-hardware inference for large open models (up to 405B) with an OpenAI-compatible API and a free developer tier. Best for agent builders.
Serverless and dedicated inference for 200+ open models (Llama, DeepSeek, Qwen, FLUX) plus fine-tuning, aimed at production AI teams. Best for agent builders.
One API key and endpoint to reach hundreds of models across providers, with zero markup, BYOK, budgets, and native AI SDK integration. Best for agent builders.
Generative-media inference platform serving 600+ image/video/audio/3D models (FLUX, Kling, Sora) through one fast, queue-based API. Best for agent builders.
By community traction, Baseten (★ 0) leads the llm gateways & inference in our directory. Production inference platform: per-token Model APIs plus dedicated GPU deployments for custom and open models.
Baseten is the most-starred open-source option; Cerebras Inference and Cloudflare AI Gateway / Workers AI are strong runners-up.
Baseten, at ★ 0 (live count).
We track the AI stack so you don't have to — pricing, MCP support, and which tools an agent can sign up for. Free.