▟ ollama/ollama
run open-weight LLMs locally with one command; wraps llama.cpp and now MLX, exposes an OpenAI-compatible server on :11434
★ 178kGoollama/ollama
▟ ggml-org/llama.cpp
the reference GGUF inference engine and llama-server — the substrate Ollama and LM Studio are both built on
★ 123kC++ggml-org/llama.cpp

Short version: if you want an agent to talk to an open-weight model running on your own machine, you have three mainstream front doors — Ollama, LM Studio, and llama.cpp — and they are more alike than the arguments about them suggest. All three put the same thing in front of your model: an OpenAI-compatible /v1/chat/completions endpoint. Your agent framework doesn't know or care which one is behind it. So the real choice isn't speed. It's how much of the plumbing you want to own.

The one thing they have in common (and why it decides less than you think)#

Point any agent framework — LangChain, LlamaIndex, the OpenAI SDK, your own loop — at a local backend by changing exactly two fields:

from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:11434/v1",   # Ollama. LM Studio: :1234  ·  llama.cpp: :8080
    api_key="local",                          # any non-empty string
)

That's it. The port is the only thing that differs — Ollama 11434, LM Studio 1234, llama.cpp's llama-server 8080. Because the wire format is identical, "which is faster" mostly resolves to which model and quantization you loaded, not which runner served it — two of the three (Ollama, LM Studio) are literally sitting on the same llama.cpp/GGUF engine underneath. Decide on ownership instead.

Ollama — the fastest path from zero to an endpoint#

Ollama (MIT, ~178k★, written in Go) is the shortest distance between nothing and a working backend. Install it, then:

ollama run qwen3          # pulls the model, then drops you into a chat
ollama serve             # the OpenAI-compatible server (auto-starts with the app)

The server is live on localhost:11434 and your agent is one base_url away. It has native tool calling with per-model parsers and structured outputs — you can constrain a response to a JSON schema, which is the single most useful feature when an agent needs a parseable answer rather than prose. It ships a background server, a scriptable CLI, and official Python/JS client libraries, and it runs on macOS (Apple Silicon), Linux, and Windows. In 2026 it added an MLX engine for Apple GPUs alongside its llama.cpp core.

If you're prototyping an agent and want a reproducible, docker-friendly local backend you can stand up in a script, this is the default. ollama run and move on.

LM Studio — the GUI-first desk for founders who don't live in a terminal#

LM Studio is a free desktop app (macOS/Windows/Linux) built around a graphical model browser: search a catalog, see what fits your RAM, download, and click Start Server. Under the hood it runs the same llama.cpp/GGUF engine plus an Apple MLX backend on Apple Silicon. The app is proprietary, but it's been free for both personal and commercial use since July 8, 2025 — no form, no fee — which removes the usual objection to putting it on a work machine.

For agent builders, two things matter: it exposes the OpenAI endpoint on localhost:1234, and it shipped first-class MCP support, so an agent can reach tools without you hand-rolling the wiring. It isn't GUI-only — the open-source lms CLI scripts the same actions:

lms get qwen3            # download a model
lms load qwen3           # load it into memory
lms server start         # start the OpenAI-compatible server on :1234

Reach for LM Studio when you want to eyeball and compare which local model is worth wiring into your agent before committing — the GUI is the product, and it's the least-CLI on-ramp of the three.

llama.cpp — the engine itself, when you want to own the plumbing#

llama.cpp (MIT, ~123k★, C/C++) isn't a wrapper — it's the engine the other two are built on. You reach for it directly when you want maximum control or the smallest footprint. Its llama-server is a full OpenAI-compatible HTTP server (default 127.0.0.1:8080) that also exposes an Anthropic-compatible /v1/messages route, an embeddings endpoint, and a built-in Web UI:

# build from source or `brew install llama.cpp`, then:
llama-server --jinja -fa -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M

The --jinja flag is the one to remember: tool calling on llama.cpp requires it, plus a tool-aware chat template (Llama 3.1+, Qwen 2.5, Hermes, Mistral Nemo, and others ship native templates). The docs also warn against aggressive KV-cache quantization — it degrades tool calling. In exchange for that fiddliness you get every knob: quantization, sampling, custom templates, and CPU/CUDA/Metal/ROCm/Vulkan backends. It's also the right pick when you're embedding inference into your own binary rather than running a separate service.

The caveat that actually decides agent projects#

Two truths sit underneath all three, and they matter more than the port numbers:

  1. Tool-calling reliability is a model property, not a runtime property. The endpoints all support function calling. Whether it works depends on the model you loaded — tool-tuned models (Qwen 2.5/3, Llama 3.1+, Hermes, gpt-oss) hold up; small heavily-quantized models emit malformed JSON and hallucinate tool names inside an agent loop. Test your specific model before you trust it.
  2. These are single-user runtimes. None of them batch concurrent requests the way a hosted API or vLLM does. For one developer or a privacy-bound workload they're excellent; for many simultaneous users they fall over. The durable pattern is local for dev and privacy, hosted for scale.

Pick on ownership: Ollama to get an agent talking to a local model in one command, LM Studio to browse and manage models without leaving a GUI, llama.cpp when you need the whole engine in your hands.