If you have watched an AI agent do something baffling in production — burn 40 tool calls, quietly loop, or return a confident wrong answer — your next move is to open a trace and find out what it actually did. The question is which tool you open. Three keep coming up: Langfuse, Arize Phoenix, and Honeycomb. Founders treat this as a bake-off and try to rank them. That is the wrong frame. They are not three answers to one question; they are three archetypes, each built for a different question. Pick the one whose question you already ask at 2am.
Here is the whole decision in one screen, then the reasoning.
- Langfuse — the LLM-native default. Open-source (MIT core), self-hostable, and it puts tracing, prompt management, datasets, and LLM-as-judge evals in one tool. Pick it when model quality is the product.
- Arize Phoenix — the eval-first open lab. OpenTelemetry-native, the strongest evaluation and local-debugging story, free to self-host. Pick it when your bottleneck is "is the output any good," not "where is the trace."
- Honeycomb — one pane of glass. A proprietary APM that now speaks the OpenTelemetry GenAI conventions, so agent spans sit next to your database and queue spans. Pick it when the agent is one slice of a bigger system you already run.
All three ingest OpenTelemetry and all three have a free tier, so the instrumentation you write is portable and the trial is cheap. The choice is about archetype, not vendor.
Archetype 1 — Langfuse: model quality is the product#
Langfuse is what most teams mean when they say "LLM observability." A single tool holds the trace of every agent turn, a prompt registry you can version and roll back, datasets you can replay, and an LLM-as-judge evaluator to score outputs. It is framework-agnostic, and the core is MIT-licensed on GitHub — verifiable, not a marketing claim — so you can self-host the whole thing.
Two things changed in 2026 worth knowing. First, ClickHouse acquired Langfuse on January 16, 2026 and publicly committed to keeping the MIT core, self-hosting, and Cloud unchanged — an open-core promise you should nonetheless keep an eye on. Second, self-hosting is not free of ops: a real deployment wants Postgres plus ClickHouse (plus Redis and blob storage at scale). If you do not want to run that, Langfuse Cloud has a free hobby tier (reported around 50,000 units/month) that a solo founder rarely outgrows early.
Pick Langfuse if the thing you tune every week is the prompt, the model, and the output — and you want prompt management and evals in the same place you read traces. This is the safe default for an LLM-first product. (For how it stacks against the other LLM-native players, we compared Langfuse vs Laminar vs Arize Phoenix head-to-head.)
Archetype 2 — Arize Phoenix: the eval-first open lab#
Phoenix comes out of Arize's ML-observability lineage, and it shows in one place: evaluation. It is OpenTelemetry-native via the OpenInference instrumentation libraries, runs locally or self-hosted for free, and its evaluation story — LLM-as-judge, versioned datasets, experiments that track a prompt/model/retrieval change, a playground to replay traced calls — is the most rigorous of the three. If your day is "change one thing, prove it got better," Phoenix is built for that loop.
The two things to check before you commit: the core is under the Elastic License 2.0, not Apache or MIT. ELv2 lets you self-host freely; the only thing it forbids is reselling Phoenix as a hosted service — a restriction that never binds an internal team but matters if that was your business model. And Phoenix (open) sits below Arize AX (the paid SaaS), so "Phoenix" and "Arize" are two products; know which one a given feature or price refers to.
Pick Phoenix if your bottleneck is output quality and iterative debugging, you want a free open-source core, and evaluation depth beats having prompt management bundled in. It is the researcher's-bench archetype.
Archetype 3 — Honeycomb: one pane of glass#
Honeycomb is the odd one out on purpose. It is a general-purpose observability/APM platform — proprietary and SaaS-only, no self-host — but in 2026 it launched Agent Observability (May 12) built on the OpenTelemetry GenAI semantic conventions (v1.40.0). The consequence: your gen_ai.* agent spans land in the same wide-events store as your HTTP, database, and queue spans, and you debug all of it with one BubbleUp query.
The trade is explicit. Honeycomb has no LLM-specific eval UI — no prompt diffs, no LLM-as-judge scores attached to spans, no dataset management. An agent turn is a generic span. What you buy is correlation: when a slow agent step is really a slow downstream API, or a spike in token cost tracks a specific customer's traffic, you see the whole causal chain in one place instead of alt-tabbing between an LLM tool and your APM.
Pick Honeycomb if the agent is one component of a larger production system you already instrument with OpenTelemetry, and unified debugging matters more than LLM-specific views. We drew this exact contrast in Honeycomb vs Langfuse: APM lineage vs LLM-native.
The 30-second decision#
Ask what you were doing right before you needed the tool:
- Tuning a prompt or model? → Langfuse. Traces and evals in one LLM-native place.
- Trying to prove an output got better? → Phoenix. The evaluation loop is the whole point.
- Chasing a slowdown or cost spike across a system? → Honeycomb. The agent is a span among spans.
Because all three speak OpenTelemetry, this is not a marriage. Instrument once with OTLP, point it at a free tier, and switch archetypes if the question you ask changes. The mistake is not picking the "wrong" tool — it is picking by brand instead of by the question you actually have. It's the same discipline that should decide the model underneath the agent you're watching: pick your open coding model by license and serving cost, not the leaderboard.
