If you read one line: Arize Phoenix is an OpenTelemetry-native tracing-plus-evals tool for LLM and agent apps that you can self-host for free with one pip install. It captures every span, prompt, tool call, and token count using open standards, then lets you run LLM-as-judge evals over that data locally — no SaaS contract to see what your agent actually did. The honest asterisk: it's source-available under the Elastic License 2.0, not OSI open source, and self-hosting means you own the ops.

You shipped an agent. It works in the demo. Then a user reports it "did something weird," and you're staring at a wall of stdout with no idea which tool call went sideways, what the model actually saw, or how many tokens it burned getting there. You need a flight recorder for your AI — and you'd rather not hand your whole trace history to a vendor to get one.

That's the gap Arize Phoenix fills. Its one idea: instrument your app with open standards — OpenTelemetry for the transport, the OpenInference semantic conventions for LLM-specific detail — so every prompt, completion, tool call, retrieval, and token count lands as a structured trace you can actually read. Then, on the same data, run evaluations to turn "looks fine" into a score. And you can run the whole thing on your own machine before you pay anyone a cent.

1. What it is#

Phoenix, built by Arize AI, is an observability and evaluation platform for LLM and agent applications. It bundles the things founders usually cobble together:

Because it's built on OpenTelemetry, it's vendor-, language-, and framework-agnostic — the same instrumentation works whether you switch models or move hosts.

2. Who it's for#

Founders and engineers building LLM features or agents who want deep visibility on open standards without a SaaS contract as the price of entry — and teams already invested in OpenTelemetry who want an LLM-aware layer that speaks the same protocol as the rest of their stack. If you've hit the point where "read the logs" stops explaining what your agent did, or you're about to swap models and need to prove the new one isn't worse, Phoenix earns its place. The repo carries ~10.8k GitHub stars and ships integrations for OpenAI, Anthropic, Google GenAI, AWS Bedrock, LangGraph, LlamaIndex, CrewAI, DSPy, and the OpenAI Agents SDK.

3. How to start#

One line installs it; one more launches the UI.

pip install arize-phoenix
phoenix serve
# or, no install at all:
# uvx arize-phoenix serve

The app comes up at http://localhost:6006. Now point your code at it and turn on auto-instrumentation:

from phoenix.otel import register

# Wire your app to the local Phoenix collector and
# auto-instrument supported libraries (OpenAI, Anthropic, LangGraph, ...)
tracer_provider = register(
    project_name="my-agent",
    endpoint="http://localhost:6006/v1/traces",
    auto_instrument=True,
)

# ...run your app as normal; traces stream into the Phoenix UI.

Make one call and refresh the UI — the first trace showing up is the whole pitch. From there, attach an evaluator to score those traces, or build a dataset and run an experiment when you change a prompt.

What it means for you: the "see what my agent did" loop — install, launch, first trace — is an afternoon, and it costs nothing to find out whether Phoenix fits before you commit to hosted infra.

4. What it costs (as of July 2026)#

What it means for you: the tool is genuinely free to run yourself; the money question is whether you want to own the operations (self-host Phoenix) or pay Arize to run it (AX). Instrumentation is the same either way, so starting self-hosted doesn't lock you out of graduating to hosted later.

Phoenix's bet is that observability shouldn't start with a sales call. Open standards in, your data on your box, evals on top — and a paid tier waiting only when you actually want someone else to run it.

5. The catch#

Three honest ones.

First, "open source" is doing some work. Phoenix ships under the Elastic License 2.0 (ELv2) — source-available, not OSI open source. You can read, modify, run, and self-host it for free, but you can't offer Phoenix itself as a hosted or managed service to third parties. For a founder self-hosting for their own product, that's effectively free-to-use; if you contrast it with Langfuse's MIT-licensed core, which is true OSI open source, the difference matters only if you planned to resell it.

Second, self-hosting means you own the ops. The free version is real, but "free" is the license, not the maintenance — you're running the Postgres, the storage, and the scaling. When trace volume climbs, that's your pager, which is exactly the cost the paid Arize AX tiers exist to remove.

Third, evals cost tokens. LLM-as-judge scoring calls a model, so a large eval run over production traces has its own (usually modest) bill — budget for it the same way you would any LLM-in-the-loop workflow.

If those aren't dealbreakers, Phoenix is one of the cleanest ways to get real, standards-based visibility into an agent without paying for the privilege first. If you want a managed option out of the gate, weigh it against Langfuse, Phoenix, and Honeycomb head-to-head, the closed-but-eval-first Braintrust, or the managed-cloud observability options — but for "see it working on my own box today," Phoenix is hard to beat.