Most prompt optimization is a slot machine. You define a metric, generate candidate prompts, score them, keep the high rollers, and pull the lever a few thousand more times. The optimizer learns exactly one bit per evaluation: the prompt scored 0.61, or it didn't. Everything else the run produced — the reasoning that went sideways, the tool it called with a malformed argument, the error string the API handed back — gets compressed into that single number and thrown away. GEPA's wager is that the discarded part was the valuable part.

GEPA — Genetic-Pareto, from the paper Reflective Prompt Evolution Can Outperform Reinforcement Learning (arXiv 2507.19457, accepted to ICLR 2026 as an Oral) — replaces the scalar with a sentence. Instead of recording that a rollout scored 0.61, it feeds the whole trajectory (the reasoning, the tool calls, the outputs, the errors) to a reflection LLM and asks, in effect, why did this fail, and what should the prompt say instead? The model writes a diagnosis in plain language and proposes a targeted edit. Then a genetic loop mutates and recombines prompts, and a Pareto frontier keeps not the single best candidate but a spread of complementary winners — the prompt that's great at multi-hop questions sitting next to the one that's great at instruction-following — so the search doesn't collapse into a local optimum that's mediocre everywhere.

The number that made people look

The claim that got attention: against GRPO, a reinforcement-learning baseline that does policy-gradient weight updates, GEPA reported beating it by up to roughly 20%, while using up to 35x fewer rollouts. Read that twice — it's not "a bit better for the same budget," it's better on a budget 35 times smaller. (Honesty note, because it matters: the original July 2025 preprint headlined ~10% average improvement; the ICLR camera-ready, after expanding from four benchmarks to six, revised the average down to about 6%. The "up to 20%" peak and the 35x efficiency figure held across versions.) The benchmarks were the usual gauntlet — HotpotQA, IFBench, HoVer, PUPA — across an open model (Qwen3-8B) and a proprietary one (GPT-4.1-mini).

Against the optimizer most DSPy users actually reach for, MIPROv2, GEPA reported a wider margin still: roughly +13% aggregate versus MIPROv2's +5.6%. That's the comparison worth internalizing, because it isolates the mechanism.

A scalar reward tells the optimizer that the agent failed. A language critique tells it why. The whole 35x is the gap between those two sentences.

Why language beats a number, mechanically

MIPROv2 is not naive — it runs Bayesian optimization over both the instruction text and the few-shot demonstrations, intelligently proposing configurations. But its feedback is still the metric score and nothing else. It's optimizing blind, inferring from whether runs succeed which direction to step. GEPA reads the runs. When a trace shows the agent confidently citing a document it never retrieved, GEPA's reflection step can say "the prompt never told it to ground claims in retrieved text" and write that instruction directly, rather than waiting for the score to drift upward over a hundred more samples.

The paper frames this as the core thesis, and it's the line worth quoting in a design review: the interpretable nature of language can provide a much richer learning medium than policy gradients derived from sparse, scalar rewards. RL squeezes a rich rollout into one number and then needs many numbers to triangulate. Reflection keeps the rollout's information intact. Information per sample is the entire ballgame for sample-efficiency, and a paragraph of diagnosis simply carries more of it than a float.

Where it sits next to TextGrad

GEPA isn't the first system to optimize with text instead of gradients — TextGrad backpropagates LLM-generated "textual gradients" through a compound system, a clean gradient-descent analogy in language space. The difference is what GEPA adds on top: evolutionary candidate generation with Pareto-frontier selection across instances. TextGrad tends to shine when your tasks are uniform in difficulty and a single descent direction helps everywhere. GEPA is built for the messier reality where your eval set is heterogeneous — some examples reward one strategy, some another — and you want to preserve the specialists on the frontier instead of averaging them into a generalist that's nobody's best.

Using it

GEPA is not a research curiosity you have to reimplement. It ships as a DSPy optimizer, dspy.GEPA, which you .compile() with a feedback metric and a reflection_lm (point that at a strong reasoning model — it's doing the diagnosis). There's also a standalone gepa library (pip install gepa) that optimizes any text artifact, not just DSPy programs — prompts, code, configs — and it's wired into MLflow's optimize_prompts as well.

The practical caution is the same one that applies to every optimizer that puts an LLM in the loop: the reflection model is now part of your cost and a part of what you're trusting. GEPA spends fewer rollouts than RL, but each rollout is read by a capable model that you pay for. The trade it offers is a good one — buy information per sample instead of buying more samples — but it is still a trade, and the right move is to measure it on your own task with your own metric, not to take a benchmark's word for it. The whole point of GEPA, after all, is that the trace tells you more than the score. That goes for evaluating GEPA, too.