In 2023, if a single chain of thought couldn't crack your problem, the answer was a topology: don't ask the model once, ask it to branch, evaluate its own branches, and search. Tree of Thoughts, Graph of Thoughts, and MCTS-based agents like LATS were the state of that art, and their headline numbers were genuinely startling — GPT-4 went from 4% to 74% on the Game of 24 puzzle simply by wrapping chain-of-thought in a tree search (Yao et al., NeurIPS 2023).

Then reasoning models arrived and quietly absorbed most of that job. A model that reasons internally, with a tunable effort budget, already does much of what an external tree search was bolted on to provide. So the 2026 question is not "which scaffold is best." It's narrower and more useful: does this problem justify structured search at all — and if so, which shape? Two gates answer the first half; the task shape answers the second.

The two gates: earn the search before you build it#

Gate 1 — do you have a reliable, cheap step-level evaluator? Every method here lives or dies on one capability: scoring partial states. Which branch to expand, which to prune, which rollout looked promising — all of it needs a signal (a verifier, a passing/failing test suite, a heuristic, a learned value function). Without one, search doesn't find better answers; it explores expensively and then picks wrong. If you can't produce that signal cheaply and reliably, stay linear. This is the prerequisite people skip, and it's why so many ToT reimplementations underperform plain prompting.

Gate 2 — are you on a reasoning model doing a mostly-linear task? If yes, your first move isn't a tree — it's the model's own internal reasoning plus a tuned reasoning-effort setting, optionally self-consistency or best-of-N for cheap variance reduction. OpenAI's guidance is explicit that reasoning models don't need "think step by step" scaffolding and can be hurt by it (reasoning best practices); Anthropic's is to find the simplest thing that works and add complexity only when it pays. Add explicit search only when Gate 1 holds and the space is genuinely branchy. The lever you should reach for before any topology is reasoning effort vs thinking budget.

Clear both gates and the question becomes which topology — and that follows directly from the shape of your task.

Tree of Thoughts: branch, judge, backtrack#

Tree of Thoughts generates multiple candidate next-steps ("thoughts"), has the model self-evaluate each, and searches the tree — expanding promising branches and backtracking from dead ends to make a global decision rather than a greedy local one. That's what turned 4% into 74% on Game of 24: the puzzle is branchy and has a clean "is this partial arithmetic still viable" check, which is exactly the discriminator ToT needs.

Two caveats keep this honest. First, ToT does not consistently beat simpler IO/CoT prompting across all models and tasks — the win is conditional (LMU Munich, 2024). Second, the same study finds the generator matters more than the discriminator: scaling the model's ability to propose good thoughts helps a lot; scaling its ability to judge them helps only marginally. So ToT earns its cost on branchy, one-shot reasoning problems where you have both a strong generator and a cheap, trustworthy branch score.

Graph of Thoughts: when partial answers combine#

A tree only branches and backtracks — every thought has exactly one parent. Graph of Thoughts models thoughts as an arbitrary graph: vertices are LLM thoughts, edges are dependencies, and the graph permits what a tree structurally cannot — aggregation (many thoughts merged into one, N-to-1) and cycles (refine a thought, feed it back) (Besta et al., AAAI 2024).

That extra expressiveness pays off precisely when sub-solutions combine. On sorting — split the list, sort the pieces, merge — GoT raised solution quality 62% over ToT while cutting cost by more than 31%, because merging is a first-class operation instead of something a tree has to fake. The routing rule writes itself: if your problem decomposes into pieces whose partial results you then fold back together (sorting, set operations, document merging, map-reduce-shaped reasoning), reach for the graph. If it doesn't, the graph's machinery is overhead.

MCTS / LATS: sequential decisions with feedback#

The third family isn't for one-shot reasoning at all — it's for sequential decision-making, where the agent takes an action, the environment responds, and an early wrong move is expensive. Monte Carlo Tree Search brings simulated rollouts and value backpropagation; LATS (Language Agent Tree Search) unifies reasoning, acting, and planning by combining MCTS with an LM value function and self-reflection (Zhou et al., ICML 2024). It reports 94.4% pass@1 on HumanEval with GPT-4 and, in its evaluation, outperforms ReAct, Reflexion, CoT, and ToT.

This is the heaviest option — rollouts multiply calls — so it's reserved for agentic tasks with real environment feedback: coding against a test suite, web navigation, multi-step tool use where you can score a trajectory. LATS is also MCTS-flavored self-correction, which is why it sits next to the verification-centric methods we compared in Reflexion vs Self-Refine vs Critic vs LATS — there the axis was who checks the work; here it's how you search the action space.

The routing rule, in one screen#

No reliable step evaluator → don't search; stay linear and tune reasoning effort. Combinable sub-results → Graph of Thoughts. Sequential actions with environment feedback → MCTS/LATS. A branchy one-shot reasoning puzzle with a good discriminator → Tree of Thoughts. Everything else on a modern reasoning model → linear chain-of-thought, which the model now does internally.

The deeper shift is that deliberate search has become a specialist tool, not a default. The 2023 instinct was to reach for a topology whenever reasoning got hard; the 2026 instinct is to reach for the model's own reasoning first and add structure only where the task's shape — branchy, combinable, or sequential-with-feedback — and a real evaluator both demand it. This is the same "which reasoning pattern" discipline we applied on a different axis in ReAct vs Plan-and-Execute vs Reflexion: the pattern isn't the point, the fit is. Build the evaluator, check the shape, and most of the time the cheapest scaffold that clears both gates is the right one.