If you read our argument that most multi-agent systems put an LLM in charge of the one layer you most want cheap and reproducible, the obvious next question is: okay, so how do I actually build the other thing? This is that build. It is not "rip out the LLM." It is a specific three-layer split that keeps the model where it earns its cost and routes for free everywhere else.

Here is the whole pattern in one screen, because that's the part you'll come back for:

The mistake the default design makes is collapsing all three layers into one always-on supervisor. Splitting them is the whole trick.

The rule that decides every fork#

Before any code, the decision rule, because it's what you'll apply dozens of times:

A fork is deterministic if you can write its condition as a boolean over state — a field, a status code, a number, a regex, a threshold — and you knew the branches at author time. A fork needs the classifier only when deciding requires reading unstructured intent a human typed in free text.

Two questions, in order:

  1. Is the input structured? order.status == "refunded", amount > daily_cap, retries >= 3 — these are lookups. An LLM here converts a lookup into a probabilistic, billable guess. Route them for free.
  2. Did you know the branches at author time? A workflow can have fully dynamic data and still have fixed structure. "After payment, either fulfill or hold for review" is a known fork even though which path any given order takes is decided at runtime. Known structure → deterministic.

Only when both answers are "no" — the input is free text and you can't enumerate the branches from state — do you reach for the model. In a typical support or ops flow, that's one fork, maybe two. The rest route on conditions.

Layer 1 in LangGraph: add_conditional_edges#

LangGraph's add_conditional_edges takes a node name and a routing function — plain Python that reads state and returns the name of the next node. No model is involved. This is your deterministic router.

from langgraph.graph import StateGraph, START, END

def route_after_intake(state) -> str:
    order = state["order"]
    if order["status"] == "refunded":
        return "close_ticket"
    if order["amount"] > state["config"]["auto_approve_cap"]:
        return "human_review"
    return "fulfill"

builder = StateGraph(OrderState)
builder.add_node("intake", intake)
builder.add_node("fulfill", fulfill_agent)
builder.add_node("human_review", human_review)
builder.add_node("close_ticket", close_ticket)

builder.add_edge(START, "intake")
builder.add_conditional_edges(
    "intake",
    route_after_intake,                       # zero tokens spent deciding
    {"fulfill": "fulfill",
     "human_review": "human_review",
     "close_ticket": "close_ticket"},
)

Every one of those branches is a condition you could have drawn on a whiteboard. None of them should cost a model call. The routing function is a total function over a fixed set of labels, which means the graph is reproducible: same state in, same path out, every time.

If you prefer to keep routing inside the node — combining the state update and the next-hop in one place — LangGraph's Command does exactly that: return Command(goto="fulfill", update={...}) instead of using a separate conditional edge. Same determinism, different ergonomics.

Layer 2: the classifier escape hatch#

Now the one fork you can't route for free: a customer typed a paragraph, and you need to know whether it's a refund request, a complaint, or a question before you can dispatch. That's unstructured intent — the classifier's job.

The critical word is constrained. Do not hand this to a supervisor agent that "figures out what to do." Make one call that can only return a label:

from typing import Literal
from pydantic import BaseModel

class Intent(BaseModel):
    label: Literal["refund", "complaint", "question"]

def classify_intent(state) -> dict:
    result = model.with_structured_output(Intent).invoke(
        [{"role": "system", "content": CLASSIFY_PROMPT},   # tiny, stable → cacheable
         {"role": "user", "content": state["message"]}]
    )
    return {"intent": result.label}                        # write the label into state

Three constraints make this cheap and reproducible:

Then the routing off that label is deterministic again — back to Layer 1:

builder.add_node("classify", classify_intent)   # the ONLY node that calls a model to decide
builder.add_conditional_edges(
    "classify",
    lambda s: s["intent"],                        # plain switch on the label
    {"refund": "refund_agent",
     "complaint": "complaint_agent",
     "question": "faq_agent"},
)

One node in the entire graph spends a token to decide anything, and it spends exactly one, on the exactly one fork that needed it.

The same shape in CrewAI Flows#

If you're on CrewAI, Flows give you the same split through decorators. @start marks the entry point, @listen(...) methods fire when an earlier step emits their branch, and @router runs your own Python and returns the string that decides which @listen fires next.

from crewai.flow.flow import Flow, start, router, listen

class SupportFlow(Flow):
    @start()
    def intake(self):
        self.state["order"] = load_order(self.state["order_id"])

    @router(intake)
    def route_order(self):
        order = self.state["order"]
        if order["status"] == "refunded":
            return "close"                    # deterministic — no crew, no tokens
        if order["amount"] > CAP:
            return "review"
        return "classify"

    @listen("classify")
    def classify_then_dispatch(self):
        # the one place a model decides: a small classification crew or single call
        self.state["intent"] = classify_crew.kickoff(inputs=self.state).pydantic.label
        return self.state["intent"]           # "refund" | "complaint" | "question"

    @listen("refund")
    def handle_refund(self):
        ...

The control graph lives in the decorators — you can read the whole thing top to bottom in a code review. The deterministic forks (route_order) spend nothing; only classify_then_dispatch touches a model, and it's a single constrained call, not an open-ended crew deciding its own next move.

Where the LLM router is still the right answer#

None of this says "deterministic always wins." When a task is genuinely open-ended — you cannot enumerate the branches in advance, and progress depends on the model reading its own intermediate results — an LLM orchestrator is correct, and the cost is the price of the capability. Anthropic's own multi-agent research system is the honest example: a lead agent spawning subagents beat a single agent by a wide margin on open-ended research, and it burned far more tokens to do it. That's a good trade for open-ended discovery. It is a bad trade for "after payment, fulfill or hold."

The point of the three-layer split is to stop paying that price on the 90% of your forks that never needed it. Route the known structure for free, spend one constrained call on the one ambiguous fork, and let your agents be as expensive as the actual work demands — not the dispatch.

Failure modes, and how each degrades#

Build the router dumb, put the one gate where the ambiguity actually is, and keep your agents smart. That's the whole thing.