Anatomy of a production LangGraph agent

Most LangGraph tutorials stop at the point where the interesting problems start. You wire up a model, give it two tools, loop back to the agent node, and it works: on your laptop, with your input, while you watch it.

Then it has to run for a month without you.

This is about what changes between those two programs. Not the whole framework: the five parts that decide whether an agent survives contact with production, the same five we build into every agentic system we ship. Snippets are Python against the LangGraph 1.x line, trimmed for reading. The API moves. Check the current signatures before you paste anything.

01. The state schema is the contract

An agent's state is not a scratchpad. It is the contract between every node, the thing persisted on every step, and the payload a human sees when they are asked to approve something. Design it before you write a node.

Two failure modes are common. The first is putting everything in the message list and letting the model's context be the source of truth. A transcript is not a data model: you cannot query it, you cannot validate it, and after twenty turns you cannot reliably find the invoice number in it. The second is the opposite: forty optional fields, most written by exactly one node and read by none.

What holds up is a small typed schema where every field has an owner and a reducer that says how concurrent writes combine.

from operator import add
from typing import Annotated, Literal, TypedDict

from langgraph.graph import add_messages


class ClaimState(TypedDict):
    # transcript: appended, never replaced
    messages: Annotated[list, add_messages]
    # extracted facts: replaced wholesale by the extractor
    claim: dict | None
    # audit trail: any node may append
    events: Annotated[list[str], add]
    # control
    attempts: int
    status: Literal["extracting", "review", "settled", "parked"]

The reducers are the part people skip. add_messages and add mean two branches running in the same step can both write without one silently overwriting the other. A field with no reducer is last-write-wins, which inside a fan-out is a race you will debug at two in the morning.

02. Nodes, edges, and who decides

Draw node boundaries where you would want to retry. A node that calls a model, parses the output, writes to your database and sends an email is one node with four failure modes and no useful retry semantics. Split it until each node has one reason to fail.

The second rule is about authority. Routing decisions belong in code you can read. LangGraph lets a conditional edge be an ordinary function over state. Use it. "The model decides what happens next" is a reasonable description of a chat assistant and a poor description of a claims pipeline.

from langgraph.graph import END, START, StateGraph
from langgraph.types import RetryPolicy


def route_after_extract(state: ClaimState) -> str:
    if state["claim"] is None:
        return "park" if state["attempts"] >= 3 else "extract"
    if state["claim"]["amount_gbp"] > 5_000:
        return "human_review"
    return "settle"


builder = StateGraph(ClaimState)
builder.add_node("extract", extract, retry_policy=RetryPolicy(max_attempts=3))
builder.add_node("human_review", human_review)
builder.add_node("settle", settle)
builder.add_node("park", park)

builder.add_edge(START, "extract")
builder.add_conditional_edges(
    "extract",
    route_after_extract,
    ["extract", "human_review", "settle", "park"],
)
builder.add_edge("settle", END)
builder.add_edge("park", END)

That router is four lines and a unit test. It is also the artefact a client's compliance reviewer will ask to see, and the reason you can answer "why did it escalate this one?" without reading a trace.

  invoke(thread_id)
        │
        ▼
   ┌─────────┐   fail   ┌──────┐
   │ extract │─────────▶│ park │──▶ END
   └────┬────┘  x3      └──────┘
        │ ok               ▲
        ▼                  │ unrecoverable
   ┌──────────┐            │
   │ validate │────────────┘
   └────┬─────┘
        │ clean
        ▼
   ┌──────────────┐  interrupt()   ~~ checkpoint written,
   │ human_review │- - - - - - - -▶ process exits, thread
   └──────┬───────┘                 waits for a decision ~~
         │ resume
         ▼
   ┌────────┐
   │ settle │──▶ END
   └────────┘

fig. 01: every arrow out of a node is either a code decision or a bounded retry. The dashed edge is the one that makes it a process rather than a function call.

03. Checkpoints

A checkpointer writes state to durable storage after every superstep (every batch of nodes that ran together), keyed by a thread id. Compile without one and your agent is a function call. Compile with one and it becomes a resumable process. Three consequences are worth stating out loud.

Crash recovery is free, at superstep granularity. If the pod dies mid-run, invoking again with the same thread id resumes from the last completed superstep rather than the top. Nodes that had finished do not re-run; the node that was in flight does. Which means every node has to be safe to run twice: idempotency keys on anything that spends money or sends mail, not "we'll be careful".

Thread ids are a domain decision. A random UUID is a thread nobody can find. Key on the claim, the conversation, the ticket: whatever an operator would type into a search box at the point they need to know what the agent did.

You can read and rewrite history. get_state returns the current values plus which nodes are queued next; update_state lets you patch state as though a given node had produced it, and continue from there. That is how a bad extraction gets fixed in production without re-running the pipeline.

from langgraph.checkpoint.postgres import PostgresSaver

with PostgresSaver.from_conn_string(DSN) as checkpointer:
    checkpointer.setup()                     # once: creates/migrates the tables
    graph = builder.compile(checkpointer=checkpointer)

    config = {
        "configurable": {"thread_id": f"claim-{claim_id}"},
        "recursion_limit": 25,
    }
    graph.invoke({"messages": inbound, "attempts": 0}, config)

    snapshot = graph.get_state(config)
    print(snapshot.next)      # ("human_review",) is queued, not yet run

The in-memory saver is for tests. It is a process-local dictionary, and two replicas of your service do not share it. Use Postgres, or whatever your ops team already backs up. And note that checkpoint rows grow: one per superstep per thread, each holding a serialised copy of state. Decide the retention policy before the table decides it for you.

04. Interrupts

Human-in-the-loop is where a lot of agent designs quietly become a chat window with an approve button. It deserves better, and the mechanism is there: interrupt() stops the graph inside a node, persists everything, and hands a payload back to the caller. The process ends. Hours or days later, a Command(resume=...) continues from the same place.

from langgraph.types import Command, interrupt


def human_review(state: ClaimState) -> dict:
    decision = interrupt({
        "kind": "claim_approval",
        "claim": state["claim"],
        "evidence": state["events"][-5:],
        "options": ["approve", "reject", "amend"],
    })
    if decision["action"] == "amend":
        return {"claim": decision["claim"], "status": "extracting"}
    approved = decision["action"] == "approve"
    return {"status": "settled" if approved else "parked"}


# Days later, from an HTTP handler, in a different process:
graph.invoke(Command(resume={"action": "approve"}), config)

The interrupt payload is an API. Whatever you pass to interrupt() is what the reviewer's screen has to render and what your audit log should keep. Pass a typed, self-contained object (the claim, the evidence, the options), not "the state". A reviewer who cannot see why they are being asked will approve everything by the second week, and then the checkpoint is the only thing left that knows the difference.

Resuming re-runs the node from the top. This one surprises people. interrupt() raises; on resume the whole node function executes again and the call returns the resume value instead of raising. Anything above it (a database write, an outbound API call, a counter increment) happens twice. Put side effects after the interrupt, or in their own node downstream of it.

05. Failure has three shapes

They want three different answers, and conflating them is how agents end up looping expensively.

Transient. Rate limits, timeouts, a 503 from a tool. Bounded retries with backoff at node granularity, which is what a retry policy on the node gives you. Cheap, and it handles most of what wakes people up.

Well-formed and wrong. The model returned valid JSON claiming the invoice date is 2031. Nothing raised. Retrying the same call gets the same answer in different words. The answer is a validation node with a bounded loop and the failure fed back as input.

def validate(state: ClaimState) -> dict:
    problems = check_claim(state["claim"])     # ordinary Python, no model
    if not problems:
        return {"status": "review"}
    return {
        "attempts": state["attempts"] + 1,
        "events": [f"validation failed: {'; '.join(problems)}"],
        "messages": [{
            "role": "user",
            "content": f"That extraction failed these checks: {problems}. Correct it.",
        }],
    }

Note that attempts lives in state rather than in a Python variable. It survives a restart, and the router can read it.

Unrecoverable. Three corrections, still wrong; or a dependency is down for the afternoon. The wrong move is to keep looping and spend the budget. Route to a terminal node that sets a status, records the reason and raises the alert. A parked thread is not a lost one: the checkpoint holds everything, so a human can patch state and resume it tomorrow. Set the recursion limit explicitly on every invocation too; the default is a safety net, not a design.

06. Evaluation

Someone will change a prompt. Someone will bump a model version. The framework will cut a minor release. Without a harness you find out from a client, which is the most expensive way to be told.

Two things are worth measuring, and they are not the same thing.

• Outcome. Given this input, is the final state correct? Cheap to build and the only measure a client cares about. Build the set out of real failures: every incident becomes a case. Fifty real ones beat five hundred synthetic ones.
• Trajectory. Did it get there sensibly? Which tools were called, in what order, how many model calls, at what cost. An agent that reaches the right answer in nine tool calls is a different program from one that does it in two, and only trajectory tests catch that drift after a prompt change.

Assert on what is genuinely deterministic: tool-call sequences, structured fields, schema validity, and explicit cost and latency budgets. Those belong in CI and should fail the build. Keep model-graded checks for open text, treat them as a noisy signal, and never let one gate a deploy on its own. Judges drift when the judge model changes, and they favour answers that look like the thing that produced them.

The unglamorous parts pay best. Pin model versions in configuration instead of letting an alias float underneath you. Log every run's trace, token counts and cost against its thread id, so a complaint about "the slow one last Tuesday" is a query. Re-run the regression set on every prompt change, not every release.

07. When not to build this

Not every problem needs a graph. If the workflow has one path, no branching, no human step and no need to survive a restart, what you want is a function with a retry decorator and a log line. It comes up more often than you might expect, and it is usually the cheaper piece of advice.

The test we use: can you draw it on a whiteboard as more than one box, and does at least one arrow depend on something you cannot know in advance? If not, the state machine is ceremony, and ceremony has an on-call rota.

When the answer is yes, the parts above are the difference between an agent that demos and one you can leave running: state you can query, a graph you can read, checkpoints you can resume, interrupts you can audit, failures that end somewhere on purpose, and a harness that tells you when it has stopped working.