Skip to main content

Stateful AI Workflows: Routing, Checkpoints, and Human Review

Updated 6 min readMarkdown
Part of seriesHarness Engineering: From Models to Reliable Agents
Stateful AI Workflows: Routing, Checkpoints, and Human Review

Harness Engineering · Module 7 of 16

Start here: understand tools and observations. The design lab needs no framework or API key.

Your goal: specify how work resumes after a pause, failure, or review decision.

The policy assistant has drafted a reply. A reviewer edits it, then the process crashes. On restart, should the system draft again, reuse the edited version, or send it? A chat transcript alone is a poor answer to that operational question.

A stateful workflow records where the task is, what evidence it used, and what is still permitted. A graph makes transitions explicit. It can contain model-driven choices while keeping review and execution routes under application control.

A question routes through retrieval and drafting to review, with an explicit missing-evidence branch and saved checkpoints.

A checkpoint records execution state. It is not an external-action receipt.

1. Start with states you can explain

For the fictional shop, use received, retrieving, drafted, awaiting_review, approved, rejected, and failed. Keep sending out of this draft-only workflow.

State field Why it exists
Task and thread IDs Associate related work without mixing users
Trusted caller context Enforce access independently of model text
Evidence IDs and versions Explain the draft's policy basis
Draft revision Identify the exact artifact under review
Review decision Bind approval or rejection to that revision
Last completed step Decide where recovery begins
Pending operation ID Inspect ambiguous execution before retrying

Use a schema rather than accumulating arbitrary fields. Validate required transitions. A newly edited draft must not inherit approval granted to an older revision.

2. Choose chains, routers, and loops deliberately

A chain follows a known sequence: retrieve, draft, validate. A router chooses among defined branches: a policy question uses policy retrieval; an order-status question uses a different authorized tool. An agent loop lets the model select subsequent actions from observations.

These can share one graph. Use ordinary code for decisions already specified by business rules. Introduce model judgment where language interpretation or variable evidence actually calls for it.

In framework terms, nodes perform work and edges define control flow. A node may return a partial state update. A reducer determines how updates combine. Appending new messages is different from replacing the message list; concurrent writers need an explicit merge rule. Two nodes independently overwriting draft is a design defect, even if each result is individually good.

3. Understand checkpoint and replay semantics

The LangGraph persistence documentation explains its checkpoint model. A configured checkpointer and thread identity let executions continue from saved state. An in-memory checkpointer is useful for experimentation but does not survive process loss like a durable store.

The current interrupt documentation describes pausing with interrupt() and resuming with Command using the same thread identity. The interrupted node restarts from its beginning. Therefore code before the interruption can execute again.

Recovery rule: place external effects behind a stable operation identity and inspect their completion state. Saving a checkpoint does not make an arbitrary external write exactly-once.

Time travel and state editing are useful for diagnosis and alternative execution. They do not erase an email already sent or a record already changed. Treat replay against external systems as a separate consequential operation.

4. Make review a state transition

Present the reviewer with the actual draft, source support, unresolved information, proposed operation if any, and the version being approved. Store who decided, what they decided, and which revision it applies to.

Here is a complete deterministic example of revision-bound review. It uses no model or external service.

def can_release(state):
    return (
        state.get("review") == "approved"
        and state.get("approved_revision") == state.get("draft_revision")
        and state.get("draft_revision") is not None
    )


state = {"draft_revision": 1, "approved_revision": 1, "review": "approved"}
assert can_release(state)
state["draft_revision"] = 2
assert not can_release(state)
state["review"] = "rejected"
assert not can_release(state)
print("3 revision checks passed")

This block was run locally. It checks one release condition, not reviewer authentication, durable storage, or policy approval authority. Those need implementation at the surrounding application boundary.

5. Stream progress without confusing completion

A useful interface shows retrieval started, evidence found, drafting in progress, and review pending. Partial text can improve responsiveness, but a partial stream must remain visibly incomplete.

Track content events separately from state updates and final status. Cancellation can stop additional generation while leaving a previously completed tool operation in place. The UI should tell the user what actually stopped and what remains saved.

For parallel retrieval, let independent branches return labelled evidence, then merge by source ID and version. A map-reduce pattern processes independent inputs before synthesis. Subgraphs can encapsulate a reusable part of a larger workflow. Do not parallelize steps where one depends on the other's output.

6. Keep framework layers understandable

LangChain offers higher-level model, message, tool, and agent abstractions. LangGraph exposes state and control flow for more explicit orchestration. Add middleware when it provides a clear lifecycle intervention, such as context reduction or tool restrictions; overlapping policies should not obscure which rule stopped a run.

Runtime context can carry trusted application configuration, while state carries execution information. Persistent stores may hold cross-thread information with different retention and access rules. A model-generated tenant value is not trusted runtime identity.

For a chef agent, an event planner, and a draft-only email assistant, the graph infrastructure may be similar. Their domain checks differ: ingredient constraints, budget arithmetic, recipient and approval validation. Reuse the mechanism without erasing those differences.

7. Follow along

Draw a graph with two retrieval branches, a merge, a draft, and review. Add a missing-evidence route. Write a table of legal transitions, including rejection and an edited draft.

Simulate a crash before review, a crash after approval, and a repeated resume request. Expected behavior: saved evidence is recoverable; approval applies only to its revision; a retry cannot silently repeat an external action.

Then introduce two messages arriving on the same thread. Choose whether the second queues, interrupts, or creates a new run. Explain how your choice preserves state consistency.

Further reading

Next module

Advanced agent patterns: coordinate longer research, delegation, and event-driven work with explicit contracts.

About the author

  • I write about AI agents, harness engineering, and reliable software. My guides turn learning notes and small experiments into clear explanations, diagrams, and exercises you can try.

The checkpoint idea is the part I would steal first. I am building Homeowner HQ for realtors and lenders, launching December 4 on Amazon AWS, and the hardest moments are exactly the ones you describe: a long-running step that should be resumable, and a message between a homeowner, their realtor and their lender that should be reviewable before it goes out. A revision-bound approval check — one approval, one revision, no silent retries — is how a small team keeps that honest. The routing taxonomy helps too: most of my "bugs" have been a step routed to the wrong next action, not a bad model answer. Great guide.

SSantosh2h ago

Glad you found it useful! I would really interesting to hear how these patterns apply to what you're building with Homeowner HQ.

Wishing you the best for the launch.