Agentic AI · L-Orch
The hand-written agent loop (→ L-ReAct) as a finite-state machine, and the two questions every orchestration mechanism answers: what computes its transition function — code, the model or a person — and what event causes it to take a transition at all.
Every mechanism in this lecture answers one of two questions about the same machine: what computes its transition function $\delta$ — the rule that selects the next state — and what event causes it to take a transition at all. Part I defines the machine — a finite-state machine whose transitions read the whole context history, not only the current state; Parts II and IV answer the first question, Part III the second.
| Part | The question it answers | Section |
|---|---|---|
| I · The State-Machine Formalism | What is the common formal object, how is its transition graph represented, and what does $\delta$ read? | Agents as Finite-State Machines |
| II · Workflow and Agent Control | What computes $\delta$ — code, the model or a person — and what follows for recovery and the choice of model per stage? | Workflow Orchestration: Fixed vs. Discovered Paths |
| III · Activation and Continuation | What event causes the machine to enter a state at all, and once inside a run, when does it continue on its own rather than wait? | Proactive & Scheduled Execution |
| IV · Frameworks and Implementations | Where do frameworks write the transition function $\delta$, and which framework is not described by one control state? | Frameworks as State Machines |
Orchestration is the control of which model call, tool call or other step is executed next; reduced to that control — its states and the rule that selects the next one — every orchestration mechanism is a finite-state machine $M=(S,\Sigma,\delta,s_0,F)$ — states, events, transition function, start state and terminal states — and every later part specifies a different component of it. This part defines the formalism: the states and the transition function, their matrix representation, and the context history the transition reads.
The control skeleton of the agent loop is a finite-state machine (FSM) — the classical model of computation with finitely many states (Hopcroft, Motwani & Ullman, 2006): $$M=(S,\;\Sigma,\;\delta,\;s_0,\;F),$$
AwaitUser, ModelCall, ToolCall, Done);AwaitUser, does not occur;AwaitUser); $F\subseteq S$ — the terminal states, in
which a run ends (here $F=\{\texttt{Done}\}$).For the agent loop, $S$, $s_0$ and $F$ stay as drawn; what varies is $\delta$ — what computes it,
and what event triggers it. In a hand-written loop $\delta$ is the if/else that inspects the model's last
reply: a reply that contains a tool request leads to ToolCall, any other reply to Done. The
branch is a line of code, but the string it tests was written by the model — code and model already
share the computation of $\delta$.
Once the states are numbered, the transition graph becomes a matrix. For $S=\{s_1,\dots,s_n\}$ the transition matrix
$T$ is the $n\times n$ adjacency matrix of the diagram: row $i$ is the current state, column $j$ the
next one, and $T_{ij}=1$ when some event $\sigma$ leads from $s_i$ to $s_j$ — that is,
$\delta(s_i,\sigma)=s_j$ — and $T_{ij}=0$ otherwise. For the machine above, numbered
$s_1=$ AwaitUser, $s_2=$ ModelCall, $s_3=$ ToolCall, $s_4=$ Done:
$$T=\begin{pmatrix} 0&1&0&0\\ 0&0&1&1\\ 0&1&0&0\\ 0&0&0&0 \end{pmatrix}$$
AwaitUser, the run is in ToolCall
or has ended in Done. $T^k$ counts walks of exactly $k$ transitions: reachability within $k$
steps is read off $\sum_{m=1}^{k} T^m$, and a run that has ended in Done appears in no later power,
since row 4 of $T$ is zero.The machine above does not model where its events come from. Whether ModelCall is left by tool request
or by final answer is decided by the model's reply, and the reply is computed from the whole
context — every prompt, reply and tool result so far. StateFlow (Wu et al., 2024), a framework that models LLM
task-solving as a state machine, makes this dependence explicit in the transition function. The event $\sigma$ is
replaced by the context history: with $\Gamma$ the set of possible messages, $\Gamma^*$ is the set
of all finite sequences of messages, and an element $\gamma\in\Gamma^*$ is the list of messages generated so far,
$$\delta : S \times \Gamma^* \to S ,$$
and the running system is described by the pair $(s, \gamma) \in S \times \Gamma^*$ — "a snapshot of a running StateFlow" (§3.1). The control state $s$ ranges over the finite set $S$; the context $\gamma$ grows by one message with every output — a prompt, a model reply, a tool result. The context is in general random: a model reply is sampled from the model's output distribution, so two runs on the same task produce different $\gamma$ and, through $\delta$, different paths through $S$.
The pair $(s, \gamma)$ determines the next state; the control state $s$ alone does not. Because
$\delta$ reads $\gamma$, the same control state ModelCall is left by tool request in one context and
by final answer in another. $T$ therefore records which transitions exist, not which one a run
takes, and reachability read from $T^k$ is an upper bound: if $(T^k)_{ij}=0$, no run reaches $s_j$
from $s_i$ in $k$ transitions, whatever $\gamma$ is.
LangGraph — "a low-level orchestration framework and runtime for building, managing, and deploying long-running, stateful agents" — builds the machine over the pair $(s, \gamma)$ as a graph (LangChain, 2026):
Workflows and agents share the structure of one state machine and differ only in what computes the transition function $\delta$.
if
statements or regular-expression matches — so the input and the code together determine the path
of a run.Hybrid systems such as StateFlow (Wu et al., 2024, §3.2) compute $\delta$ in code on some edges and by the model on others, and a human-in-the-loop step assigns a transition to a person.
A state machine differs from a dataflow graph, which has no control state: in a dataflow graph a step is executed as soon as its inputs are available, whereas in a state machine a step is executed because a transition has selected it.
Whether a system is a dataflow graph (driven by the availability of data), a workflow (a state machine whose $\delta$ is computed in code) or an agent (a state machine whose $\delta$ is computed by the model) determines how it recovers after a crash and whether each step requires a model call.
The standard agent loop is therefore a hybrid of code and model. Of its four transitions, one is
triggered by a person (user message), one is unconditional and executed by code (tool result),
and two — tool request and final answer — leave ModelCall, the only state with more than one
successor, where the model's reply decides between them. Code executes every transition; the
model decides the only one that is a choice. A state with a single successor involves no decision
at all, so what classifies a system is who decides at its branching states, not how many transitions
code carries out.
A workflow is the machine above with $\delta$ computed in code. A workflow orchestrates the model "through predefined code paths"; an agent is a system in which the model "dynamically direct[s] [its] own processes and tool usage" (Schluntz & Zhang, 2024). Workflows are composed from three shapes of the transition graph:
The model is needed only at the transitions whose outcome cannot be specified in advance. A transition computed in code costs no model call and can be tested like any other function; a transition computed by the model is needed only where no rule is known. A system is therefore built mostly as workflow, selectively agentic: code computes $\delta$ wherever a rule is known, and the model computes only the remaining transitions. Graph frameworks such as LangGraph implement exactly this machine.
A sequential workflow — prompt chaining — is a chain of states in which each stage processes the output of the one before (Schluntz & Zhang, 2024). A gate, a test in code on an intermediate result, turns one link of the chain into a branch with an exit:
Every state except the gate has exactly one successor, so the path is fixed except at the gate, and the gate is decided by code.
A parallel workflow — parallelization — runs independent calls concurrently and aggregates their results (Schluntz & Zhang, 2024): in sectioning the calls solve different subtasks, in voting they are repeated attempts at the same task. A fork enters all its successors at once, and a join is the barrier that is left only when every branch has finished:
In the transition matrix $T$ the fork's row has three entries $1$, exactly like the row of a state with three alternative successors, although all three are taken: $T$ records which transitions exist, not whether they are alternatives or run concurrently.
During the fork the machine is in several states at once. A finite-state machine with one current state represents this only as the product of the branch states — one position per branch, their number the product of the branch lengths. A parallel stage is therefore better read as one composite state of the outer machine whose inside is a dataflow graph: each branch step runs when its input is available, and the join is left when all have finished.
A conditional workflow — routing — classifies the input and sends it to a specialised follow-up (Schluntz & Zhang, 2024). The customer-support agent of the LangGraph documentation (LangChain, 2026) routes an e-mail after classifying it:
Classify Intent (classify_intent in the page's code) asks the model for the intent and urgency of
the e-mail and then tests them in code, in the order of the edges above: the model writes the label,
code selects the edge. A dashed edge is selected inside its source node, a solid edge is fixed. The
router is a workflow in its graph and a
hybrid in its computation of $\delta$. Node names and layout follow
the page's figure, the edges follow the code of its nodes. The figure differs from that code: it draws Human Review as two boxes, draws an edge Human Review → Draft Reply that
no node returns, and omits both the branch Classify Intent → Draft Reply and the exit from
Human Review on rejection.
Durable execution is the property that a run survives the failure of the process executing it: after a crash or a restart, the run continues where it stopped, and work already finished is not repeated. A durable engine "keeps one run alive across crashes and restarts" (Pydantic, 2026b). The agent loop as written so far lacks this property, because its only record of progress is the memory of the process that executes it.
The mechanism is to record every result that cannot be recomputed and to re-run everything else (Pydantic, 2026d):
A run with two non-deterministic calls that is interrupted during the second call: the replay reads result 1 from the journal and executes only call 2.
Replay makes a model-computed $\delta$ deterministic up to the interruption. A recorded model reply replaces a new sample, so every transition that reads it takes the same edge as before, and the run retraces its path up to the interruption.
The mechanism imposes four conditions. A recorded call needs an identifier that is the same before and after the restart; a call interrupted before its result was recorded is executed again from its beginning, so a side effect it had already caused — a sent e-mail, a written file — can occur twice; every journal entry must be serialisable; and the deterministic code must issue the same calls in the same order on every execution, so it reads neither the clock nor a random number directly. Durability concerns one run; it is not the storage of a conversation that is continued later — "Durability is not storage" (Pydantic, 2026b).
The data plane of a pipeline is the set of stages that only transform data; the control plane is the set of stages that compute $\delta$ — terms taken from network engineering. The two call for models of different size. Which model runs a stage is a parameter of the stage, not of the workflow. Some stages read bulk material — a hundred search results, a long filing, a directory of source files — and emit a few hundred tokens; others decide — plan, route, choose the next branch — reading little and producing little:
| Data plane | Control plane | |
|---|---|---|
| The stage does | ingest bulk context, emit a short text | plan, route, decide |
| Tokens in | many | few |
| Model needed | small, and it may run locally | large |
| What its failure looks like | a fact never reaches the decision | a wrong decision on good facts |
The control plane computes $\delta$; the data plane never does, but it determines what $\delta$ can read. The model sizes follow from the price: a model call is billed per token read and written, the data-plane stages read most of the tokens, and so a small model there lowers the price of most tokens, while the few tokens on which a decision rests go to the large model.
The text passed between the two planes is a channel — in information theory, a process that maps an input to a possibly distorted output — and a channel can be measured. A control-plane stage can decide only on what the data-plane stage kept. He et al. (2026) treat the data-plane stage — in their terms the compressor, a small language model that reads the raw context $X$ and writes a short text $Z$ for a query $Q$ — as a noisy channel from $X$ to $Z$, and measure it by the conditional mutual information $I(X;Z\mid Q)$ — the reduction in uncertainty about $X$ once $Z$ is known, with $Q$ given throughout. The quantity is computed from the compressor's input and output alone, without running the downstream stages, and it predicts their quality:
A stage therefore becomes testable on its own, in the same way that a schema-validated return value makes a stage checkable: the question "did this summariser lose what the next stage needs?" has an answer that does not require the rest of the graph to be executed.
An activation is the sequence of model calls between an external event and the next yield; activation asks which events start one, continuation how the model decides after each tool call whether it goes on. A transition requires a triggering event as well as a rule, and the rule $\delta$ does not supply the event. A user message is one event source among several — a runtime condition, a timer, a finished background job — and once a run has started, the model decides after each tool call whether the run continues immediately or waits for the next event.
Proactive execution means that the agent starts a run without a user message; the event that
starts it must then come from the runtime. $\delta(s,\sigma)$
determines which state follows $s$ once an event $\sigma$ has occurred; it does not determine when
the machine leaves AwaitUser, and in the loop so far only a user message does that. A
scheduler — often called a heartbeat — supplies the event on a timer, so that the agent can
act unprompted, the proactivity capability (→ L-Intro), for monitoring, follow-ups and
scheduled work. With the timer, the agent becomes an event-driven loop instead of a
request–response component: the runtime, not the user, supplies the event that starts a run.
A scheduler and a background executor are the two event sources that the runtime adds beside the user. The teaching repository Learn Claude Code (shareAI-lab, 2025) implements both in plain Python: a cron scheduler, which starts a run at a time fixed in advance (shareAI-lab, 2025, Cron Scheduler), and background execution, which reports the result of a slow tool call as a later event (shareAI-lab, 2025, Background Tasks).
0 9 * * * is daily at 09:00), and a prompt.agent_lock) prevents a scheduled run from changing the session while
a user's turn is running.A scheduler and a notification queue are one mechanism with two event sources. Both place an event in a queue that the runtime reads when the agent is idle or about to call the model; the scheduler produces the event from the clock, the background executor from a finished job.
In event-driven control flow, the model is called whenever an event arrives, not only after a user message; a timer is then one event source among several. In MemGPT (Packer et al., 2024, §2.4), a system that manages the model's context as an operating system manages memory, inference is triggered by events.
The MemGPT runtime parses each event into a text message, appends it to the context and calls the model (Packer et al., 2024, §2.4):
| Event type | Source | Example in MemGPT |
|---|---|---|
| user message | the user | the ordinary request–response turn in a chat application |
| system message | the runtime | a "memory pressure" warning once the prompt exceeds a warning threshold (e.g. 70 % of the context window), so that the model can move information into working context — a writable section of the prompt — or into archival storage, an external database, before older messages are evicted (§2.2) |
| user interaction | the application | an alert that the user has just logged in, or has finished uploading a document |
| timed event | a scheduler | a run on a regular schedule, letting the agent act "unprompted", "without user intervention" |
Chaining continues a run with a further model call immediately after a tool call; yielding stops it
until the next external event. The model selects between them by a flag, request_heartbeat, which
it sets as an argument of the tool call; after the call has been executed, the runtime reads it
(Packer et al., 2024, §2.4):
Idle, Inference and Execute are AwaitUser, ModelCall and ToolCall of the first machine.
The new edge is Execute → Idle: in the first machine every tool result returns to the model, which
is chaining without a choice. The machine has no Done: a reply to the user returns it to Idle, so
$F$ is empty and the machine runs until the process is stopped.
The word heartbeat names two different things:
request_heartbeat is
an argument the model writes into a tool call to request one more inference immediately after the
call, within the same activation; no clock is involved.This course writes scheduler for the timer and chaining for the flag; in a framework's documentation, the context shows which of the two is meant.
A framework is a library that supplies the agent loop, tool calling and the handling of state as reusable code; it implements the workflow or the agent and adds no third kind of control flow. LangGraph and PydanticAI implement the machine explicitly; LlamaIndex Workflows is a framework whose run has no single control state.
A framework does not change what computes $\delta$: LangGraph and PydanticAI each implement the
machine $M=(S,\Sigma,\delta,s_0,F)$ and differ in where the transition function is written.
PydanticAI runs every agent on the library pydantic-graph, "a generic, type-centric library for
building and running finite state machines in Python"
(Pydantic, 2026a,
2026e).
| Component of $M$ | Hand-written loop | LangGraph | pydantic-graph |
|---|---|---|---|
| control state $s \in S$ | the value of a variable | a node: a function | a node: a class |
| transition function $\delta$ | one if/else |
the edges; a conditional edge is a function of the state record that returns the next node | the run method of each node, which returns the next node |
| transitions that exist ($T_{ij}=1$) | the branches of the if/else |
the declared edges | the return type of run, which lists the possible next nodes |
| context history $\gamma$ | the list of messages | the typed state record, which holds the part of $\gamma$ that its schema keeps | the state object passed to every node; in an agent run it holds the message history |
| terminal states $F$ | Done |
the node END |
the return value End |
| update of $\gamma$ | the loop appends to the list | a node returns an update, which one reducer per key merges into the record | a node changes the state object in place |
| check of the declared transitions | none | compile() rejects an edge that ends at an undeclared node |
build() rejects a node that cannot be reached from the start node |
| persistence of a run | none | a checkpointer stores the state record after every step | none in the library (Pydantic, 2026f); a durable-execution engine is attached instead |
An agent run of PydanticAI is a machine close to the machine of the agent loop. A run
without a tool call visits UserPromptNode → ModelRequestNode → CallToolsNode → End
(Pydantic, 2026a). CallToolsNode is entered after every model reply, including a reply without a
tool request: its run method inspects the reply, executes the requested tools and returns either
ModelRequestNode or End (pydantic-ai 2.24.0, module _agent_graph). The if/else of the
hand-written loop is therefore the body of one node, and the reply it tests is written by the
model: in the framework, as in the hand-written loop, code and model share the computation of $\delta$.
A counter-example: a workflow framework whose run has no single control state. In LlamaIndex
Workflows, a workflow is "an event-driven, step-based way to control the execution flow of an
application" (LlamaIndex, 2026): a step is a function that receives an event and returns one, no
edge is declared, and the runtime delivers each event to every step whose input type accepts it
(function _route_to_accepting_steps in llama-index-workflows 2.23.3). An event in LlamaIndex
Workflows is the typed value passed from one step to the next inside a run, not an external trigger
that starts an activation, as in Event-Driven Control Flow. When two steps accept the
same event type, one event starts both, and the run is in two steps at the same time. Such a run is
not described by one state $s \in S$ of a machine $M=(S,\Sigma,\delta,s_0,F)$: its configuration is
the set of pending events and running steps. The machine exists one level down, in the runtime,
whose control loop maps a tick (a step result, a new event, a timeout) and a broker state — the
event queues and the state of every step — to the next broker state and a list of commands
(function _reduce_tick); that machine is not the graph the user writes.
A workflow is evaluated on the guarantees that orchestration is introduced to give: every run reaches a terminal state, takes only legal transitions, survives a crash of its process, and succeeds on every repetition rather than on average. Each guarantee is a property of the run's trace — the sequence of control states and transitions — and is measured with the instruments for a single agent (→ L-Eval):
| Guarantee | Metric | How it is measured |
|---|---|---|
| Reliability | pass^$k$ — the probability that all $k$ independent runs of a task succeed, averaged over tasks | $k$ runs per task; pass^$k$ penalises the variance that a model-chosen path adds, and whether a given workflow removes that variance is measured, not assumed |
| Legal transitions | violation rate — the share of runs containing a transition outside $\delta$, such as a mutation before the user's confirmation | each run's state sequence checked against the transition matrix $T$ |
| Termination | the share of runs that end in a state of $F$; runs stopped by the budget are counted apart, as failures | the transition counter |
| Durability | the share of runs that complete after the executing process is killed at a random step and restarted, with no journaled call executed twice | the journal compared with the trace |
An outcome metric cannot detect an illegal transition. τ-bench, a benchmark of simulated customer-service conversations against a tool API, grades a run by the final database state, and its authors state that a correct final state is "a necessary but not sufficient condition for a successful episode — e.g. the agent might issue the return without explicit user confirmation, which violates the policy" (Yao et al., 2024). State-Machine Augmented Generation (SMAG), in which the model requests a transition and a state machine decides whether it is legal, closes this gap by construction, and is measured on the same benchmark: with GPT-4o and the rest of the agent unchanged, the success rate on τ-bench retail rises from 71.7 % to 80.3 % (mean of five runs; Wu et al., 2025, Tab. 1).
lecture-Orch-exercises.ipynb — the mathematics: draw a ReAct agent as a finite-state
machine; write its transition matrix $T$; compute the reachable states from $T^k$; mark the
transitions that the model computes.
lecture-Orch-lab.ipynb — a hand-written orchestrator, without a framework: implement a small
state-machine orchestrator for the ReAct agent in plain Python — an explicit state and a δ
dispatch — with a sequential and a parallel (barrier) step.
The frameworks remain in the lecture as illustration: once the machine has been built by hand, the LangGraph and PydanticAI versions above are the same machine with more code around it.
pydantic-graph. A broad survey is outlook material.Peer-reviewed papers
Preprints & other non-peer-reviewed papers
Textbooks & monographs
Blogs, documentation & other web sources
bash tool and a linear history. https://github.com/SWE-agent/mini-swe-agentllama-index-workflows 2.23.3.agent.iter() and AgentRun.next(node) drive the run node by node.pydantic-graph: nodes whose run method returns the next node, edges read from return annotations.graph.iter(); neither graph API stores the state of a run, and the persistence module was removed in version 2.Framework documentation. PydanticAI — https://ai.pydantic.dev/ · LangGraph — https://langchain-ai.github.io/langgraph/ (agent as a state machine; persistence / checkpointing).