Week 2 — The ReAct Loop & Tool Use¶

Agentic AI · B.Sc. Applied Computer Science

How to sample tokens (softmax, temperature, top-$k$/$p$), the ReAct loop that turns a chat model into an agent, message types, and the system prompt / chat template.

Learning objectives¶

After this week you can:

1. Compute softmax and analyse temperature $T$, top-$k$, top-$p$.
2. Explain Chain-of-Thought, describe ReAct, and write the agent execution loop.
3. Explain function calling — tool schemas and how the model requests a tool.
4. Explain the message types UserMessage / AssistantMessage / ToolResultMessage.
5. Explain the system prompt, the chat template, and statelessness.

Recall (W1): the model outputs a distribution over the vocabulary $P(W_t\mid W_{1:t-1})$. This week we open it up — logits → softmax → sample — and then loop with tools.

Sampling: from logits to a token — the softmax¶

An LLM never emits a word directly: the network produces scores (one per vocabulary token), softmax turns them into a distribution, and we sample. The one-step pipeline:

$$ \underbrace{W_{1:t-1}}_{\text{previous tokens (W1)}}\;\xrightarrow{\text{Transformer}}\; \underbrace{\mathbf z\in\mathbb R^{|V|}}_{\text{logits: one score per token}}\; \xrightarrow{\text{softmax}}\; \underbrace{P(W_t\mid W_{1:t-1})}_{\text{distribution over }V}\; \xrightarrow{\text{sample}}\; w_t $$

• The neural network (today: Transformer) reads the previous tokens and outputs one logit $z_i$ per vocabulary token — an unnormalised score. (This is Week 1's "the conditional is computed by a neural net"; its internals are Week 7.)
• softmax turns those logits into the Week 1 probability distribution: $$ P(W_t=i\mid W_{1:t-1})=\operatorname{softmax}(\mathbf z)_i=\frac{e^{z_i}}{\sum_j e^{z_j}} $$
• We then sample the next token $w_t$ from that distribution (or take the most likely one).

Properties: every entry is positive and they sum to $1$ — a valid distribution. (Why? You prove it in the exercises.)

How the next token is sampled¶

The softmax gives a distribution $P(\cdot)$ over the vocabulary $V$. To emit a token:

• Greedy (argmax): take the most probable token $w_t=\arg\max_i P(i)$ — deterministic, but repetitive and prone to loops: model gets stuck in a cycle of repeating the exact same sequence of words or phrases over and over again.
• Random sampling (categorical draw): pick $w_t$ at random, weighted by $P$ — token $i$ with probability $P(i \mid W_{1:t-1})$. A single categorical draw.

In code, that weighted draw is one line:

w_t = np.random.choice(len(p), p=p)   # draw a token, weighted by the probabilities p

Then repeat autoregressively (Week 1's chain rule, one step at a time): append $w_t$, feed $w_{1:t}$ back in, get the next distribution $P$ resp. p, sample again — until an end-of-sequence token.

The decoding strategy (temperature, top-$k$, top-$p$ — next slides) reshapes $P$ before this draw, trading determinism for diversity (see below).

Temperature scaling — the determinism ↔ creativity knob¶

Rescale the logits by temperature $T$ before softmax, changing how peaked the distribution is:

$$ P_T(W_t=i \mid W_{1:t-1} )=\frac{e^{z_i/T}}{\sum_j e^{z_j/T}}, \qquad T>0 $$

• $T\to 0^+$: all mass on $\arg\max_i z_i$ → greedy / deterministic. (setting temperature=0.0 internally bypasses the softmax sampling and executes argmax(logits) directly to prevent a divide-by-zero error.)
• $T=1$: the model's native distribution.
• $T\to\infty$: → uniform (maximally random).

Why agents care:

• low $T$ → reliable, repeatable output — what you want for valid tool calls (this week);
• higher $T$ → diverse output — needed to generate different candidate thoughts in Tree-of-Thoughts (Week 4).

How you then choose among many sampled sequence candidates — beam search vs. sampling, best-of-$N$, self-consistency — is Week 4.

Aside (train vs. inference). The shape of the learned distribution also depends on training (e.g. label smoothing / weight decay flatten it) — but that is baked into the model, distinct from the inference-time temperature knob here.

Truncated sampling — cut the unreliable tail¶

Sampling from the full distribution occasionally draws a very unlikely token from the long tail, which can derail the text. Both methods below zero out the tail, renormalise, then sample:

• Top-$k$ — keep the $k$ highest-probability tokens, drop the rest, renormalise. Fixed count. Simple, but $k$ ignores the distribution's shape: too permissive when the model is confident (one obvious token), too restrictive when it is genuinely uncertain (many plausible tokens).
• Top-$p$ (nucleus sampling; Holtzman et al., 2020) — keep the smallest set — the nucleus — whose cumulative probability $\ge p$, then renormalise. The nucleus *size*** (how many tokens it contains) is **adaptive: few when the distribution is peaked (model confident), many when it is flat. Typical $p \approx 0.9\text{–}0.95$.

Example — probabilities $(0.6,\,0.3,\,0.05,\,0.03,\,0.02)$:

• top-$k{=}2$ → keep $\{0.6, 0.3\}$, renormalise to $\{0.67, 0.33\}$.
• top-$p{=}0.9$ → keep $\{0.6, 0.3\}$ (cumulative $=0.9$); on a flatter distribution the same $p$ would keep more tokens.

In practice these are often combined (e.g. top-$p$ + temperature — temperature reshapes the distribution before truncation). Note the special case greedy = top-$k$ with $k{=}1$ (equivalently $T\to 0$).
(Newer variant: min-$p$ — keep every token within a factor $\texttt{min\_p}$ of the top token's probability, i.e. $P(\text{token}) \ge \texttt{min\_p}\cdot p_{\max}$; the cutoff adapts to the model's confidence.)

Setting them in practice: all these knobs are parametrizable — temperature and top_p via the OpenAI client against Ollama; top_k / min_p via the native ollama library. You try them in lab.

Choosing the knob — determinism vs. creativity by use case¶

Same model, different temperature (and top_p) per job — pick for "how many acceptable answers are there?":

• Reproducibility: for tests / evals / a linter, pin temperature=0.0 (and a fixed seed if the API exposes one) so runs are comparable.
• Tune one knob. Usually leave top_p ≈ 0.9–0.95 and adjust temperature; cranking both at once makes output hard to predict.
• This is only the dial setting — the mechanics are above (T reshapes $P$ before the draw; top-$p$ trims the tail).

Two dials, not one. This is the parameter-level dial. The prompt-level dial — clarity, examples, a role, structure — is the other, and usually the stronger one. Reach for the prompt first, the temperature second.

Decoding strategy ≠ decoding speed¶

Every knob above changes the distribution you sample from: temperature rescales it, top-$k$ and top-$p$ truncate it. Different setting, different text.

A second family makes generation faster while sampling from the same distribution — speculative decoding: a small, cheap model drafts the next few tokens, the real model verifies them all in one pass, and a modified rejection-sampling rule guarantees the emitted tokens are distributed exactly as the real model's (Leviathan et al., 2023; Chen et al., 2023). Reported: 2–3× less wall-clock time, identical outputs.

Why an agent course cares: an agent turn is many generations in a loop, so decode speed multiplies over every step of a ReAct trajectory — but it must never silently change what the model would have said, or your tool calls would depend on your inference settings. The mechanism, the acceptance rate $\alpha$, and how many tokens to draft are W7.

Chain-of-Thought (CoT)¶

From generating to acting. So far we have a fluent text generator and know how to sample from it. An agent must also reason before acting.

Chain-of-Thought (CoT; Wei et al., 2022): prompt the model to emit intermediate reasoning steps before the final answer ("think step by step"). CoT turns a single monolithic prediction into an explicit, multi-step trajectory.

How Chain-of-Thought works in practice¶

At first glance CoT is a prompting trick. Under the hood it changes how much computation the Transformer can spend on a problem:

1. Compute per token — the "scratchpad" effect. An autoregressive Transformer performs a fixed number of operations (layers, attention heads, feed-forward passes) per generated token.

   • Without CoT: the model must solve a multi-step problem inside a single generation step — at the first token of the answer. That frequently exceeds the computational depth of its layers.
   • With CoT: every intermediate token is a scratchpad entry in the context window. Emitting 100 intermediate tokens buys $100\times$ more forward passes before the answer must be committed.

2. Causal attention turns steps into premises. Each token attends only to earlier tokens, so writing out step 1 makes step 2 cheaper: the intermediate result becomes part of the prefix, and the model conditions on its own sub-result instead of re-deriving it.

3. In native reasoning models (e.g. o1, DeepSeek-R1). CoT is no longer a manual prompt technique. Such models are trained — with reinforcement learning (DeepSeek-AI, 2025) — to produce a reasoning trajectory before answering, and to spend a larger "thinking budget" on harder problems. The reasoning is emitted as a separately marked span, which is what lets the runtime treat it differently from the answer.

Open-loop vs. Closed-loop¶

CoT improves the reasoning, but the model still commits to the entire chain before anything happens in the world. The control-theory terms name the difference exactly:

• Open loop — execute a fixed plan, with no feedback from the result. A washing machine runs its programme whether or not the laundry ends up clean.
• Closed loop — each result is measured and fed back, steering the next step. A thermostat observes the temperature it caused and corrects.

Classic CoT is open-loop: the model reasons, but it cannot act on the outside world, gather new data, or notice that step 3 rests on a false premise from step 2. Every fact it uses comes strictly from its parametric memory (training data), so an early error is carried, undetected, to the end.

Acting changes the requirement. The moment a step has an effect — a query, a file write, an email — its outcome is information the model did not have, and the next step ought to depend on it. Feeding that outcome back is the closed loop, and that is the step from CoT to ReAct.

From CoT to ReAct¶

ReAct = Reasoning + Acting (Yao et al., 2023): Extends CoT into a closed-loop feedback cycle. It interleaves CoT-style Thoughts with concrete Actions (tool executions) and environment Observations (tool outputs), repeating until the task is solved:

$$\text{Thought}_k \;\longrightarrow\; \text{Action}_k (\text{tool call}) \;\longrightarrow\; \text{Observation}_k (\text{tool result}) \;\longrightarrow\; \text{Thought}_{k+1} \;\dots$$

ReAct grounds intermediate thoughts in real-world feedback—preventing hallucination drift and enabling recovery when an execution step fails. This interleaving loop forms the operational core of the agent executor.

Implicit vs. explicit planning¶

Two ways to plan. ReAct plans implicitly, just in time: there is no plan object anywhere. The "plan" is whatever the next thought turns out to be, re-derived from the whole history at every turn. The alternative is explicit planning — first produce a plan (a list of steps, a tree, a graph), then execute it, replanning when a step fails.

W3–W5 take the explicit route — decomposition (W3), deliberate search over plans (Tree-of-Thoughts, W4), planning as a decision problem (MDPs, W5). This week stays with the implicit loop, which is what most production agents actually run.

The execution loop¶

history ← [ system_prompt ]                 # created ONCE; persists across turns (the session)

function prompt(user_input):                # called once per user turn
    history.append(user_input)
    loop:
        response ← LLM(history, tools)    # condition on the FULL history so far
        if response has no tool calls:
            history.append(response)             # final answer → stop
            return response.text
        for each call in response.tool_calls:    # a batch of ≥1 calls
            result ← execute(call)               # runtime runs the tool
            history.append(call, result)         # feed the observation back
        # loop again: the next LLM call sees the appended results

Reading it: ← is assignment. history (the conversation) is created once and only ever grows via append — it is never reset. Each user turn calls prompt(), which appends the new message; the loop appends the assistant reply and each tool result. Every LLM(history, …) re-sends the whole growing history.

The agent loop, drawn¶

The pseudocode above as a control-flow graph — the minimal ReAct loop:

The only exit is no tool calls → return; after running tools the loop always returns to the model, which re-reads the appended results. Note who decides to stop: the model does, by emitting no tool call. A richer loop adds a separate Goal reached? test the runtime evaluates — the goal-loop refinement (W13, Sanoja 2026).

Key points

• The step chain is not pre-planned — each turn re-conditions on the whole history. (This is just-in-time planning — one form of the W1 planning capability; deliberate planning — decompose, look ahead, replan — is W3–5.)
• (Recall W1.) The model requests; the runtime (gateway) executes — the security boundary (W12).
• Termination: the model returns text with no tool call.
• History only grows — every turn re-sends the full conversation (statelessness); long runs eventually need compaction (W9).
• A turn may request several tools at once (a batch); all run before the next LLM call.

Note: Commercial AI-Assistants/agents use the same loop, see e.g. for Claude Code:

• https://code.claude.com/docs/en/how-claude-code-works

Function calling: defining & advertising tools¶

The model has no built-in knowledge of available tools — the same base model powers many agents, each with a different toolset, so every call must tell it which tools exist right now (it can't call what it doesn't know).

To let the model use a tool you advertise a schema — name, description, and a JSON-Schema of its parameters — via the tools argument of the model call:

{ "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "parameters": {
      "type": "object",
      "properties": { "city": { "type": "string" } },
      "required": ["city"] } } }

What is fixed, and what is yours. The envelope — which fields exist and how they nest — is fixed by the API you call, not by the model, and it differs between providers. The parameters follow the JSON Schema standard. What you own is the name, the description, and the argument definitions — and the description is what the model reasons over when deciding whether to call. (Concrete envelopes per provider: Exercise 13.)

• The description teaches the model when to use the tool; the schema constrains the arguments it may produce.
• The model replies with message.tool_calls — a list of {id, name, arguments} (arguments is a JSON string).
• The runtime parses it, runs the function, and returns the result tagged with the matching tool_call_id, which pairs result ↔ request — needed when a turn issues several calls.

How the call comes back — two routes. With a tool-calling-tuned model (rung 3 of the W1 ladder) you pass tools=[…] and get a structured tool_calls object back — native function calling, which we use throughout. Rung 3 is what makes that reliable, but it is not a requirement: a model without it can still be driven prompt-based — write the tool descriptions into the prompt text, then parse the call back out of the generated text ("stop-and-parse"). That works with any instruction-following model and is markedly more brittle; you build it in Exercise 14.

(Training is not one fixed recipe either: Toolformer learns tool use self-supervised — the model inserts candidate API calls into ordinary text and keeps only those that reduce its own next-token loss (Schick et al., 2023).)

Specific tools vs. a code tool¶

Sticking to native (JSON) tool calling, the action is always a JSON tool call — but the tool comes in two styles:

• Specific, predefined tools — e.g. get_weather(city), send_email(to, body). One clear capability each; easy to validate and keep safe.
• A general code tool (code as the action — CodeAct; Wang et al., 2024) — a single tool like run_python(code) whose argument is code. Far more expressive (loop, compose several steps, use variables within one action) — but it runs arbitrary code, so it needs strong sandboxing (W12) and is harder to constrain.

Both are the same JSON tool call; the code tool just carries code in arguments. Same ReAct loop — only the tool differs. (Which task suits which, and what the sandbox must block: Exercise 11.)

Two axes, not a menu. Format — how the call comes back: native JSON vs prompt-based (Function calling slide; you build the prompt-based route in Exercise 14). Tool kind — what the tool does: specific vs code tool (this slide). The axes are independent: a code tool can be requested natively or parsed out of free text. Some frameworks (e.g. smolagents) take the code · prompt-based corner — code is the model's own output, parsed from text instead of wrapped in JSON. We stay in the specific · native JSON corner throughout.

Standardising tools: MCP (Model Context Protocol)¶

Definition. MCP is an open protocol / contract by which a server exposes tools, data (resources), and prompt-templates, and an agent's runtime discovers and communicates with them — decoupling tool providers from agents so a tool written once works with any MCP-capable runtime.

So far each tool is hand-wired into our runtime. MCP (Model Context Protocol) is an open standard (2024) that decouples tools from the agent: tools, data, and prompts are exposed by an MCP server, and any MCP-capable client (the agent runtime) can discover and call them — without bespoke glue per tool.

• What it standardises (three server primitives, by who controls them): tools — callable functions the model invokes; resources — data/context the app reads; prompts — reusable prompt templates the user picks (slash-commands / menu items) — not a tool-call format.
• Why it matters: write a tool once, reuse it across agents & providers; grow a large toolset by plugging in servers (files, web, databases, coding tools — the domains in W3).
• Under the hood: still function calling — MCP standardises discovery & transport; the model still emits the same structured tool call as before (the tool_calls schema from the Function calling slide).
• Security: an MCP server is external, possibly untrusted code/data → the gateway's security boundary and prompt-injection defences apply (W12). Ecosystem outlook: W15.

A third, orthogonal axis. MCP standardises how tools are exposed and discovered, not how the model emits a call — an MCP tool is still invoked by ordinary function calling underneath. (How it composes with the other two axes: Exercise 13(c).)

MCP is only a contract. It standardises the connection & discovery (a client can ask a server at runtime what tools it offers — unlike static function-calling), not the quality of what's behind it. A server that returns raw PDFs, or one record at a time, is useless to an agent — the underlying API must be agent-friendly (deterministic filtering / sorting, machine-readable output). Agents don't replace deterministic workflows; they need strong deterministic support underneath (cf. W3: fix the workflow when the how is known).

Takeaway. The model knows nothing about MCP — the runtime does. The model just sees a list of tools and emits an ordinary tool call; the runtime handles discovery, transport (JSON-RPC), routing, and security.

Tool use: the bigger picture¶

Everything in this section is one pattern — Tool Use (a.k.a. function calling). Its loop (Gulli, 2025): define → the model decides → it emits a structured call → the runtime executes → observe the result → the model uses it — which is the ReAct loop with tools.

• A "tool" is broader than a function. It can be an API, a database, a code interpreter, web search, or even another specialised agent — "agents as tools" (→ W11).
• Who executes stays the same (W1, and the Function calling slide): the runtime runs the call — the security boundary (W12). Some managed platforms auto-execute server-side instead, trading that control for convenience.
• Why it matters: tools break the model out of its frozen training data — real-time data, private data, exact calculations, and real-world actions.

Designing tools well: the Agent–Computer Interface (ACI)¶

Definition. The Agent–Computer Interface (ACI) is the interface a tool presents to a model: the tool's name, its description, its argument names and types, and the shape of the result handed back. It is the agent-side counterpart of a human user interface (Schluntz & Zhang, 2024).

These are the fields the Function calling slide attributed to the developer. The call's JSON envelope belongs to the endpoint and is fixed; the content of those fields is a design problem, and it is the whole of the ACI.

A human UI is judged by whether a person can operate it without a manual; an ACI is judged by whether a model can operate it without guessing. Anthropic report investing as much effort in the ACI as in a human UI — on SWE-bench, tuning the tools took more effort than tuning the prompt (Schluntz & Zhang, 2024). Their design rules:

• Give the model room to think — do not force a commitment to rigid structure at the first token. The CoT argument applies directly: computation per token is fixed, and an emitted token cannot be revised. A tool demanding a fully-formed JSON argument immediately forces the model to decide and format in one step, so the first field is fixed before the answer has been worked out, and the remaining fields are then written to stay consistent with that first guess. Room to reason in plain text before the call — or a free-text reasoning field ordered first — buys forward passes to settle the decision before the format locks it in. Reasoning models apply the same principle: thinking first, answer second.
• Use natural formats — prefer what occurs often in training data: Markdown over heavily-escaped JSON, and no overhead the model must produce by hand, such as manual line counts or escaping a code block inside a string. Each such demand spends attention on bookkeeping rather than on the task, and each is an opportunity to malform the call.
• Write the description from the model's perspective — usage, edge cases, input format, boundaries; clear parameter names; an example call. It is a docstring for a competent colleague who cannot ask a follow-up question: this text is all the model has when deciding whether the tool applies.
• Poka-yoke the arguments ("mistake-proofing") — shape arguments so misuse is hard rather than merely forbidden. The standard example is requiring an absolute path: a relative path is correct only if the model tracked the working directory across the whole trajectory, an absolute one cannot fail for that reason. Likewise an enum over a free string, and one unambiguous unit over "a number".
• Test the tool on many inputs and iterate on the mistakes actually observed — the failure modes are empirical, not deducible from the schema.

The mechanism is unchanged. ACI concerns the content of description + parameters, not the call machinery of the Function calling slide. Its effect is measured in call quality: fewer missed calls, fewer malformed arguments, fewer retries.

Workflows vs. agents (start simple). Not every task needs this loop. A workflow (LLM calls on predefined code paths) is more predictable and cheaper; a full agent (the LLM directs its own steps) is reserved for open-ended tasks with verifiable outcomes (Schluntz & Zhang, 2024).

Message types (what the loop stores)¶

W1 introduced the conversation as a list of role-tagged messages — system, user, assistant, tool. Those role strings are what goes on the wire. In code you want a type per role, because each carries different fields — this is what the lab's history holds:

Same three things, named for code rather than for JSON. The new part this week is the pairing: an AssistantMessage may carry several tool_calls, and each needs its own ToolResultMessage carrying the matching tool_call_id.

A concrete trace¶

User asks "What's the weather in Berlin?", with a get_weather tool available.

history: [ system, User("What's the weather in Berlin?") ]

turn 1   LLM(history) → Assistant(tool_calls=[get_weather(city="Berlin")])   # no text yet
         runtime runs get_weather("Berlin") → "12°C, rainy"
         history += Assistant(tool_call) , ToolResult("12°C, rainy")

turn 2   LLM(history) → Assistant("It's 12°C and rainy in Berlin.")          # no tool call → STOP

As typed messages (previous slide), history grew to:

1. UserMessage("What's the weather in Berlin?")
2. AssistantMessage(tool_calls=[{name:"get_weather", arguments:{city:"Berlin"}}])
3. ToolResultMessage(tool_call_id=…, content="12°C, rainy")
4. AssistantMessage("It's 12°C and rainy in Berlin.") ← returned

Two turns, one tool call — and every turn re-sent the whole history.

The next cell runs this same round-trip against a real model.

Why tool_call_id is mandatory¶

In a single turn, the model may issue multiple tool calls in parallel (e.g., get_weather("Berlin") AND get_weather("Paris")). The runtime returns multiple ToolResultMessage objects. The model uses tool_call_id to match each result back to its original request.

System prompt, chat template & statelessness¶

The model has no memory: the entire conversation is re-sent every turn.

• System prompt = standing instructions, sent first every time.
• Messages are role-tagged; a chat template wraps them in special tokens:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
...<|im_end|>
<|im_start|>assistant

This format is ChatML (<|im_start|> … <|im_end|>), used by e.g. Qwen — our lab model qwen2.5 — and originally OpenAI. Other families use different delimiters (Llama, Mistral); see the format note on the previous tool-calling slide.

Tool use adds a tool role for results, and the model's tool request is carried in the assistant turn — the exact tokens are model-specific.

What's really in the context?¶

Each LLM call sends more than the conversation. The loop maintains history — the running conversation (system prompt + all prior turns + this turn's tool calls/results). Around it, the runtime assembles extra context on every call:

context sent to the LLM =
    history              # system prompt + all turns + this turn's tool calls/results (W2)
  + tool schemas         # the advertised tools   (W2)
  + retrieved documents  # RAG results for this query   (W8)
  + long-term memory     # relevant remembered facts  (W9)

• history only grows — so when it gets too large it is compacted (summarised) — W9.
• Tool schemas, RAG, and memory are not in history; the runtime injects them each call — memory & RAG are fetched by similarity (embeddings, W6).
• All of it must fit the context window (W7) — which is exactly why retrieval (W8) and compaction (W9) exist: you cannot append forever.

This is context engineering (W1): deciding what to assemble into the context each call — and, under a finite budget, what to retrieve (W8) or compact (W9) — is the discipline itself.

history is a construction, not a transcript. The runtime chooses what to send: it can prune, merge, or summarise turns (drop a redundant reply, elide a consumed tool output) — as long as it keeps a paired tool_call / tool_result together. The messages you send ≠ the raw conversation.

The context budget¶

The context window (W7) is a fixed token allowance, and every part of the context spends from it: more tool schemas or retrieved documents ⇒ less room for history, and vice-versa.

Because the model is stateless, the whole budget is re-assembled and re-sent on every call — so each token costs latency and money every turn, not once.

Deciding what to include, retrieve, or compact to stay within this budget is a central agent-design problem — hence embeddings / retrieval (W6 / W8) and compaction (W9). It is context engineering (W1) in practice.

The same reasoning governs procedural instructions. Rather than keeping every how-to resident in the system prompt, an agent can file them as skills — directories whose one-line description is all that occupies the context until a task matches it, at which point the full instructions load (W9). Tool schemas, retrieval and skills are three instances of one move: index first, load on demand.

Ordering Matters for Caching (Week 7 Preview):¶

To maximize GPU KV-cache reuse (Prompt Caching), the static parts (System Prompt and Tool Schemas) MUST be placed at the very beginning of the context. Variable data (user messages, retrieved docs) should come after.

In practice / this week's lab¶

• CLI coding agents — Claude Code, Aider — are this loop, with file/shell tools instead of get_weather. A while loop, a growing message list, a json.loads.
• Lab: build your own run_agent() from scratch (no framework) — the loop function at the heart of any agent runtime. Details in week-02-exercises.ipynb.
• Keep ollama serve running; tool calling gets unreliable below ~3B.

Summary¶

• softmax + $T$ / top-$k$ / top-$p$ shape token sampling (determinism ↔ diversity).
• CoT makes reasoning explicit (a token scratchpad); ReAct = reason + act in a loop — just-in-time planning (deliberate planning: W3–5).
• Function calling: advertise a tool schema each call; the model returns tool_calls; the runtime executes and feeds results back. MCP standardises exposing tools.
• The loop stores UserMessage / AssistantMessage / ToolResultMessage in a persistent, growing history.
• Model is stateless → the full context budget is re-assembled & re-sent each turn (motivates memory / RAG / compaction, W8–9).

Next week (W3): breaking a task apart — task decomposition & a coding/web agent.

References¶

• Anthropic (2026). Prompting best practices (Claude platform docs) — prompt-level steering (clarity, examples, roles, XML). https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practices
• Chen, C., et al. (2023). Accelerating Large Language Model Decoding with Speculative Sampling. arXiv:2302.01318. — concurrent formulation; Chinchilla 70 B (→ W7).
• DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948.
• Holtzman, A., et al. (2020). The Curious Case of Neural Text Degeneration. ICLR. arXiv:1904.09751.
• Leviathan, Y., Kalman, M., & Matias, Y. (2023). Fast Inference from Transformers via Speculative Decoding. ICML. arXiv:2211.17192. — faster decoding, unchanged output distribution (→ W7).
• Model Context Protocol (MCP). Open standard — modelcontextprotocol.io.
• Sanoja, D. (2026). What Are AI Agents? JetBrains — the agent control loop (reason → tool? → act → observe → goal?). https://www.jetbrains.com/pages/ai-agents/what-are-ai-agents/
• Schick, T., et al. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS. arXiv:2302.04761.
• Schluntz, E., & Zhang, B. (2024). Building Effective Agents. Anthropic. — tool design (ACI); workflows vs. agents.
• Wang, X., et al. (2024). Executable Code Actions Elicit Better LLM Agents. ICML. arXiv:2402.01030.
• Wei, J., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS. arXiv:2201.11903.
• Yao, S., et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR. arXiv:2210.03629.

Further reading — tool use. Gulli, A. (2025). Agentic Design Patterns, Ch. 5 — Tool Use (Function Calling) (Springer): the tool-use process, agents as tools, code execution, and extensions vs. function calling.

Further reading — MCP. Gulli, A. (2025). Agentic Design Patterns, Ch. 10 — Model Context Protocol (Springer): MCP vs. function calling, client/server architecture, tools · resources · prompts, transport (STDIO / HTTP+SSE), and the agent-friendly-API caveat.