{ "cells": [ { "cell_type": "markdown", "id": "cell00", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "# Lab\n", "\n", "**Agentic AI** · `L-ReAct`\n" ] }, { "cell_type": "markdown", "id": "cell01", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Setup: Ollama, the Client and the Tool Schemas\n", "\n", "**Ollama.** The lab uses the local Ollama server and the model of the lab of → L-Intro, where the\n", "installation is described. Start the server in each session, and pull the model once:\n", "\n", "```bash\n", "ollama serve & # local server on 127.0.0.1:11434 — leave it running while you work\n", "ollama pull qwen3.5:4b\n", "ollama pull qwen2.5:3b # for the tutorial on the ReAct reference implementation only\n", "```\n", "\n", "**Python packages.** `openai`, `numpy` and `pydantic`; the last cell of the tutorial\n", "*Sampling Parameters on a Real Model* and the tutorials\n", "*Chain-of-Thought of a Thinking Model* and\n", "*The ReAct Loop of the Reference Implementation* additionally use the native `ollama` library.\n", "\n", "**Thinking is switched off.** `qwen3.5:4b` is a **thinking model**: by default it generates a\n", "reasoning text before every answer. In one measurement a model call with thinking took about 20 s\n", "and the same call without thinking less than 1 s. The model calls of the lab therefore pass\n", "`reasoning_effort=\"none\"` to the OpenAI-compatible endpoint and `think=False` to the native\n", "library. Only the tutorial\n", "*Chain-of-Thought of a Thinking Model* switches thinking on.\n", "\n", "**The setup cell** below defines the client, the model and the two **tool schemas** (`add`,\n", "`word_count`) that Part B advertises to the model. Execute it first.\n", "\n", "**Tool calling is unreliable below about 3B parameters.** `qwen3.5:4b` or a larger model\n", "(`qwen3.5:9b`) is preferable; for a model that emits no native tool call, prompt-based tool calling, in which the runtime\n", "parses the call from the reply text, is the fallback. A small model that omits a tool call is an observation to record, not a defect of the\n", "notebook: it is the behaviour the lecture predicts." ] }, { "cell_type": "code", "execution_count": null, "id": "cell02", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Run me first — client, model, and the tool SCHEMAS. The tool FUNCTIONS are defined in the\n", "# tutorial \"Tool Functions\".\n", "# MODEL is a thinking model: pass reasoning_effort=\"none\" with every model call (see the setup text).\n", "import json\n", "from openai import OpenAI\n", "client = OpenAI(base_url=\"http://localhost:11434/v1\", api_key=\"ollama\")\n", "MODEL = \"qwen3.5:4b\" # try \"qwen3.5:9b\" for more reliable tool calls (RAM permitting)\n", "#MODEL = \"qwen3.5:9b\"\n", "# Schemas advertised to the model (the JSON envelope + JSON-Schema parameters):\n", "tools = [\n", " {\"type\": \"function\", \"function\": {\"name\": \"add\", \"description\": \"Add two numbers.\",\n", " \"parameters\": {\"type\": \"object\",\n", " \"properties\": {\"a\": {\"type\": \"number\"}, \"b\": {\"type\": \"number\"}},\n", " \"required\": [\"a\", \"b\"]}}},\n", " {\"type\": \"function\", \"function\": {\"name\": \"word_count\", \"description\": \"Count words in a text.\",\n", " \"parameters\": {\"type\": \"object\",\n", " \"properties\": {\"text\": {\"type\": \"string\"}},\n", " \"required\": [\"text\"]}}},\n", "]\n", "# TOOLS_IMPL (the Python functions) is defined in the tutorial \"Tool Functions\" — the model's\n", "# arguments are NOT trusted." ] }, { "cell_type": "markdown", "id": "cell03", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Overview\n", "\n", "**The lab contains no programming task: every code cell is complete.** The work consists of three\n", "steps for each section: read the code, execute the cells in order, and explain what the code does.\n", "Each tutorial ends with questions on its cells, and the two exercises of Part A ask questions about\n", "code that is given.\n", "\n", "| Part | Section | Subject |\n", "|---|---|---|\n", "| **A — Sampling & decoding** | *Tutorial: Temperature, Top-$k$ and Top-$p$* | the softmax with temperature, top-$k$ and top-$p$, implemented with `numpy` and sampled from |\n", "| | [*Sampling Parameters on a Real Model*](#Tutorial:-Sampling-Parameters-on-a-Real-Model) | the same parameters as arguments of a request to a real model |\n", "| | Exercise 1 *(optional)* | **constrained decoding**: a complete masker for a toy grammar |\n", "| | Exercise 2 *(optional)* | a JSON schema requested in the prompt and enforced in the sampler of a real model |\n", "| **B — The ReAct loop & tools** | [*Chain-of-Thought of a Thinking Model*](#Tutorial:-Chain-of-Thought-of-a-Thinking-Model) | the chain of thought of a thinking model, switched on and off |\n", "| | [*Tool Functions*](#Tutorial:-Tool-Functions) | the Python functions behind the two tool schemas |\n", "| | [*Validation of a Tool Call*](#Tutorial:-Validation-of-a-Tool-Call) | the three checks that a tool call passes before a tool is executed |\n", "| | [*Function-Calling Round-Trip*](#Tutorial:-Function-Calling-Round-Trip) | one tool call: request, execution, tool result, answer |\n", "| | [*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) | the function `run_agent()`, which repeats the round-trip until the model answers in text |\n", "| | [*Message Roles and the Pairing Invariant*](#Tutorial:-Message-Roles-and-the-Pairing-Invariant) | the message history, shown on six hand-written histories |\n", "| | [*The Pairing Invariant as a Check*](#Tutorial:-The-Pairing-Invariant-as-a-Check) | a function that reports a violation of the pairing invariant |\n", "| | [*Termination of a Run*](#Tutorial:-Termination-of-a-Run-%28optional%29) *(optional)* | three ways to decide that a run has ended |\n", "| | [*The ReAct Loop of the Reference Implementation*](#Tutorial:-The-ReAct-Loop-of-the-Reference-Implementation) | the loop of Yao et al., with a text prompt and a parser in place of `tool_calls` |\n", "\n", "A section marked *(optional)* extends a topic; the unmarked tutorials are the required work.\n", "\n", "**The central function of the lab is `run_agent()`.** It advertises the tools to the model, detects\n", "`tool_calls` in the reply, executes each call, appends the result with the matching `tool_call_id`,\n", "and repeats until the model answers in plain text. A step cap bounds the run, and a `try/except`\n", "returns the error of a tool to the model instead of ending the run. This loop is the core of an\n", "agent runtime, where it has a name such as `_run_loop()`; the labs that follow extend it\n", "(→ L-Decomp)." ] }, { "cell_type": "markdown", "id": "cell04", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Part A — Sampling & Decoding\n" ] }, { "cell_type": "markdown", "id": "tut1-lead", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Temperature, Top-$k$ and Top-$p$\n", "\n", "**A language model does not return a token: it returns one logit — an unnormalised score — per token\n", "of the vocabulary, and a sampling rule converts the logits into the next token.** The softmax with\n", "**temperature** $T>0$ converts the logits $\\mathbf z$ into a probability distribution,\n", "\n", "$$P_T(i) = \\frac{e^{z_i/T}}{\\sum_j e^{z_j/T}}.$$\n", "\n", "**Top-$k$** keeps the $k$ most probable tokens; **top-$p$** (nucleus sampling) keeps the smallest set\n", "of most probable tokens whose probabilities sum to at least $p$. Both set the probabilities of the\n", "remaining tokens to $0$ and renormalise. The cells below implement the three functions with `numpy`,\n", "then sample 1000 tokens at $T\\in\\{0.2,1.0,2.0\\}$ and print the entropy of the result. Execute them\n", "in order.\n", "\n", "**Entropy.** The [(Shannon) entropy](http://christianherta.de/lehre/dataScience/entropy-informationGain.pdf) of a distribution $p$ measures its uncertainty:\n", "\n", "$$H(p) = -\\sum_i p_i \\log_2 p_i \\quad\\text{(bits).}$$\n", "\n", "It is $0$ when one token has probability $1$, and maximal, $\\log_2 n$, for the uniform distribution\n", "over $n$ tokens. A higher entropy therefore means a flatter distribution and more diverse samples." ] }, { "cell_type": "code", "execution_count": null, "id": "tut1-inputs", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "import numpy as np\n", "\n", "# The inputs: the logits of one sampling step over a toy vocabulary of 20 tokens,\n", "# the temperatures to compare, and the number of tokens sampled per temperature.\n", "rng = np.random.default_rng(0)\n", "logits = rng.normal(size=20)\n", "temperatures = (0.2, 1.0, 2.0)\n", "n_samples = 1000" ] }, { "cell_type": "code", "execution_count": null, "id": "tut1-softmax", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def softmax(z, T=1.0):\n", " \"\"\"Softmax with temperature T: the logits are divided by T before the exponential.\"\"\"\n", " z = np.asarray(z, float) / T\n", " z = z - z.max() # numerical stability: the largest exponent is 0, so exp() cannot overflow\n", " e = np.exp(z)\n", " return e / e.sum()\n", "\n", "# A low T concentrates the probability on the largest logit, a high T flattens the distribution.\n", "for T in temperatures:\n", " p = softmax(logits, T)\n", " print(f\"T={T}: largest probability = {p.max():.3f}, sum = {p.sum():.3f}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "tut1-topk-topp", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def top_k(p, k):\n", " \"\"\"Keep the k most probable tokens, set the others to 0, renormalise.\"\"\"\n", " keep = np.argsort(p)[::-1][:k] # indices of the k largest probabilities\n", " m = np.zeros_like(p)\n", " m[keep] = p[keep]\n", " return m / m.sum()\n", "\n", "def top_p(p, thresh=0.9):\n", " \"\"\"Keep the smallest set of most probable tokens whose probabilities sum to at least thresh.\"\"\"\n", " idx = np.argsort(p)[::-1] # token indices, most probable first\n", " before = np.cumsum(p[idx]) - p[idx] # probability mass in front of each token\n", " keep = idx[before < thresh] # a token is kept while that mass is below thresh\n", " m = np.zeros_like(p)\n", " m[keep] = p[keep]\n", " return m / m.sum()\n", "\n", "# The number of tokens top-k keeps is fixed; the number top-p keeps depends on the distribution.\n", "for T in temperatures:\n", " p = softmax(logits, T)\n", " print(f\"T={T}: top_k(k=5) keeps {np.count_nonzero(top_k(p, 5))} tokens, \"\n", " f\"top_p(0.9) keeps {np.count_nonzero(top_p(p, 0.9))} tokens\")" ] }, { "cell_type": "code", "execution_count": null, "id": "tut1-entropy", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def entropy(p):\n", " \"\"\"Shannon entropy in bits; a token with probability 0 contributes nothing.\"\"\"\n", " p = p[p > 0]\n", " return float(-(p * np.log2(p)).sum())\n", "\n", "# Sample n_samples tokens at each temperature and compare the entropy of the empirical\n", "# distribution (the relative frequencies of the samples) with that of the distribution itself.\n", "print(f\"maximum for {len(logits)} tokens: {np.log2(len(logits)):.3f} bits\")\n", "for T in temperatures:\n", " p = softmax(logits, T)\n", " samples = rng.choice(len(p), size=n_samples, p=p)\n", " empirical = np.bincount(samples, minlength=len(p)) / n_samples\n", " print(f\"T={T}: entropy of p = {entropy(p):.3f} bits, \"\n", " f\"of the {n_samples} samples = {entropy(empirical):.3f} bits\")" ] }, { "cell_type": "markdown", "id": "tut1-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. Why do `top_k` and `top_p` divide by `m.sum()` before they return?" ] }, { "cell_type": "markdown", "id": "tut2-lead", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Sampling Parameters on a Real Model\n", "\n", "**The sampling parameters are arguments of the request: the weights and the prompt stay the same,\n", "and only the rule that draws the next token from the logits changes.** Ollama offers two ways to\n", "call the model, and they expose different parameters:\n", "\n", "- the **OpenAI-compatible endpoint** — the `client` of the setup cell — accepts `temperature` and\n", " `top_p` as arguments of `client.chat.completions.create(...)`;\n", "- the **native `ollama` library** additionally accepts `top_k`, `min_p` and `seed`, passed in an\n", " `options` dictionary. `min_p` keeps the tokens whose probability is at least `min_p` times the\n", " probability of the most probable token; `seed` fixes the random numbers the sampler draws.\n", "\n", "Both take the same `messages` list. The cells below send one prompt several times per setting: a\n", "sweep of the temperature $T\\in\\{0.2,1.0,1.6\\}$, a sweep of top-$p$ $\\in\\{0.5,0.9,1.0\\}$ at\n", "$T=1.0$, and two calls through the native library with the same seed. Execute them in order." ] }, { "cell_type": "code", "execution_count": null, "id": "tut2-inputs", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "# The inputs: one prompt, the values of each sweep, and the number of replies per value.\n", "PROMPT = [{\"role\": \"user\", \"content\": \"Give a one-line startup idea.\"}]\n", "temperatures = (0.0, 0.2, 1.0, 1.6, 100)\n", "top_ps = (0.5, 0.9, 1.0)\n", "n_replies = 3" ] }, { "cell_type": "code", "execution_count": null, "id": "tut2-temperature", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Sweep the temperature. Several replies per value: the diversity of a setting is visible\n", "# only in the differences between replies to the same request.\n", "for T in temperatures:\n", " for _ in range(n_replies):\n", " r = client.chat.completions.create(model=MODEL, messages=PROMPT, temperature=T,\n", " reasoning_effort=\"none\")\n", " print(f\"T={T:<4} {r.choices[0].message.content!r}\")\n", " print()" ] }, { "cell_type": "code", "execution_count": null, "id": "tut2-top-p", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Sweep top-p at the fixed temperature T = 1.0.\n", "for p in top_ps:\n", " for _ in range(n_replies):\n", " r = client.chat.completions.create(model=MODEL, messages=PROMPT,\n", " temperature=1.0, top_p=p, reasoning_effort=\"none\")\n", " print(f\"top_p={p:<4} {r.choices[0].message.content!r}\")\n", " print()" ] }, { "cell_type": "code", "execution_count": null, "id": "tut2-native", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The native ollama library: the same prompt with top_k, min_p and a fixed seed.\n", "# Two calls with identical options, so that the two replies can be compared.\n", "import ollama\n", "\n", "options = {\"temperature\": 1.0, \"top_p\": 0.9, \"top_k\": 40, \"min_p\": 0.05, \"seed\": 0}\n", "for _ in range(2):\n", " r = ollama.chat(model=MODEL, messages=PROMPT, options=options, think=False)\n", " print(f\"seed=0 {r['message']['content']!r}\")" ] }, { "cell_type": "markdown", "id": "tut2-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. How does the diversity of the replies change with $T$, and how does that follow from the entropy\n", " computed in the previous tutorial?\n", "2. What does a lower `top_p` remove from the distribution, and why does `top_p=1.0` change nothing?\n", "3. The two calls with the same seed returned the same reply. Is that reply the most probable one?\n", " What does a fixed seed provide for a test, and what does it not provide?" ] }, { "cell_type": "markdown", "id": "cell09", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Exercise 1 — Read the Masker *(code given, conceptual, optional)*\n", "\n", "The cell below is a complete constrained decoder in miniature: a toy \"model\" that emits random\n", "logits over a 6-token vocabulary, and an FSM for the language `[AB][12];`. **Run it, then answer\n", "the questions** — the code is given; what is being asked for is the mechanism behind it.\n", "\n", "**(a) What language?** Give the regular expression the FSM accepts and list *every* string the\n", "constrained loop can produce. The token `!` gets a perfectly ordinary logit at every step — why\n", "does it never appear in the constrained column?\n", "\n", "**(b) Point at the maths.** Name the line that implements the indicator $\\mathbb{1}[i \\in A(s_t)]$,\n", "the line that performs the renormalisation in $P'$, and the line that performs $\\delta$. Where is\n", "$\\sigma$ built — and why *there* rather than inside `generate`?\n", "\n", "**(c) The empty set.** `ALLOWED[3]` is `[]`. What would `softmax(mask_logits(logits, 3))` return,\n", "and why? The loop never asks, because it breaks first. State what a real implementation must\n", "guarantee about the states it can reach, and what it should do with a grammar that has a reachable\n", "state with no legal continuation.\n", "\n", "**(d) One character per token is a lie.** A real tokenizer has multi-character tokens: a single\n", "token may be `\"A1\"`, or `\";\\n \"`. One token can therefore cross **several** FSM transitions. What\n", "must $A(s)$ mean then, and how would you build $\\sigma$ for a real vocabulary? Why does this make\n", "the index **tokenizer-specific** as well as grammar-specific?\n", "\n", "**(e) Only one way out.** In state `2` exactly one token is legal. What can the runtime **skip**\n", "here — and what must it still do anyway, given that the model is a function of the whole prefix?\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cell10", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "import numpy as np\n", "\n", "# The \"model\": a toy vocabulary, one character per token. \"!\" is legal nowhere.\n", "vocab = {0: \"A\", 1: \"B\", 2: \"1\", 3: \"2\", 4: \";\", 5: \"!\"}\n", "\n", "# The target language [AB][12]; as an FSM: delta(s, token) -> s'\n", "# (a missing key means \"illegal in this state\"; state 3 is accepting)\n", "delta = {0: {\"A\": 1, \"B\": 1},\n", " 1: {\"1\": 2, \"2\": 2},\n", " 2: {\";\": 3},\n", " 3: {}}\n", "\n", "# Compiled ONCE, outside the generation loop: sigma(s) = A(s), as token IDs.\n", "ALLOWED = {s: [i for i, tok in vocab.items() if tok in delta[s]] for s in delta}\n", "print(\"index sigma:\", ALLOWED)\n", "\n", "def mask_logits(logits, state):\n", " \"\"\"1[i in A(s)] in logit space: illegal tokens -> -inf, i.e. probability 0.\"\"\"\n", " masked = np.full_like(logits, -np.inf)\n", " masked[ALLOWED[state]] = logits[ALLOWED[state]] # lookup, not a scan over V\n", " return masked\n", "\n", "def softmax(z):\n", " z = z - z.max()\n", " e = np.exp(z)\n", " return e / e.sum()\n", "\n", "def generate(rng, constrained=True, max_steps=6):\n", " state, out = 0, []\n", " for _ in range(max_steps):\n", " logits = rng.normal(size=len(vocab)) # what the \"model\" wants\n", " if constrained:\n", " logits = mask_logits(logits, state) # what the grammar allows\n", " token = vocab[rng.choice(len(vocab), p=softmax(logits))]\n", " out.append(token)\n", " if constrained:\n", " assert token in delta[state] # the invariant the mask buys\n", " state = delta[state][token] # s_{t+1} = delta(s_t, W_t)\n", " if not delta[state]: # accepting: nothing may follow\n", " break\n", " return \"\".join(out)\n", "\n", "rng = np.random.default_rng(0)\n", "print(\"constrained :\", [generate(rng) for _ in range(8)])\n", "rng = np.random.default_rng(0)\n", "print(\"unconstrained:\", [generate(rng, constrained=False) for _ in range(8)])" ] }, { "cell_type": "markdown", "id": "cell12", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Exercise 2 — Constrained Decoding on the Real Model *(implementation + conceptual, optional)*\n", "\n", "The cell below asks the model for a tool call **twice**: once with the format merely *asked for* in\n", "the prompt, once with the schema *enforced* through `response_format`. Ollama compiles the JSON\n", "Schema into a grammar and masks its sampler — the mechanism of Exercise 1, at $|V| \\approx 150{,}000$.\n", "Note that the prompt asks for a **one-sentence justification first**, then the JSON. Run it a few\n", "times (it is a sampling experiment — the counts move).\n", "\n", "**(a) Report the counts** for both settings and describe the prompt-only outputs that failed. For\n", "each failure say whether it is a **parse** failure (the string is not JSON) or a **content** failure\n", "(it is JSON, but says the wrong thing).\n", "\n", "**(b) What did enforcement cost?** The constrained column is valid by construction — but look for\n", "the one-sentence justification you asked for. Where did it go? Which claim of pen-and-paper Exercise 4(e) does this\n", "confirm?\n", "\n", "**(c) Who enforces, and who only checks?** `ToolCall.model_validate_json(text)` runs on your machine\n", "*after* the string exists; the mask runs in the server's sampler *while* it is produced. State what\n", "each of them does when the answer is \"no\", and what your agent must do in each case. Why can\n", "validation never replace the mask — and why do you still want both? *(Name at least two reasons for\n", "keeping the validator.)*\n", "\n", "**(d) Back to the loop.** The tutorials [*Function-Calling Round-Trip*](#Tutorial:-Function-Calling-Round-Trip) and\n", "[*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) pass `tools=[...]` and read `message.tool_calls`. Given\n", "pen-and-paper Exercise 4 and Exercises 1–2, explain why those calls arrive as schema-valid JSON. Which of the defensive coercions of the tutorial\n", "[*Tool Functions*](#Tutorial:-Tool-Functions) would you keep if the arguments\n", "were schema-enforced, and why?\n", "\n", "> **Setup.** `response_format` with a JSON *schema* needs an Ollama build with structured-output\n", "> support (tested with 0.32.5); an older one accepts only `{\"type\": \"json_object\"}`, which forces\n", "> *some* JSON but not *your* schema — a useful third column if you want it.\n", "\n", "> **Beyond this cell.** The same schema can be enforced *client-side* over a local model with\n", "> **Outlines** (`outlines.generate.json(model, ToolCall)`), the paper's own library, or with\n", "> llama.cpp **GBNF** grammars and **XGrammar** in vLLM. The API used here is the same idea reached\n", "> through the server. `response_format` is the OpenAI-compatible spelling; the native `ollama`\n", "> library calls it `format=ToolCall.model_json_schema()`." ] }, { "cell_type": "code", "execution_count": null, "id": "cell13", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "from typing import Literal\n", "from pydantic import BaseModel\n", "\n", "# `client` and `MODEL` are those of the setup cell.\n", "\n", "class ToolCall(BaseModel): # the target language, written as a schema\n", " tool_name: Literal[\"add\", \"word_count\"]\n", " arguments: dict\n", "\n", "SCHEMA = {\"type\": \"json_schema\",\n", " \"json_schema\": {\"name\": \"tool_call\", \"schema\": ToolCall.model_json_schema()}}\n", "\n", "TASK = (\"Which tool fits the task, and with which arguments? Tools: add(a, b), word_count(text). \"\n", " \"Task: add 19 and 23. First explain your choice in one sentence, \"\n", " 'then output the call as JSON {\"tool_name\": ..., \"arguments\": {...}}.')\n", "\n", "def trial(constrained, n=4, T=1.0):\n", " kw = {\"response_format\": SCHEMA} if constrained else {}\n", " valid = 0\n", " for _ in range(n):\n", " r = client.chat.completions.create(model=MODEL, temperature=T, reasoning_effort=\"none\",\n", " messages=[{\"role\": \"user\", \"content\": TASK}], **kw)\n", " text = r.choices[0].message.content\n", " try:\n", " ToolCall.model_validate_json(text) # client-side CHECK, not enforcement\n", " valid += 1; verdict = \"ok \"\n", " except Exception:\n", " verdict = \"FAIL\"\n", " print(f\" {verdict} {text[:100]!r}\")\n", " print(f\" -> {valid}/{n} parse as a ToolCall\\n\")\n", "\n", "print(\"prompt only (the format is asked for):\"); trial(constrained=False)\n", "print(\"constrained (the format is enforced):\"); trial(constrained=True)" ] }, { "cell_type": "markdown", "id": "cell15", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Part B — The ReAct Loop & Tools\n", "\n", "The tutorials of this part use `client`, `MODEL` and the **tool schemas** `tools` of the setup cell.\n", "The tool *functions* are defined in the tutorial [*Tool Functions*](#Tutorial:-Tool-Functions). The cells of\n", "Part B are executed in order: a later tutorial uses the functions that an earlier one defines." ] }, { "cell_type": "markdown", "id": "tut-cot-lead", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Chain-of-Thought of a Thinking Model\n", "\n", "**A chain of thought is the text of intermediate reasoning steps that a model writes before its\n", "answer ([Wei et al., 2022](https://arxiv.org/html/2201.11903v6)). A thinking model, also called a\n", "reasoning model, is trained to write this text without an exemplar in the prompt, and the server\n", "returns the text in a field of its own, separate from the answer.** `qwen3.5:4b`, the model of the\n", "setup cell, is a thinking model. The request states whether the model thinks, and the reply\n", "carries the two texts in two fields:\n", "\n", "| | Native `ollama` library | OpenAI-compatible endpoint (`client`) |\n", "|---|---|---|\n", "| Thinking on | `think=True` | the default |\n", "| Thinking off | `think=False` | `reasoning_effort=\"none\"` |\n", "| Reasoning text | `message.thinking` | `message.reasoning` |\n", "| Answer | `message.content` | `message.content` |\n", "| Number of generated tokens | `eval_count` | `usage.completion_tokens` |\n", "\n", "**The reasoning text consists of generated tokens.** The model samples it token by token, as it\n", "samples the answer, and generates it before the first token of the answer. The reasoning text is\n", "therefore part of the context when the answer is generated, and its tokens are counted in the\n", "number of generated tokens of the reply.\n", "\n", "The cells below send one arithmetic question, whose answer Python recomputes: through the native\n", "library with thinking switched on, then with thinking switched off, and finally in both variants\n", "through the OpenAI-compatible endpoint. Execute them in order." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-cot-inputs", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "# The inputs: one question whose answer can be recomputed, and the sampling options of every request.\n", "QUESTION = [{\"role\": \"user\", \"content\": \"What is 17 * 23 + 46 * 12? Reply with the number only.\"}]\n", "OPTIONS = {\"temperature\": 0, \"seed\": 0} # greedy search, so that a repeated request is comparable\n", "\n", "print(\"recomputed in Python:\", 17 * 23 + 46 * 12)" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-cot-think", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "import ollama\n", "\n", "def ask(messages, think):\n", " \"\"\"Send one request through the native library and print the reply while it is generated.\n", " Returns (reasoning text, answer, number of generated tokens).\"\"\"\n", " reasoning, answer, n_tokens = \"\", \"\", 0\n", " stream = ollama.chat(model=MODEL, messages=messages, think=think, options=OPTIONS, stream=True)\n", " for chunk in stream:\n", " if chunk.message.thinking: # a piece of the reasoning text\n", " if not reasoning:\n", " print(\"--- reasoning text (message.thinking)\")\n", " reasoning += chunk.message.thinking\n", " print(chunk.message.thinking, end=\"\", flush=True)\n", " if chunk.message.content: # a piece of the answer\n", " if not answer:\n", " print((\"\\n\" if reasoning else \"\") + \"--- answer (message.content)\")\n", " answer += chunk.message.content\n", " print(chunk.message.content, end=\"\", flush=True)\n", " if chunk.done: # the last chunk carries the token counts\n", " n_tokens = chunk.eval_count\n", " print(f\"\\n--- generated tokens: {n_tokens}\")\n", " return reasoning, answer, n_tokens\n", "\n", "# Thinking switched on: the model writes its reasoning text first, then the answer.\n", "reasoning, answer_think, n_think = ask(QUESTION, think=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-cot-direct", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The same request with thinking switched off: the answer is the first text the model generates.\n", "_, answer_direct, n_direct = ask(QUESTION, think=False)\n", "\n", "print()\n", "print(f\"think=True : answer {answer_think.strip()!r} after {n_think} generated tokens, \"\n", " f\"{len(reasoning)} characters of reasoning text\")\n", "print(f\"think=False: answer {answer_direct.strip()!r} after {n_direct} generated tokens\")" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-cot-openai", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The OpenAI-compatible endpoint: the same model through the `client` of the setup cell.\n", "# The reasoning text is the field `reasoning` of the assistant message, beside `content`;\n", "# `reasoning_effort=\"none\"` switches thinking off.\n", "for effort in (None, \"none\"):\n", " extra = {} if effort is None else {\"reasoning_effort\": effort}\n", " r = client.chat.completions.create(model=MODEL, messages=QUESTION, temperature=0, **extra)\n", " m = r.choices[0].message\n", " text = getattr(m, \"reasoning\", None) or \"\"\n", " print(f\"reasoning_effort={effort!r}\")\n", " print(f\" message.reasoning: {len(text)} characters, beginning {text[:60]!r}\")\n", " print(f\" message.content : {m.content!r}\")\n", " print(f\" completion_tokens: {r.usage.completion_tokens}\")" ] }, { "cell_type": "markdown", "id": "tut-cot-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. The first cell recomputes $17\\cdot23+46\\cdot12$ in Python. Which of the two requests through the\n", " native library returns that value? How many tokens does the model generate in each of the two\n", " requests, and how many of them precede the first token of the answer?\n", "2. A Transformer executes one forward pass for each generated token. Explain the two answers from\n", " this fact: how many forward passes precede the first token of the answer in each request, and\n", " which intermediate results does the context contain at that token?\n", "3. No program executed a step of the reasoning text. Where does each intermediate result come from,\n", " and what happens to an error in one of them? What does the model have to return so that a\n", " product is computed instead of generated, and how does the result reach the model?\n", "4. A program that calls the OpenAI-compatible endpoint reads only `message.content`. What does it\n", " receive in each of the two calls of the last cell, and how many tokens does each call generate?\n", " The loop `run_agent` of the tutorial\n", " [*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) calls the model several times in one run. State the consequence of a\n", " thinking model for the duration of a run, and the argument that removes it." ] }, { "cell_type": "markdown", "id": "cell16", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Tool Functions\n", "\n", "**A tool function is the Python function that the runtime executes when the model requests a tool.\n", "The tool schema of the setup cell describes the function to the model; the function itself is never\n", "sent to the model.** The dictionary `TOOLS_IMPL` maps the name of each advertised tool to its\n", "function, and the runtime selects the function by the name in the tool call.\n", "\n", "**The arguments of a tool call are not guaranteed to match the schema.** The model may send a number\n", "as a string, omit a field, or send `null`. The two functions therefore **coerce** their arguments:\n", "they convert each argument to the expected type where a conversion exists, as `float(\"19\")` gives\n", "`19.0`. A missing argument and the value `None` have no conversion. The function then raises an\n", "exception, and the loop of the tutorial [*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) catches it.\n", "\n", "The cells define the two functions and the dictionary, call the functions with arguments of the\n", "declared type and with a number sent as a string, and call `add` with `None`." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-tools-def", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "def add(a, b):\n", " \"\"\"Add two numbers. float() coerces: the model may send a number as a string,\n", " and the result of an earlier tool call is a string as well.\"\"\"\n", " s = float(a) + float(b)\n", " return int(s) if s.is_integer() else s # 42, not 42.0\n", "\n", "def word_count(text):\n", " \"\"\"Count the words of a text. str() coerces an argument that is not a string.\"\"\"\n", " return len(str(text).split())\n", "\n", "# The name in a tool call selects the function.\n", "TOOLS_IMPL = {\"add\": add, \"word_count\": word_count}" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-tools-calls", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Arguments of the declared type, and a number that arrives as a string.\n", "print(add(4821, 3907)) # 8728\n", "print(add(\"19\", 23)) # 42\n", "print(word_count(\"the quick brown fox jumps\")) # 5" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-tools-none", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# An argument without a conversion: the function raises.\n", "try:\n", " add(4821, None)\n", "except TypeError as e:\n", " print(\"TypeError:\", e)" ] }, { "cell_type": "markdown", "id": "tut-tools-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. The setup cell defines `tools`, this tutorial defines `TOOLS_IMPL`. Which of the two does the\n", " model receive, and which does the runtime execute? What happens for a name that occurs in `tools`\n", " and not in `TOOLS_IMPL`?\n", "2. `add` converts both arguments with `float`. Which arguments does the conversion accept that the\n", " schema type `number` excludes? Why does the function return `42` and not `42.0`?\n", "3. `word_count` applies `str` to its argument. What does `word_count(42)` return, and does the\n", " result answer a question that a user could have asked?\n", "4. `add(4821, None)` raises a `TypeError`. Why does the function not replace `None` by a default\n", " value such as `0`?" ] }, { "cell_type": "markdown", "id": "cell18", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Validation of a Tool Call\n", "\n", "**Validation of a tool call is the sequence of mechanical checks that the runtime applies to the\n", "request of the model before any tool code is executed.** The request is a string that a sampler\n", "produced. Three checks establish that it is **well-formed**:\n", "\n", "1. **valid JSON** — the argument string can be parsed;\n", "2. **tool existence** — the requested name is the name of an advertised tool;\n", "3. **schema conformity** — the arguments match the advertised signature: the required fields are\n", " present, the types are those of the schema, and an `enum` is respected.\n", "\n", "**A well-formed call is not necessarily a permitted call.** Whether this agent may execute a\n", "well-formed call is a question of **policy**, the set of rules that state which calls an agent is\n", "allowed to make. Policy is checked at a different place (→ L-Gateway).\n", "\n", "**The code.** `validate_call(name, raw_arguments, tools)` implements the three checks by hand from\n", "the JSON-Schema fragment under `parameters`, without the libraries `jsonschema` and Pydantic, so\n", "that the work of those libraries is visible. It returns `(True, arguments)` or `(False, message)`.\n", "The second cell applies it to four faulty calls. In the third cell, `execute` places the validator\n", "in front of the tool functions of the tutorial [*Tool Functions*](#Tutorial:-Tool-Functions)." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-valid-def", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "import json\n", "\n", "# JSON-Schema type name -> Python type(s) accepted for it.\n", "SCHEMA_TYPES = {\"string\": str, \"integer\": int, \"number\": (int, float), \"boolean\": bool,\n", " \"object\": dict, \"array\": list}\n", "# In Python, bool is a subclass of int: True passes \"number\" and \"integer\" here.\n", "\n", "def validate_call(name, raw_arguments, tools):\n", " \"\"\"-> (True, args) | (False, message for the MODEL).\"\"\"\n", " # Check 2, tool existence. It is executed first: it selects the schema that check 3 needs.\n", " spec = next((t[\"function\"] for t in tools if t[\"function\"][\"name\"] == name), None)\n", " if spec is None:\n", " known = \", \".join(t[\"function\"][\"name\"] for t in tools)\n", " return False, f\"no tool named {name!r}. Available tools: {known}.\"\n", " # Check 1, valid JSON.\n", " try:\n", " args = json.loads(raw_arguments)\n", " except json.JSONDecodeError as e:\n", " return False, f\"{name}: arguments are not valid JSON ({e.msg} at position {e.pos}).\"\n", " if not isinstance(args, dict):\n", " return False, f\"{name}: arguments must be a JSON object, got {type(args).__name__}.\"\n", " # Check 3, schema conformity: required fields, then name, type and enum of each argument.\n", " params = spec.get(\"parameters\", {})\n", " props, required = params.get(\"properties\", {}), params.get(\"required\", [])\n", " for key in required:\n", " if key not in args:\n", " want = props.get(key, {}).get(\"type\", \"?\")\n", " return False, f\"{name}: missing required argument {key!r} ({want}).\"\n", " for key, val in args.items():\n", " if key not in props:\n", " return False, f\"{name}: unknown argument {key!r}. Expected: {', '.join(props)}.\"\n", " want = props[key].get(\"type\")\n", " if want in SCHEMA_TYPES and not isinstance(val, SCHEMA_TYPES[want]):\n", " return False, (f\"{name}: argument {key!r} must be of type {want}, \"\n", " f\"got {type(val).__name__} ({val!r}).\")\n", " if \"enum\" in props[key] and val not in props[key][\"enum\"]:\n", " return False, f\"{name}: {key!r} must be one of {props[key]['enum']}, got {val!r}.\"\n", " return True, args" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-valid-faulty", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Four faulty calls: (i) to (iv), in this order.\n", "faulty_calls = [(\"add\", '{\"a\": 1, \"b\": '),\n", " (\"rm_rf\", '{\"path\": \"/\"}'),\n", " (\"add\", '{\"a\": 1}'),\n", " (\"word_count\", '{\"text\": 42}')]\n", "\n", "for name, raw in faulty_calls:\n", " print(f\"{name:<11}\", validate_call(name, raw, tools))" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-valid-execute", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def execute(name, raw_arguments):\n", " \"\"\"Validate, then execute. A failure is returned as the tool result; nothing is raised.\"\"\"\n", " ok, value = validate_call(name, raw_arguments, tools)\n", " if not ok:\n", " return f\"ERROR: {value}\" # the model reads this text and can correct the call\n", " try:\n", " return str(TOOLS_IMPL[name](**value))\n", " except Exception as e: # the tool itself can still fail (I/O, division, ...)\n", " return f\"ERROR: {type(e).__name__}: {e}\"\n", "\n", "print(execute(\"add\", '{\"a\": 19, \"b\": 23}'))\n", "print(execute(\"add\", '{\"a\": 19}'))" ] }, { "cell_type": "markdown", "id": "tut-valid-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. In which order does `validate_call` execute the three checks? Why can the function not report a\n", " schema error before the existence check?\n", "2. Which check rejects each of the four faulty calls (i) to (iv)?\n", "3. Compare the message `\"ValidationError\"` with `\"add: missing required argument 'b' (number)\"`.\n", " Which information does the second message give to the model? Why does `execute` return a failed\n", " validation as a string and not raise an exception?\n", "4. Call (iv) is rejected, although `word_count` coerces a non-string argument and returns a result.\n", " Give one argument for accepting the call and one for rejecting it, and name the cost of the\n", " rejection.\n", "5. The comment below `SCHEMA_TYPES` states that `True` passes the check for `\"number\"`. Why, and\n", " what does a complete validator do about it?\n", "6. Give one call of a tool `read_file(path)` and one call of a tool `send_email(to, body)` that pass\n", " all three checks and must still be refused. At which place is that refusal decided, and why is\n", " the place not the schema?" ] }, { "cell_type": "markdown", "id": "cell20", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Function-Calling Round-Trip\n", "\n", "**A function-calling round-trip is one complete exchange in which the model requests a tool, the\n", "runtime executes it, and the model formulates its answer from the result. It takes two model\n", "calls.**\n", "\n", "1. The runtime sends the user message together with the tool schemas.\n", "2. The model replies with an `assistant` message whose field `tool_calls` names a tool and its\n", " arguments. The arguments are a string that holds JSON.\n", "3. The runtime parses the arguments, executes the tool function, and appends the result as a `tool`\n", " message with the `tool_call_id` of the call.\n", "4. The runtime sends the extended history, and the model replies with text.\n", "\n", "`word_count_via_tool` performs these four steps for the question *\"How many words are in: …\"* and\n", "prints the tool call and the tool result. It uses `TOOLS_IMPL` of the tutorial\n", "[*Tool Functions*](#Tutorial:-Tool-Functions).\n", "\n", "**Comparison without a tool.** The second cell sends the question about a sentence of 15 words\n", "once without `tools` and once through the round-trip. Without a tool, the number in the reply is\n", "generated text; with the tool, the number is computed by `word_count`, and the model only has to\n", "report it." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-round-def", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "def word_count_via_tool(text):\n", " \"\"\"One function-calling round-trip: model -> tool call -> execution -> tool result -> answer.\"\"\"\n", " msgs = [{\"role\": \"user\", \"content\": f\"How many words are in: {text}\"}]\n", " # Model call 1: the request carries the user message and the tool schemas.\n", " r = client.chat.completions.create(model=MODEL, messages=msgs, tools=tools, reasoning_effort=\"none\")\n", " m = r.choices[0].message\n", " if not m.tool_calls: # the model answered without a tool call\n", " return m.content\n", " msgs.append(m) # the assistant message that holds the tool call\n", " call = m.tool_calls[0]\n", " # The runtime executes the call. `arguments` is a STRING that holds JSON.\n", " result = TOOLS_IMPL[call.function.name](**json.loads(call.function.arguments))\n", " print(f\" tool call : {call.function.name}({call.function.arguments})\")\n", " print(f\" tool result: {result}\")\n", " msgs.append({\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": str(result)})\n", " # Model call 2: the model formulates the answer from the tool result.\n", " r = client.chat.completions.create(model=MODEL, messages=msgs, tools=tools, reasoning_effort=\"none\")\n", " return r.choices[0].message.content\n", "\n", "print(word_count_via_tool(\"the quick brown fox jumps\")) # e.g. \"There are 5 words.\"" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-round-compare", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The same question about a sentence of 15 words: computed in Python, asked without a tool,\n", "# and asked through the round-trip.\n", "long = \"the quick brown fox jumps over the lazy dog near the river bank at dawn\"\n", "no_tool = client.chat.completions.create(model=MODEL, reasoning_effort=\"none\", messages=[\n", " {\"role\": \"user\", \"content\": f\"How many words are in: {long}\"}]).choices[0].message.content\n", "\n", "print(\"exact ->\", word_count(long), \"words\") # computed by Python: 15\n", "print(\"no tool ->\", no_tool) # generated text\n", "print(\"with tool ->\", word_count_via_tool(long)) # generated from the tool result" ] }, { "cell_type": "markdown", "id": "tut-round-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. How many model calls does `word_count_via_tool` make? Which messages does the second request\n", " contain, and which component wrote each of them?\n", "2. Which type does `call.function.arguments` have? Which expression converts it, and what would\n", " `TOOLS_IMPL[...](**call.function.arguments)` do without the conversion?\n", "3. The function appends `m` before it appends the `tool` message. Why is the `assistant` message\n", " needed in the second request, and which field connects the `tool` message to the call?\n", "4. The function returns `m.content` if the reply has no `tool_calls`. Why can this case occur\n", " although the request contains `tools`, and on what does the returned number then rest?\n", "5. The second cell prints three lines. Which of the three numbers are computed by Python, and which\n", " are generated by the model? Why can the line `with tool` contain a wrong number although the tool\n", " returned the correct one?" ] }, { "cell_type": "markdown", "id": "cell22", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: The ReAct Loop with Tool Calls\n", "\n", "**The ReAct loop repeats the function-calling round-trip until the model answers without a tool\n", "call. Each iteration is one model call, followed by the execution of every tool call in the\n", "reply.** `run_agent(user_input)` keeps one list `messages` for the whole run. The list begins with\n", "the `system` and the `user` message, and each iteration appends the `assistant` message and one\n", "`tool` message per call.\n", "\n", "**Two guards make the runtime responsible for safe execution, not the model.**\n", "\n", "- A `try/except` encloses each tool execution. If the function raises, for instance on an argument\n", " that cannot be coerced (tutorial [*Tool Functions*](#Tutorial:-Tool-Functions)), the error text is\n", " appended as the tool result, and the model can send a corrected call.\n", "- A **step cap** `max_steps` ends a run in which the model continues to request tools.\n", "\n", "**A tool call is necessary for a grounded answer and not sufficient.** The tool returns the exact\n", "value, and the model then has to report it. A weak model can call the tool and state a different\n", "value in its final sentence. Where the value has to be dependable, it is read from the `tool`\n", "message and not from the text of the model; a larger model (`qwen3.5:9b`) reduces the fault and\n", "does not exclude it." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-loop-def", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "def run_agent(user_input, system=\"You are a helpful assistant.\", max_steps=6, verbose=False):\n", " messages = [{\"role\": \"system\", \"content\": system},\n", " {\"role\": \"user\", \"content\": user_input}] # one list for the whole run\n", " for _ in range(max_steps): # step cap: the run is bounded\n", " r = client.chat.completions.create(model=MODEL, messages=messages, tools=tools,\n", " reasoning_effort=\"none\")\n", " m = r.choices[0].message\n", " if not m.tool_calls: # no tool call: the reply is the answer\n", " if verbose:\n", " print(\"messages\", messages)\n", " return m.content\n", " messages.append(m) # the assistant message that holds the tool calls\n", " for call in m.tool_calls:\n", " try: # the arguments of the model are not trusted\n", " out = TOOLS_IMPL[call.function.name](**json.loads(call.function.arguments))\n", " except Exception as e: # the error text becomes the tool result\n", " out = f\"ERROR: {e}\"\n", " messages.append({\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": str(out)})\n", " return \"[stopped: max steps reached]\" # the step cap was reached\n", "\n", "print(run_agent(\"Add 19 and 23, then say the result in a sentence.\"))" ] }, { "cell_type": "markdown", "id": "tut-loop-large", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Large numbers show two independent faults.** The next cell adds a number of 20 digits and a\n", "number of 21 digits in three ways: in Python, by a single model call without tools, and by\n", "`run_agent`.\n", "\n", "1. **Precision, in the tool function.** `add` converts its arguments with `float`. A `float` holds\n", " 15 to 16 significant decimal digits, so the sum that `add` returns differs from the exact sum\n", " `i1 + i2`, which Python computes with integers of unlimited length.\n", "2. **Transcription, in the model.** The model has to copy both operands into its tool call. In a\n", " long digit string it can change digits, insert thousands separators\n", " (`48,828,287,837,983,875,000`), round, or append a decimal part (`…092.8`). The tool then\n", " receives numbers that differ from those of the task." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-loop-compare", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "i1, i2 = 48828287837983872631, 763399939933907673738\n", "task = f\"Add {i1} and {i2}, then give the total in a sentence.\"\n", "\n", "# A single model call without tools and without a loop.\n", "direct = client.chat.completions.create(model=MODEL, messages=[{\"role\": \"user\", \"content\": task}],\n", " reasoning_effort=\"none\")\n", "\n", "print(\"exact ->\", i1 + i2) # Python integers: exact\n", "print(\"add() ->\", add(i1, i2)) # the tool function: float arithmetic\n", "print(\"no tool ->\", direct.choices[0].message.content) # generated text\n", "print(\"with loop ->\", run_agent(task)) # generated from the tool result" ] }, { "cell_type": "markdown", "id": "tut-loop-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. Name the line of `run_agent` for each of the four steps: the model call, the termination test,\n", " the execution of a tool, and the return of the result to the model. Which messages does one\n", " iteration append if the reply contains two tool calls?\n", "2. What does the model receive if a tool function raises an exception? What does the caller of\n", " `run_agent` receive if the model requests a tool in each of the `max_steps` iterations?\n", "3. The lines `exact` and `add()` of the last cell differ. Why? Which change to `add` makes the\n", " function exact for integers?\n", "4. The line `with loop` can contain a wrong sum although `add` was called. Name the two places at\n", " which the model can introduce the fault. From which message would a runtime read the dependable\n", " value?\n", "5. `run_agent` returns the text of every reply without `tool_calls` as the answer. Describe one\n", " reply without a tool call that is not an answer to the task." ] }, { "cell_type": "markdown", "id": "tut3-lead", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Message Roles and the Pairing Invariant\n", "\n", "**The message history is the list of role-tagged messages that the runtime sends to the model on\n", "every call.** A message has one of four roles: `system`, `user`, `assistant` or `tool`. The model\n", "produces the `assistant` messages only; the runtime writes the messages of the other three roles.\n", "\n", "**The pairing invariant** links two of the roles: an `assistant` message may carry **several**\n", "`tool_calls`, and each call is answered by **exactly one** `tool` message that carries the matching\n", "`tool_call_id`.\n", "\n", "The tutorial writes two histories by hand — one tool call, then two tool calls in one `assistant`\n", "message — and sends each to the model. It then sends four faulty variants of the second history and\n", "prints the reply to each. The tool `get_weather(city)` is not implemented: its results are written\n", "into the history directly, because the subject is the content of the history, not the execution of a\n", "tool." ] }, { "cell_type": "code", "execution_count": null, "id": "tut3-inputs", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "# The inputs: the history of a run with ONE tool call, up to the tool result.\n", "# The runtime wrote the messages 0, 1 and 3; the model produced message 2.\n", "def tool_call(call_id, city):\n", " \"\"\"One entry of an assistant message's tool_calls list. `arguments` is a STRING that holds JSON.\"\"\"\n", " return {\"id\": call_id, \"type\": \"function\",\n", " \"function\": {\"name\": \"get_weather\", \"arguments\": json.dumps({\"city\": city})}}\n", "\n", "history_one = [\n", " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", " {\"role\": \"user\", \"content\": \"What's the weather in Berlin?\"},\n", " {\"role\": \"assistant\", \"content\": None, \"tool_calls\": [tool_call(\"call_1\", \"Berlin\")]},\n", " {\"role\": \"tool\", \"tool_call_id\": \"call_1\", \"content\": \"12°C, rainy\"},\n", "]\n", "\n", "def show(messages):\n", " \"\"\"Print one line per message: its index, its role and the fields that carry the pairing.\"\"\"\n", " for i, m in enumerate(messages):\n", " if m.get(\"tool_calls\"):\n", " calls = \", \".join(f'{c[\"id\"]}: {c[\"function\"][\"name\"]}({c[\"function\"][\"arguments\"]})'\n", " for c in m[\"tool_calls\"])\n", " print(f'{i} {m[\"role\"]:<9} tool_calls = [{calls}]')\n", " elif m[\"role\"] == \"tool\":\n", " print(f'{i} {m[\"role\"]:<9} tool_call_id = {m[\"tool_call_id\"]} content = {m[\"content\"]!r}')\n", " else:\n", " print(f'{i} {m[\"role\"]:<9} content = {m[\"content\"]!r}')\n", "\n", "show(history_one)" ] }, { "cell_type": "code", "execution_count": null, "id": "tut3-one-call", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def reply(messages):\n", " \"\"\"Send a history to the model and return the text of the assistant message it produces.\"\"\"\n", " r = client.chat.completions.create(model=MODEL, messages=messages, temperature=0,\n", " reasoning_effort=\"none\")\n", " return r.choices[0].message.content\n", "\n", "# The second model call of the run: four messages are sent, the model returns the fifth.\n", "print(reply(history_one))" ] }, { "cell_type": "code", "execution_count": null, "id": "tut3-two-calls", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Two tool calls in ONE assistant message, and one tool message per call.\n", "history_two = [\n", " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", " {\"role\": \"user\", \"content\": \"What's the weather in Berlin and in Paris?\"},\n", " {\"role\": \"assistant\", \"content\": None,\n", " \"tool_calls\": [tool_call(\"call_1\", \"Berlin\"), tool_call(\"call_2\", \"Paris\")]},\n", " {\"role\": \"tool\", \"tool_call_id\": \"call_1\", \"content\": \"12°C, rainy\"},\n", " {\"role\": \"tool\", \"tool_call_id\": \"call_2\", \"content\": \"18°C, sunny\"},\n", "]\n", "show(history_two)\n", "print()\n", "print(reply(history_two))" ] }, { "cell_type": "code", "execution_count": null, "id": "tut3-faulty", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# Four faulty variants of history_two. Each differs from it in the tool messages only.\n", "head, berlin, paris = history_two[:3], history_two[3], history_two[4]\n", "\n", "ids_swapped = head + [{**berlin, \"tool_call_id\": \"call_2\"}, {**paris, \"tool_call_id\": \"call_1\"}]\n", "order_swapped = head + [paris, berlin] # ids correct, order exchanged\n", "one_dropped = head + [berlin] # call_2 is never answered\n", "merged = head + [{\"role\": \"tool\", \"tool_call_id\": \"call_1\",\n", " \"content\": \"12°C, rainy; 18°C, sunny\"}] # one message for two calls\n", "\n", "for name, history in [(\"ids swapped\", ids_swapped), (\"order swapped\", order_swapped),\n", " (\"one dropped\", one_dropped), (\"merged\", merged)]:\n", " print(f\"--- {name}\")\n", " show(history[3:])\n", " try:\n", " print(\"reply:\", reply(history))\n", " except Exception as e: # an endpoint that validates the history\n", " print(\"rejected:\", e)\n", " print()" ] }, { "cell_type": "markdown", "id": "tut3-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. The run of `history_one` ends with the reply of the model as a fifth message. How many of the five\n", " messages did the model produce, and how many model calls does the run take?\n", "2. `tool_call` stores `arguments` with `json.dumps`. Which type does the field have, and which step\n", " does the runtime therefore perform before it executes the tool?\n", "3. Compare the replies to `ids swapped` and to `order swapped` with the reply to `history_two`.\n", " Which of the two variants changes the answer? By which property does this endpoint assign a tool\n", " result to its call? Why does the runtime nevertheless set `tool_call_id` correctly?\n", "4. `one dropped` and `merged` violate the pairing invariant. Does the endpoint reject them? Which of\n", " the four variants is the hardest to detect, and why?\n", "5. To save tokens, a runtime removes old messages from the history (→ L-Memory). Which condition\n", " does every removal have to preserve? Give one removal from `history_two` that preserves it and\n", " one that violates it." ] }, { "cell_type": "markdown", "id": "cell24", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: The Pairing Invariant as a Check\n", "\n", "**A check of the pairing invariant is a function that reads a message history and reports a\n", "violation before the history is sent to the model.** The invariant is stated in the tutorial\n", "[*Message Roles and the Pairing Invariant*](#Tutorial:-Message-Roles-and-the-Pairing-Invariant); there the reaction of the model to a faulty history was only observed.\n", "\n", "`check_history(messages)` raises a `ValueError` in three cases:\n", "\n", "- an `assistant` message issues a `tool_call_id` that is already in use;\n", "- a `tool` message answers a call that was never issued or is already answered;\n", "- a call is still unanswered at the end of the history.\n", "\n", "The second cell applies the function to the six histories of the tutorial\n", "[*Message Roles and the Pairing Invariant*](#Tutorial:-Message-Roles-and-the-Pairing-Invariant): the two valid histories and the four faulty variants." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-check-def", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "def check_history(messages):\n", " \"\"\"Raise unless every tool call is answered exactly once, and every answer has a call.\"\"\"\n", " pending = {} # tool_call_id -> (index, tool name)\n", " for i, m in enumerate(messages):\n", " # A message is a dict (written by the runtime) or an SDK object (returned by the client).\n", " role = m[\"role\"] if isinstance(m, dict) else m.role\n", " get = (lambda k: m.get(k)) if isinstance(m, dict) else (lambda k: getattr(m, k, None))\n", " if role == \"assistant\":\n", " for call in (get(\"tool_calls\") or []):\n", " cid = call[\"id\"] if isinstance(call, dict) else call.id\n", " if cid in pending:\n", " raise ValueError(f\"message {i}: duplicate tool_call_id {cid!r}\")\n", " name = (call[\"function\"][\"name\"] if isinstance(call, dict) else call.function.name)\n", " pending[cid] = (i, name) # issued, not yet answered\n", " elif role == \"tool\":\n", " cid = get(\"tool_call_id\")\n", " if cid not in pending:\n", " raise ValueError(f\"message {i}: tool result for unknown call {cid!r} \"\n", " f\"(never issued, or already answered)\")\n", " del pending[cid] # answered exactly once\n", " if pending:\n", " raise ValueError(\"unanswered tool calls: \"\n", " + \", \".join(f\"{cid} ({n})\" for cid, (_, n) in pending.items()))\n", " return True" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-check-apply", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The two valid histories.\n", "print(\"ok:\", check_history(history_one))\n", "print(\"ok:\", check_history(history_two))\n", "\n", "# Two of the faulty variants: a tool message is missing.\n", "for name, h in [(\"one dropped\", one_dropped), (\"merged\", merged)]:\n", " try:\n", " check_history(h)\n", " except ValueError as e:\n", " print(f\"caught ({name}):\", e)\n", "\n", "# The two variants with exchanged tool results.\n", "print(\"ok:\", check_history(ids_swapped), check_history(order_swapped))" ] }, { "cell_type": "markdown", "id": "tut-check-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. What does the dictionary `pending` contain while the loop is at message `i`? At which line is an\n", " entry added, and at which line is it removed?\n", "2. Name the line that raises for each of the three cases: a duplicate id, a `tool` message without\n", " a call, and an unanswered call. Which of the three does a `tool` message that precedes its\n", " `assistant` message produce?\n", "3. The function reads every message through `isinstance(m, dict)`. Why can one history contain\n", " messages of two forms?\n", "4. `one_dropped` and `merged` are reported; `ids_swapped` and `order_swapped` pass. Why do the last\n", " two pass, although the model reports wrong temperatures for one of them? What can a check of the\n", " structure of a history establish, and what can it not establish?" ] }, { "cell_type": "markdown", "id": "cell26", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: Termination of a Run *(optional)*\n", "\n", "**The termination test is the condition under which the loop ends a run and returns an answer.**\n", "`run_agent` of the tutorial [*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) uses the **two-branch test**: a\n", "reply with tool calls continues the run, and a reply without tool calls is the answer. Two replies\n", "without a tool call are not answers: a text that only announces a tool call, and a text that the\n", "token limit ended. The field `finish_reason` of a reply states why generation stopped: `stop`,\n", "`tool_calls`, or `length` when the token limit was reached.\n", "\n", "**The tutorial shows three further encodings of the test.** A **sentinel** is a reserved value\n", "whose only function is to mark the end.\n", "\n", "| Encoding | The run ends when … | Code |\n", "|---|---|---|\n", "| two-branch test | the reply contains no tool call | `run_agent` |\n", "| sentinel string | a text reply begins with `FINAL ANSWER:`; every other text reply is a **thought**, which is appended, and the loop continues | `run_agent_v2(sentinel=False)` |\n", "| sentinel tool | the model calls the tool `final_answer(response)`; a second tool `think(thought)` returns the constant string `\"OK.\"` | `run_agent_v2(sentinel=True)` |\n", "| exception | a *terminating* exception is raised at the place where the end is known | `run_agent_v3` |\n", "\n", "The environment of the ReAct reference implementation offers the four actions `search[…]`,\n", "`lookup[…]`, `think[…]` and `finish[answer]`, and `finish` is the only exit (Yao et al., 2023,\n", "reference implementation): this is the sentinel-tool encoding.\n", "\n", "**References.**\n", "- Lieret, Kilian, et al. *Minimal AI agent tutorial*. 2026. — the terminating / non-terminating\n", " exception split, and the principle of handing a failure back to the model.\n", " \n", "- Yao, Shunyu, et al. ReAct reference implementation, 2023. — the `search` / `lookup` / `think` /\n", " `finish` action space." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-tools", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "import json\n", "\n", "# The two sentinel tools, advertised to the model beside `add` and `word_count`.\n", "FINAL_TOOL = {\"type\": \"function\", \"function\": {\n", " \"name\": \"final_answer\",\n", " \"description\": \"Call this tool ONLY when you have fully completed the user's task and wish to present the final answer.\",\n", " \"parameters\": {\"type\": \"object\",\n", " \"properties\": {\"response\": {\"type\": \"string\",\n", " \"description\": \"The final response to present to the user.\"}},\n", " \"required\": [\"response\"]}}}\n", "\n", "THINK_TOOL = {\"type\": \"function\", \"function\": {\n", " \"name\": \"think\",\n", " \"description\": \"Think out loud. Has no effect in the world; use it to reason before acting.\",\n", " \"parameters\": {\"type\": \"object\",\n", " \"properties\": {\"thought\": {\"type\": \"string\"}},\n", " \"required\": [\"thought\"]}}}\n", "\n", "SENTINEL_IMPL = {\"think\": lambda thought: \"OK.\"} # the tool result carries no information" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-v2", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def run_agent_v2(user_input, system=\"You are a helpful assistant.\", max_steps=6,\n", " sentinel=True, verbose=True, **kw):\n", " \"\"\"sentinel=True -> the run ends when the model calls the TOOL final_answer\n", " sentinel=False -> the run ends on a text reply that begins with 'FINAL ANSWER:'\"\"\"\n", " schemas = tools + ([FINAL_TOOL, THINK_TOOL] if sentinel else [])\n", " if not sentinel:\n", " system += \"\\nWhen you are done, reply with a line starting exactly 'FINAL ANSWER: '.\"\n", " messages = [{\"role\": \"system\", \"content\": system},\n", " {\"role\": \"user\", \"content\": user_input}]\n", " for step in range(max_steps):\n", " r = client.chat.completions.create(model=MODEL, messages=messages, tools=schemas,\n", " reasoning_effort=\"none\", **kw)\n", " m, why = r.choices[0].message, r.choices[0].finish_reason\n", " if verbose:\n", " print(f\" step {step}: finish_reason={why!r} tool_calls={bool(m.tool_calls)}\")\n", " if why not in (\"stop\", \"tool_calls\"): # the truncation guard\n", " return f\"[incomplete: finish_reason={why}]\"\n", " messages.append(m)\n", " if not m.tool_calls:\n", " text = (m.content or \"\")\n", " if not sentinel and text.strip().startswith(\"FINAL ANSWER:\"): # the sentinel string\n", " return text.split(\"FINAL ANSWER:\", 1)[1].strip()\n", " continue # a THOUGHT: the history grew, no tool result follows\n", " for call in m.tool_calls:\n", " name = call.function.name\n", " if name == \"final_answer\": # the sentinel tool: return\n", " return json.loads(call.function.arguments)[\"response\"]\n", " impl = {**TOOLS_IMPL, **SENTINEL_IMPL}.get(name)\n", " try:\n", " out = impl(**json.loads(call.function.arguments)) if impl else f\"ERROR: no tool {name}\"\n", " except Exception as e:\n", " out = f\"ERROR: {e}\"\n", " messages.append({\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": str(out)})\n", " return \"[stopped: step budget exhausted]\" # the step cap" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-run", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The same task with the sentinel tool and with the sentinel string.\n", "TASK = \"Add 19 and 23, then say the result in a sentence.\"\n", "print(\"sentinel tool :\", run_agent_v2(TASK))\n", "print(\"sentinel string:\", run_agent_v2(TASK, sentinel=False))" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-narrate", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# A system prompt that asks the model to announce the tool before it calls it,\n", "# given to the two-branch loop `run_agent`.\n", "NARRATE = (\"You are a helpful assistant. First say out loud, in one sentence, which tool you are \"\n", " \"about to use and why — then use it.\")\n", "print(\"two-branch :\", run_agent(TASK, system=NARRATE))" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-truncate", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# A token limit of 20 tokens per reply: generation stops with finish_reason \"length\".\n", "print(\"truncated :\", run_agent_v2(TASK, max_tokens=20))" ] }, { "cell_type": "markdown", "id": "tut-term-exc-lead-todo", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "**Termination as an exception.** In `run_agent` and `run_agent_v2` the loop inspects each reply and\n", "decides whether the run has ended. `run_agent_v3` removes that decision from the loop and uses two\n", "exception classes:\n", "\n", "- `Terminated` is **terminating**: it is raised at the place where the final answer is produced,\n", " and the function returns its value.\n", "- `AgentError` is **non-terminating**: it is raised on a recoverable failure, such as an unknown\n", " tool, a tool function that raised, or a reply that the token limit ended. The loop appends the\n", " error text to `messages` and continues, so that the next model call is conditioned on it.\n", "\n", "[Lieret et al. (2026)](https://minimal-agent.com/) state the principle of the second class for a\n", "minimal agent:\n", "\n", "> *\"Whenever a known exception arises (timeouts, format errors, etc.), let's just tell the LM and let it\n", "> handle it itself.\"* — Lieret et al., *Minimal AI agent tutorial* (2026)" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-term-v3", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "class AgentError(RuntimeError):\n", " \"\"\"Recoverable. The text is returned to the model, and the run continues.\"\"\"\n", "\n", "class Terminated(RuntimeError):\n", " \"\"\"The run has ended. .value holds what is returned to the caller.\"\"\"\n", " def __init__(self, value):\n", " super().__init__(value)\n", " self.value = value\n", "\n", "def run_agent_v3(user_input, system=\"You are a helpful assistant.\", max_steps=6, **kw):\n", " schemas = tools + [FINAL_TOOL, THINK_TOOL]\n", " messages = [{\"role\": \"system\", \"content\": system},\n", " {\"role\": \"user\", \"content\": user_input}]\n", "\n", " def act(call):\n", " \"\"\"Execute one call. Raises Terminated to end the run, AgentError to report a failure.\"\"\"\n", " name = call.function.name\n", " if name == \"final_answer\": # raised at the place where the end is known\n", " raise Terminated(json.loads(call.function.arguments)[\"response\"])\n", " impl = {**TOOLS_IMPL, **SENTINEL_IMPL}.get(name)\n", " if impl is None:\n", " raise AgentError(f\"no such tool {name!r}; available: {sorted({**TOOLS_IMPL, **SENTINEL_IMPL})}\")\n", " try:\n", " return impl(**json.loads(call.function.arguments))\n", " except Exception as e:\n", " raise AgentError(f\"{name} failed: {e}\") from e\n", "\n", " try:\n", " for _ in range(max_steps):\n", " try:\n", " r = client.chat.completions.create(model=MODEL, messages=messages, tools=schemas,\n", " reasoning_effort=\"none\", **kw)\n", " m, why = r.choices[0].message, r.choices[0].finish_reason\n", " if why not in (\"stop\", \"tool_calls\"):\n", " raise AgentError(f\"your previous reply was cut off (finish_reason={why}); \"\n", " \"answer more concisely\")\n", " messages.append(m)\n", " for call in (m.tool_calls or []):\n", " try:\n", " out = act(call)\n", " except AgentError as e: # a failure of THIS call: its tool result\n", " out = f\"ERROR: {e}\" # (pairing invariant: one tool message per id)\n", " messages.append({\"role\": \"tool\", \"tool_call_id\": call.id, \"content\": str(out)})\n", " except AgentError as e: # a failure without a pending tool_call_id\n", " messages.append({\"role\": \"user\", \"content\": f\"ERROR: {e}\"})\n", " raise Terminated(\"[stopped: step budget exhausted]\")\n", " except Terminated as t:\n", " return t.value\n", "\n", "print(\"exception-based:\", run_agent_v3(TASK))" ] }, { "cell_type": "markdown", "id": "tut-term-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. With the system prompt `NARRATE`, a model can reply with the announcement and without a tool\n", " call. What does `run_agent` return in that case? Is the result an exception or a wrong answer,\n", " and which of the two is harder to detect?\n", "2. In `run_agent_v2(sentinel=False)`, which line treats a text reply as a thought? What ends the\n", " run if the model never writes the sentinel string, and what does the caller receive?\n", "3. In `run_agent_v2(sentinel=True)`, how is the tool `final_answer` executed, and how the tool\n", " `think`? Compare the two tools with the four actions of the ReAct reference implementation.\n", "4. Which values of `finish_reason` does a complete run print? Which value does the run with\n", " `max_tokens=20` print, what does `run_agent_v2` return for it, and what would `run_agent` return?\n", " Name the line that makes the difference.\n", "5. A model that is tuned for tool calling serves a chat interface that displays the answer while\n", " it is generated. Which of the encodings two-branch test, sentinel string and sentinel tool fits\n", " this case, and which cost does the choice have?\n", "6. `run_agent_v3` contains no test of the form \"is this reply final?\". Which component raises\n", " `Terminated` when the model has finished, and which one when the step budget is exhausted? Is\n", " that component the first one that has the information?\n", "7. `run_agent_v3` appends the text of an `AgentError` to `messages` and does not repeat the model\n", " call unchanged. Why? The text is appended at two different places, once as a `tool` message and\n", " once as a `user` message. Which rule decides between the two?" ] }, { "cell_type": "markdown", "id": "tut-react-ref-intro", "metadata": { "editable": true, "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Tutorial: The ReAct Loop of the Reference Implementation\n", "\n", "**The reference implementation of ReAct is the code that Yao et al. published with their paper. Its\n", "loop needs no tool-calling interface: it consists of a text prompt, a stop sequence and a string\n", "parser.** The loop `run_agent` of the tutorial\n", "[*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls) receives a tool call as the structured field `tool_calls`. The\n", "reference implementation calls a **completion model**, a model that continues a given text and has\n", "neither message roles nor a `tools` parameter. Thought, action and observation are lines of one text\n", "that grows with every step.\n", "\n", "**Source.** The code of this tutorial is adapted from the notebook `hotpotqa.ipynb` and the module\n", "`wikienv.py` of the repository\n", "[Yao, Shunyu, et al. \"ReAct Prompting.\" GitHub repository, 2023](https://github.com/ysymyth/ReAct),\n", "which accompanies the paper of [Yao et al. (2023)](https://arxiv.org/html/2210.03629). The\n", "instruction text, the first exemplar, the line format `Thought i / Action i / Observation i`, the\n", "stop sequence, the separation of thought and action, the four actions of the environment and their\n", "observation strings are taken from that repository. The tutorial differs from it in five points:\n", "\n", "| | Reference implementation | This tutorial |\n", "|---|---|---|\n", "| Model | `text-davinci-002`, a completion model | the local model `qwen2.5:3b` through `ollama.generate(..., raw=True)`: the prompt is sent without the chat template, and the model continues the text |\n", "| Environment | `WikiEnv`: Wikipedia, read over the network | `PassageEnv`: three invented passages in a dictionary, with the same four actions |\n", "| Exemplars | six solved questions (`webthink_simple6`) | the first of the six, verbatim, and one written for this tutorial |\n", "| Task | the questions of HotpotQA, a multi-hop question-answering benchmark, scored against the dataset | one question; the answer is compared with the passages by hand |\n", "| Action line | passed to the environment unchanged | cut after the first `]`, because a small local model sometimes continues the line |\n", "\n", "The first cell defines the environment and executes seven hand-written actions without a model. The\n", "second cell defines the prompt and the model call, the third cell the loop, which it then runs on one\n", "question. The last cell repeats the first model call of that run with and without the stop sequence." ] }, { "cell_type": "code", "execution_count": null, "id": "tut-react-ref-env", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "outputs": [], "source": [ "# The environment: three passages in place of Wikipedia, and the question of the run.\n", "# The persons and places are invented, so that no answer can come from the weights of the model.\n", "PAGES = {\n", " \"Mira Kessler\": \"Mira Kessler was a cartographer. She was born in the town of Aldenau. \"\n", " \"In 1911 she drew the first complete map of the Tarn valley. \"\n", " \"She taught surveying at the technical school of Brunn until 1934.\",\n", " \"Aldenau\": \"Aldenau is a town on the river Lome. It has 4,200 inhabitants. \"\n", " \"The town hall dates from 1702. The weekly market takes place on Thursdays. \"\n", " \"The railway station of Aldenau opened in 1887.\",\n", " \"Tarn valley\": \"The Tarn valley is a valley in the Brunn mountains. It is 31 km long. \"\n", " \"The river Tarn flows into the river Lome near Aldenau.\",\n", "}\n", "QUESTION = \"In which year did the railway station open in the town where Mira Kessler was born?\"\n", "\n", "\n", "class PassageEnv:\n", " \"\"\"The four actions of WikiEnv.step (wikienv.py), on PAGES instead of Wikipedia.\"\"\"\n", "\n", " def __init__(self, pages, n_first=2):\n", " self.pages, self.n_first = pages, n_first\n", " self.page = None # the passage opened by the last successful search\n", " self.keyword, self.hits, self.cnt = None, [], 0\n", " self.answer = None\n", "\n", " @staticmethod\n", " def sentences(text):\n", " \"\"\"Split a passage into its sentences.\"\"\"\n", " return [s.strip().rstrip(\".\") + \".\" for s in text.split(\". \") if s.strip()]\n", "\n", " def step(self, action):\n", " \"\"\"Execute one action string. Returns (observation, done).\"\"\"\n", " action, done = action.strip(), False\n", " if action.startswith(\"search[\") and action.endswith(\"]\"):\n", " entity = action[len(\"search[\"):-1]\n", " title = next((t for t in self.pages if t.lower() == entity.lower()), None)\n", " if title is None: # no such passage: name the passages instead\n", " obs = f\"Could not find {entity}. Similar: {list(self.pages)}.\"\n", " else: # open the passage, return its first sentences only\n", " self.page, self.keyword = self.pages[title], None\n", " obs = \" \".join(self.sentences(self.page)[:self.n_first])\n", " elif action.startswith(\"lookup[\") and action.endswith(\"]\"):\n", " keyword = action[len(\"lookup[\"):-1]\n", " if self.keyword != keyword: # a new keyword: build the hit list\n", " self.keyword, self.cnt = keyword, 0\n", " self.hits = [s for s in self.sentences(self.page or \"\")\n", " if keyword.lower() in s.lower()]\n", " if self.cnt >= len(self.hits):\n", " obs = \"No more results.\"\n", " else:\n", " obs = f\"(Result {self.cnt + 1} / {len(self.hits)}) {self.hits[self.cnt]}\"\n", " self.cnt += 1\n", " elif action.startswith(\"finish[\") and action.endswith(\"]\"):\n", " self.answer, done = action[len(\"finish[\"):-1], True # the ONLY exit\n", " obs = \"Episode finished.\"\n", " elif action.startswith(\"think[\") and action.endswith(\"]\"):\n", " obs = \"Nice thought.\" # a thought: a constant observation\n", " else:\n", " obs = f\"Invalid action: {action}\"\n", " return obs, done\n", "\n", "\n", "# The environment alone, without a model: one action of each kind, written by hand.\n", "env = PassageEnv(PAGES)\n", "for action in [\"search[Kessler]\", \"search[Aldenau]\", \"lookup[station]\", \"lookup[station]\",\n", " \"think[The station opened in 1887.]\", \"Search Aldenau\", \"finish[1887]\"]:\n", " print(f\"{action:<40} -> {env.step(action)}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-react-ref-prompt", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "import ollama\n", "\n", "# The prompt. INSTRUCTION is the instruction of hotpotqa.ipynb and EXEMPLAR the first of the six\n", "# exemplars of `webthink_simple6` (prompts/prompts_naive.json), both verbatim. EXEMPLAR_LOCAL is\n", "# written for this tutorial, in the same format, on a passage collection like PAGES.\n", "INSTRUCTION = \"\"\"Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: \n", "(1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.\n", "(2) Lookup[keyword], which returns the next sentence containing keyword in the current passage.\n", "(3) Finish[answer], which returns the answer and finishes the task.\n", "Here are some examples.\n", "\"\"\"\n", "EXEMPLAR = \"\"\"Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?\n", "Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area.\n", "Action 1: Search[Colorado orogeny]\n", "Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas.\n", "Thought 2: It does not mention the eastern sector. So I need to look up eastern sector.\n", "Action 2: Lookup[eastern sector]\n", "Observation 2: (Result 1 / 1) The eastern sector extends into the High Plains and is called the Central Plains orogeny.\n", "Thought 3: The eastern sector of Colorado orogeny extends into the High Plains. So I need to search High Plains and find its elevation range.\n", "Action 3: Search[High Plains]\n", "Observation 3: High Plains refers to one of two distinct land regions:\n", "Thought 4: I need to instead search High Plains (United States).\n", "Action 4: Search[High Plains (United States)]\n", "Observation 4: The High Plains are a subregion of the Great Plains. From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3]\n", "Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft.\n", "Action 5: Finish[1,800 to 7,000 ft]\n", "\"\"\"\n", "EXEMPLAR_LOCAL = \"\"\"Question: Which river flows through the town in which the painter Jon Arvid died?\n", "Thought 1: I need to search Jon Arvid, find the town in which he died, then find the river of that town.\n", "Action 1: Search[Jon Arvid]\n", "Observation 1: Jon Arvid was a painter. He was born in Selby.\n", "Thought 2: It does not mention where he died. So I need to look up died.\n", "Action 2: Lookup[died]\n", "Observation 2: (Result 1 / 1) He died in the town of Morwen in 1950.\n", "Thought 3: Jon Arvid died in Morwen. So I need to search Morwen and find its river.\n", "Action 3: Search[Morwen]\n", "Observation 3: Morwen is a town in the north. It has a harbour.\n", "Thought 4: It does not mention the river. So I need to look up river.\n", "Action 4: Lookup[river]\n", "Observation 4: (Result 1 / 1) The river Esk flows through Morwen.\n", "Thought 5: The river Esk flows through Morwen, so the answer is Esk.\n", "Action 5: Finish[Esk]\n", "\"\"\"\n", "REACT_PROMPT = INSTRUCTION + EXEMPLAR + EXEMPLAR_LOCAL\n", "\n", "# The model of this tutorial, not MODEL of the setup cell: when qwen3.5:4b continues a raw prompt,\n", "# words are missing from its text.\n", "COMPLETION_MODEL = \"qwen2.5:3b\"\n", "\n", "\n", "def llm(prompt, stop):\n", " \"\"\"One model call: the model CONTINUES the text `prompt` until a stop sequence or 100 tokens.\n", " raw=True sends the prompt without the chat template, as to a completion model; there is no\n", " message list and no `tools` parameter.\"\"\"\n", " r = ollama.generate(model=COMPLETION_MODEL, prompt=prompt, raw=True,\n", " options={\"temperature\": 0, \"num_predict\": 100, \"stop\": stop})\n", " return r[\"response\"]\n", "\n", "\n", "print(REACT_PROMPT[-900:])" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-react-ref-loop", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "def react(question, env, max_steps=7, to_print=True):\n", " \"\"\"The loop `webthink` of hotpotqa.ipynb. Returns (answer, prompt, number of model calls).\"\"\"\n", " prompt = REACT_PROMPT + f\"Question: {question}\\n\"\n", " n_calls, done = 0, False\n", " for i in range(1, max_steps + 1):\n", " n_calls += 1\n", " # One call produces the thought AND the action; generation stops before the observation.\n", " thought_action = llm(prompt + f\"Thought {i}:\", stop=[f\"\\nObservation {i}:\"])\n", " thought_action = thought_action.strip()\n", " try:\n", " thought, action = thought_action.split(f\"\\nAction {i}: \")\n", " except ValueError: # no well-formed action line: ask for the action alone\n", " n_calls += 1\n", " thought = thought_action.split(\"\\n\")[0]\n", " action = llm(prompt + f\"Thought {i}: {thought}\\nAction {i}:\", stop=[\"\\n\"])\n", " thought = thought.strip()\n", " action = action.strip().split(\"\\n\")[0] # one action per step\n", " action = action[:action.find(\"]\") + 1] or action # cut text after the closing bracket\n", " obs, done = env.step(action[:1].lower() + action[1:]) # Search[..] -> search[..]\n", " step_str = f\"Thought {i}: {thought}\\nAction {i}: {action}\\nObservation {i}: {obs}\\n\"\n", " prompt += step_str # the context grows by one thought, one action, one observation\n", " if to_print:\n", " print(step_str)\n", " if done:\n", " break\n", " if not done: # step limit reached: finish with an empty answer\n", " env.step(\"finish[]\")\n", " return env.answer, prompt, n_calls\n", "\n", "\n", "env = PassageEnv(PAGES)\n", "answer, trajectory, n_calls = react(QUESTION, env)\n", "print(\"answer:\", repr(answer), \" model calls:\", n_calls)" ] }, { "cell_type": "code", "execution_count": null, "id": "tut-react-ref-stop", "metadata": { "editable": true, "slideshow": { "slide_type": "-" }, "tags": [] }, "outputs": [], "source": [ "# The first model call of the run again, once with the stop sequence of the loop and once without.\n", "first_prompt = REACT_PROMPT + f\"Question: {QUESTION}\\n\" + \"Thought 1:\"\n", "\n", "print(\"--- with stop sequence '\\\\nObservation 1:'\")\n", "print(\"Thought 1:\" + llm(first_prompt, stop=[\"\\nObservation 1:\"]))\n", "print()\n", "print(\"--- without a stop sequence\")\n", "print(\"Thought 1:\" + llm(first_prompt, stop=[]))\n", "print()\n", "print(\"characters in the prompt of the first call:\", len(first_prompt))\n", "print(\"characters in the context after the run: \", len(trajectory))" ] }, { "cell_type": "markdown", "id": "tut-react-ref-questions", "metadata": { "editable": true, "slideshow": { "slide_type": "subslide" }, "tags": [] }, "source": [ "**Questions.**\n", "\n", "1. The first cell executes seven hand-written actions. Which of them are answered from a passage,\n", " which one is answered by a constant string, and which one ends the episode? What does the\n", " environment return for `Search Aldenau`, and what follows for a model that writes an action in a\n", " wrong format?\n", "2. A step of the printed run has three lines. Which of them does the model generate, and which does\n", " the program write? What does the model generate in the last cell when no stop sequence is set?\n", " Compare the `Observation 1` of that output with `PAGES`.\n", "3. Under which condition does `react` end, and under which condition does `run_agent` of the tutorial\n", " [*The ReAct Loop with Tool Calls*](#Tutorial:-The-ReAct-Loop-with-Tool-Calls)\n", " end? What does each of the two loops do with a reply of the model that contains no action?\n", "4. The lecture writes the context as $c_k = (c_0, o_1, t_1, a_1, \\dots, o_k)$. Which Python object\n", " of the tutorial is the prefix $c_0$, and which line is the first observation $o_1$? Which symbol\n", " does the line `Observation i` have? How many model calls does the run take?\n", "5. The instruction names three actions, and the environment implements four. How does `react`\n", " obtain the thought of a step, and which observation does that thought receive? Which observation\n", " does the action `think[…]` receive?\n", "6. The persons and places of `PAGES` are invented. What would a run on a question about a\n", " well-known fact not show? Find a step of the printed run whose observation does not contain the\n", " fact that the thought expected, and describe the next thought." ] }, { "cell_type": "code", "execution_count": null, "id": "3c7c7967-4b50-499b-894a-05e929de5a98", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" }, "rise": { "scroll": true, "theme": "simple", "transition": "none" } }, "nbformat": 4, "nbformat_minor": 5 }