{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6da19a97",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Lab\n",
    "\n",
    "**Agentic AI** · `L-Intro`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Setup: Ollama + the OpenAI Client\n",
    "\n",
    "**Install (no root needed).** Ollama runs without admin rights — install a user-local binary:\n",
    "\n",
    "```bash\n",
    "mkdir -p ~/.local\n",
    "curl -L https://ollama.com/download/ollama-linux-amd64.tgz | tar -xz -C ~/.local\n",
    "export PATH=\"$HOME/.local/bin:$PATH\"      # add this line to ~/.bashrc\n",
    "```\n",
    "\n",
    "**Start the server — each session** (lab machines have no auto-start service and no\n",
    "Docker/Podman):\n",
    "\n",
    "```bash\n",
    "ollama serve &        # local server on 127.0.0.1:11434 — leave it running while you work\n",
    "```\n",
    "\n",
    "**Pull a model and test it on the command line:**\n",
    "\n",
    "```bash\n",
    "ollama pull qwen2.5:3b\n",
    "ollama list                                    # list installed models\n",
    "ollama run qwen2.5:3b \"Say hello in one word.\" # quick sanity check that Ollama works\n",
    "```\n",
    "\n",
    "**Try different models** — depending on the machine's free RAM. Smaller = lighter and faster on\n",
    "CPU, but tool calling gets less reliable below ~3B:\n",
    "\n",
    "| free RAM | model | tool calling |\n",
    "|---|---|---|\n",
    "| ~4 GB | `llama3.2:1b`, `qwen2.5:1.5b` | flaky |\n",
    "| ~6–8 GB | `qwen2.5:3b`, `llama3.2:3b` | reliable ✅ |\n",
    "| 8+ GB | `qwen2.5:7b` | best |\n",
    "\n",
    "Pull a couple and **compare speed and answer quality** — set `MODEL` below to whichever you use.\n",
    "\n",
    "> *On your **own** computer you may auto-start Ollama with `systemctl --user` or Docker/Podman.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55322415-a86d-4820-a272-5d1d26277453",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Run me first — defines `client` and `MODEL` used by all tutorials below.\n",
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(base_url=\"http://localhost:11434/v1\", api_key=\"ollama\")  # key is ignored\n",
    "MODEL = \"qwen2.5:3b\"   # try \"llama3.2:3b\", \"qwen2.5:1.5b\", or \"qwen2.5:7b\" (RAM permitting)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "625c2379",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Chat Completions: Request and Response\n",
    "\n",
    "Every tutorial in this lab is one call of the same shape — only the `messages` list (and, at the\n",
    "end, a `tools` argument) changes:\n",
    "\n",
    "```python\n",
    "resp = client.chat.completions.create(\n",
    "    model=MODEL,                    # which model Ollama runs\n",
    "    messages=[                      # the WHOLE conversation, oldest message first\n",
    "        {\"role\": \"system\", \"content\": \"standing instructions\"},   # optional\n",
    "        {\"role\": \"user\",   \"content\": \"the question\"},\n",
    "    ],\n",
    ")\n",
    "```\n",
    "\n",
    "Where the answer lives in `resp`:\n",
    "\n",
    "| Expression | Content |\n",
    "|---|---|\n",
    "| `resp.choices[0].message.content` | the reply text — `None` when the model requests a tool instead |\n",
    "| `resp.choices[0].message.tool_calls` | the structured tool requests — `None` when there are none |\n",
    "| `resp.choices[0].finish_reason` | why generation stopped, e.g. `\"stop\"` for a normal end, `\"length\"` when the token limit cut the answer off |\n",
    "| `resp.usage.prompt_tokens`, `.completion_tokens`, `.total_tokens` | the token counts of this one call |\n",
    "\n",
    "**Check the connection before the first tutorial:**\n",
    "\n",
    "```python\n",
    "print([m.id for m in client.models.list().data])   # the models Ollama has pulled\n",
    "```\n",
    "\n",
    "- `APIConnectionError` → the server is not running: execute `ollama serve &` in a terminal.\n",
    "- `MODEL` missing from the list → `ollama pull <model>` in a terminal, or change `MODEL`.\n",
    "- `print(resp.model_dump())` shows the complete response object — the quickest way to find a field."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a218174",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Tutorial: First API Call\n",
    "\n",
    "**One request to the model is one call of `client.chat.completions.create(...)` with a list of\n",
    "messages; the reply is in `resp.choices[0].message.content`.** The cells below send a single user\n",
    "message and print the reply, then the token counts of the call, then the complete response object.\n",
    "Execute them in order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78e2b737",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The question, set in a variable of its own so that it can be changed without touching the call.\n",
    "query_string = \"In one sentence: what is an AI agent?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f62eb4a6",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# One request: the model, and a message list with one user message.\n",
    "# The first call after `ollama serve` takes longer — the model is loaded into memory once.\n",
    "resp = client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages=[{\"role\": \"user\", \"content\": query_string}],\n",
    ")\n",
    "print(resp.choices[0].message.content)     # the reply text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d172d751",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# What the call cost, in tokens: the prompt (the question plus the chat template around it)\n",
    "# and the completion (the answer).\n",
    "print(resp.usage)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "555db918",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The complete response object — every field the API returns.\n",
    "print(resp.model_dump_json(indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "256af56b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "**Questions.**\n",
    "\n",
    "1. What does one call of a language model consist of — what goes in, what comes out, and what\n",
    "   remains afterwards?\n",
    "2. Why does a question cost more prompt tokens than it has words?\n",
    "3. Why is the cost of a call measured in tokens, and not in requests or in characters?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db1dda12",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Tutorial: The System Prompt\n",
    "\n",
    "**The system prompt is sent before the user's message and sets the model's standing behaviour —\n",
    "tone, rules, role.** The cells below send the *same* question under several system prompts, so that\n",
    "the system prompt is the only difference between the answers, and then one system prompt that\n",
    "conflicts with the user's request. Execute them in order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eedeb5c2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The question stays fixed; only the system prompt changes between the calls.\n",
    "query_string = \"Explain what a token is.\"\n",
    "system_prompts = [\n",
    "    \"Answer in one terse sentence.\",\n",
    "    \"You are a pirate; answer in pirate speak.\",\n",
    "    \"You are a university professor; answer in academic language.\",\n",
    "    \"Answer only in German.\",\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ba9d49f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import textwrap\n",
    "\n",
    "def ask(system: str, user: str) -> str:\n",
    "    \"\"\"Send one system prompt and one user message; return the reply text.\"\"\"\n",
    "    resp = client.chat.completions.create(model=MODEL, messages=[\n",
    "        {\"role\": \"system\", \"content\": system},\n",
    "        {\"role\": \"user\",   \"content\": user},\n",
    "    ])\n",
    "    return resp.choices[0].message.content\n",
    "\n",
    "def show(system: str, user: str, width: int = 80) -> None:\n",
    "    \"\"\"Print one answer under a header naming the system prompt and the question.\"\"\"\n",
    "    print(\"=\" * width)\n",
    "    print(f\"SYSTEM: {system}\")\n",
    "    print(f\"USER:   {user}\")\n",
    "    print(\"-\" * width)\n",
    "    answer = ask(system, user)\n",
    "    print(\"\\n\".join(textwrap.fill(line, width) for line in answer.splitlines()))\n",
    "    print()\n",
    "\n",
    "for system_prompt in system_prompts:\n",
    "    show(system_prompt, query_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e929e4bb",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# A system prompt that conflicts with the user's request: the output shows which of the two\n",
    "# instructions the model follows. The priority of the system prompt is learned in alignment,\n",
    "# not enforced by the API.\n",
    "conflicting_system = \"Answer only in Chinese.\"\n",
    "conflicting_query = \"In English, please: what is a token?\"\n",
    "conflicting_query = \"Que es un token? Reposponde in espanol, por favor\"\n",
    "show(conflicting_system, conflicting_query)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91734d5e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "**Questions.**\n",
    "\n",
    "1. What does a system prompt set, and why is it a message with a role of its own rather than part of\n",
    "   the user's text?\n",
    "2. Is the precedence of the system prompt over the user's request guaranteed? What follows for the\n",
    "   security of an agent?\n",
    "3. The question was the same in every call and the answers differed. What does that say about where\n",
    "   a model's behaviour is determined?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1242bb8c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Tutorial: Statelessness & Memory\n",
    "\n",
    "**The model keeps no state between two calls: whatever it should know about the conversation must\n",
    "be in the `messages` list of the call itself.** The cells below introduce a name in one request, ask\n",
    "for it in a second request without the history, and ask once more with the history resent. Execute\n",
    "them in order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dcb83c6c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The two user messages of the conversation.\n",
    "introduction = \"My name is Ada.\"\n",
    "question = \"What is my name?\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fed3917b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 1. A first request that only introduces the name.\n",
    "r1 = client.chat.completions.create(model=MODEL, messages=[\n",
    "    {\"role\": \"user\", \"content\": introduction}])\n",
    "print(\"reply to the introduction ->\", r1.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fe28c56",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 2. A NEW request WITHOUT the history: it contains only the question,\n",
    "#    so the name is nowhere in what the model receives.\n",
    "r2 = client.chat.completions.create(model=MODEL, messages=[\n",
    "    {\"role\": \"user\", \"content\": question}])\n",
    "print(\"without history ->\", r2.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef8f9012",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 3. The same question WITH the history resent: introduction, reply, question.\n",
    "import json\n",
    "\n",
    "history = [\n",
    "    {\"role\": \"user\",      \"content\": introduction},\n",
    "    {\"role\": \"assistant\", \"content\": r1.choices[0].message.content},\n",
    "    {\"role\": \"user\",      \"content\": question},\n",
    "]\n",
    "print(json.dumps(history, indent=2))   # exactly this list is the request — nothing else is sent\n",
    "r3 = client.chat.completions.create(model=MODEL, messages=history)\n",
    "print(\"with history ->\", r3.choices[0].message.content)\n",
    "# The model \"remembers\" the name only because the runtime resent the conversation:\n",
    "# memory is the runtime's job, not the model's."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "722e659e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "**Questions.**\n",
    "\n",
    "1. Why could the model not answer the question when the history was not resent?\n",
    "2. Where does the \"memory\" of a conversation live, and whose task is it?\n",
    "3. Why does the resent history also contain the model's own earlier reply, and not only the user's\n",
    "   messages?\n",
    "4. If every call must carry the whole history, which decisions does the program around the model\n",
    "   have to make as a conversation grows?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99d0c2d6",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Tutorial: Non-determinism & the Token Bill\n",
    "\n",
    "**The same request does not always return the same reply, and the tokens a conversation costs grow\n",
    "quadratically in its number of turns, because the whole history is resent on every turn.** The cells\n",
    "below send one request five times, then hold a four-turn conversation and print the prompt tokens of\n",
    "every turn. Execute them in order. *(Why the replies differ — sampling — is → L-ReAct.)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9b29d97",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The inputs: one request sent several times, and the four user turns of a conversation.\n",
    "repeat_prompt = [{\"role\": \"user\", \"content\": \"Name one animal.\"}]\n",
    "n_repeats = 5\n",
    "turns = [\"My name is Ada.\", \"I live in Berlin.\", \"What is my name?\", \"Where do I live?\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48d9ee36",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The same request, five times. repr() also shows differences in whitespace and punctuation.\n",
    "for _ in range(n_repeats):\n",
    "    r = client.chat.completions.create(model=MODEL, messages=repeat_prompt)\n",
    "    print(repr(r.choices[0].message.content))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e2ac977",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# A four-turn conversation, the full history resent on every turn.\n",
    "history = []\n",
    "for k, turn in enumerate(turns, start=1):\n",
    "    history.append({\"role\": \"user\", \"content\": turn})\n",
    "    r = client.chat.completions.create(model=MODEL, messages=history)\n",
    "    history.append({\"role\": \"assistant\", \"content\": r.choices[0].message.content})\n",
    "    print(f\"turn {k}: messages sent = {len(history) - 1}, \"\n",
    "          f\"prompt_tokens = {r.usage.prompt_tokens}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d34991e2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "source": [
    "**Why `prompt_tokens` grows every turn.** Turn $k$ sends the $k$ user messages and the $k-1$ replies\n",
    "so far, $2k-1$ messages. With every message about $m$ tokens long, a conversation of $n$ turns sends\n",
    "\n",
    "$$\\sum_{k=1}^{n} (2k-1)\\,m \\;=\\; n^2 m$$\n",
    "\n",
    "tokens in total, which is $\\Theta(n^2)$. Cost and latency therefore grow quadratically, while the\n",
    "context window is a fixed upper bound: an agent that runs for hours has to manage its history — trim\n",
    "it, summarise it (→ L-Memory), or move it into a store it retrieves from (→ L-RAG)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7eced39",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "**Questions.**\n",
    "\n",
    "1. The same request returned different replies. What follows for testing or evaluating a system\n",
    "   built on a model?\n",
    "2. Why does the cost of a conversation grow quadratically in the number of turns, and not linearly?\n",
    "3. A conversation of 20 turns, every message about 50 tokens long, the history resent in full each\n",
    "   turn: how many tokens are sent in total?\n",
    "4. Name two ways in which an agent that runs for hours can keep its history within the context\n",
    "   window."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e794897",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## Tutorial: A Structured Tool Call\n",
    "\n",
    "**A tool-calling model answers a request that an advertised tool fits with a structured request —\n",
    "the tool's name and its arguments as JSON — instead of text, and it never executes the tool itself.**\n",
    "The cells below advertise one tool, `get_weather`, ask one question it fits and one it does not fit,\n",
    "and print what comes back. Execute them in order. *(The schema format, and the loop that executes\n",
    "the requested call, are → L-ReAct.)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "582b110e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Two questions, and the schema of the one tool the model is told about.\n",
    "weather_query = \"What's the weather in Berlin?\"\n",
    "no_tool_query = \"What is 2 + 2?\"          # a question no tool helps with\n",
    "tools = [{\n",
    "    \"type\": \"function\",\n",
    "    \"function\": {\n",
    "        \"name\": \"get_weather\",\n",
    "        \"description\": \"Get the current weather for a city.\",\n",
    "        \"parameters\": {\n",
    "            \"type\": \"object\",\n",
    "            \"properties\": {\"city\": {\"type\": \"string\"}},\n",
    "            \"required\": [\"city\"],\n",
    "        },\n",
    "    },\n",
    "}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f39147b8",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "-"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "def request_with_tools(query_string):\n",
    "    \"\"\"Send one question with the tool schema advertised; print what the model returns.\"\"\"\n",
    "    resp = client.chat.completions.create(\n",
    "        model=MODEL,\n",
    "        messages=[{\"role\": \"user\", \"content\": query_string}],\n",
    "        tools=tools,                       # advertise the tool\n",
    "    )\n",
    "    msg = resp.choices[0].message\n",
    "    print(\"question:     \", query_string)\n",
    "    print(\"content:      \", msg.content)                    # often None when a tool is requested\n",
    "    print(\"finish_reason:\", resp.choices[0].finish_reason)  # why generation stopped\n",
    "    if msg.tool_calls:\n",
    "        call = msg.tool_calls[0]\n",
    "        print(\"tool:         \", call.function.name)\n",
    "        print(\"arguments:    \", json.loads(call.function.arguments))  # a JSON string -> a dict\n",
    "    else:\n",
    "        print(\"no tool call\")\n",
    "    print()\n",
    "\n",
    "request_with_tools(weather_query)\n",
    "request_with_tools(no_tool_query)\n",
    "# The model only REQUESTS the call: running get_weather and returning its result to the model\n",
    "# is the runtime's job. Small models sometimes answer in text instead — set MODEL to a model of\n",
    "# at least 3B parameters if no tool call appears for the weather question."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c7d145f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "**Questions.**\n",
    "\n",
    "1. When a tool fits the question, what does the model do — and what does it not do?\n",
    "2. Who executes the tool, and what has to happen next before the user receives an answer?\n",
    "3. The tool's arguments are generated by the model. Why must the program check them before it\n",
    "   executes anything?\n",
    "4. From what does the model decide whether a tool fits a question, and what follows for writing a\n",
    "   tool's description?"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  },
  "rise": {
   "scroll": true,
   "theme": "simple",
   "transition": "none"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}