1The lecture map
Five acts, ten chapters. Prompting tricks grow into a loop; the loop gets four elements — planning, tools, memory and action; then the second half engineers the harness around the model, takes agents into the wild and its benchmarks, and ends with whole societies of agents. Click a chapter to watch it.
Chapter by chapter what each chapter covers, and the question it leaves for the next
- IFrom prompts to a loopOne call with imagined feedback becomes a loop with real feedback
- 1
From Prompts to Agents: ICL, CoT and Self-Refine
2026 is the race for the personal agent. The road there starts with tricks inside one model call — in-context examples, chain of thought, sampling and voting, self-critique — which later become planning, tools, memory and sensors.
→ If an agent is these tricks running in a loop, what exactly is an agent? - 2
What Is an Agent? From ELIZA to the Agent Loop
A system that acts in an environment and learns from feedback. The idea runs from Minsky's Society of Mind through ELIZA and RL agents to LLM agents, and fits in a five-line loop: think, act, observe.
→ The loop is simple — what makes it useful? - IIThe four elementsPlanning, tools, memory and action
- 3
Planning and Reflection: ToT, LLM+P and ReAct
Decompose with least-to-most, search over thoughts with Tree of Thoughts, hand planning to a classical planner with LLM+P, reflect on real feedback, and interleave reasoning with acting in ReAct.
→ Plans need hands — how does a model act? - 4
Tools: From Program of Thoughts to MCP
Let code do the math, let expert models and robot skills do the work, then see a tool call step by step: the model only writes a request, the harness decides whether to run it. MCP standardizes the connection.
→ Every call adds tokens to a finite window — what does the agent remember? - 5
Memory and Action: Embeddings, RAG and Robots
Short-term memory is the context window; long-term memory is retrieval, built on text embeddings, or simply files with an index. Then action leaves the screen: PaLM-E, vision-language-action models and π0.
→ We have all four elements — why does the same model make a mediocre agent in one system and an excellent one in another? - IIIHarness engineeringEverything in an agent except the model
- 6
Harness Engineering: Context, Skills, Guides and Sensors
Prompt, then context, then harness engineering. The harness alone moves scores by 10–20 points. Manage the finite attention budget, package know-how as skills, connect tools through MCP, and steer with guides and sensors.
→ How do real products apply these principles? - 7
Harnesses in Practice: Claude Code, Codex and Multi-Agent
Three products, one loop: Claude Code bets on a simple single loop, Codex on one core for many surfaces, OpenClaw on being always on. When one agent is not enough, A2A lets agents hire agents — but more agents often hurt.
→ Agents run for hours and crash — how do we keep them running, and prove they work? - 8
Fault Tolerance, Evaluation and the Bitter Lesson
Retries, checkpoints and durable execution; sessions that outlive the context window; safety for agents that touch real systems; evals graded on outcomes with pass@k and pass^k — and why the best harness gets thinner as models improve.
→ Where do agents already work, and how do we know? - IVIn the wildWhere agents work, and how we measure them
- 9
Agents in the Wild and How We Benchmark Them
Single agents (Devin), open worlds (Minecraft), multi-agent games and markets, and science (ChemCrow). Benchmarks moved from WebShop and SWE-bench to finishing whole jobs: fast kernels, unknown games, companies and AI research.
→ What happens when the environment itself is made of agents? - VSocietiesWhen the environment is made of other agents
- 10
Agent Societies and Economic World Models
Planets don't react to predictions; people do. Economic world models simulate the economy through the beliefs and actions of many LLM agents, as in WonderEcon, and double as training gyms and safety tests. The lecture closes with five takeaways.
2Try it in Colab
Three short notebooks turn the lecture into experiments. Click Open in Colab, then Runtime → Run all. Each ends with exercises. All three run on a free CPU runtime; a T4 GPU only makes Notebook 1 faster and lets you try bigger models.
A ReAct agent loop in 40 lines
- Declare tools with JSON schemas and write the five-line loop
- Watch a real 0.5B model (Qwen2.5) write its own tool calls
- Add a permission gate, errors that teach and a step budget
- Measure the same model in two harnesses
.ipynb · CPU or GPUpass@k vs. pass^k
- Reproduce slide 125: 75% per try gives 98% pass@3 but 42% pass^3
- Estimate both from a finite number of trials
- See why retries need a grader, and test whether a harness fix really helped
.ipynb · CPU, no modelAgent memory: files vs. embeddings
- Embed English and Chinese sentences and see meaning as geometry
- Build a
MEMORY.mdfile memory and search it with grep - Compare grep and a vector index on paraphrases, Chinese queries and updated facts
- Compute the InfoNCE loss with in-batch negatives
.ipynb · CPU3Concept index
Key terms of the lecture. Each link opens the video at the slide where the idea is explained.
4What's new in 2026
The lecture's outlook (slide 132) lists where harness engineering is going. Six trends to watch:
The harness got a name
"Harness engineering" became the 2026 term for building the system a model runs in. In OpenAI's case study, Codex agents wrote a whole internal product — about 1,500 merged pull requests and roughly a million lines in five months — while engineers designed the agents' workplace (slide 95).
A score belongs to a model–harness pair
On Terminal-Bench, the same model scores 10–20 points apart in different harnesses, more than the gap between many frontier models. Leaderboards now list every model–harness pair as its own row (slide 81, slide 107).
One open protocol stack
MCP for tools, A2A between agents, AGENTS.md and skills for know-how. MCP, AGENTS.md and goose moved to a neutral foundation, and Agent Skills became an open standard (slide 109, slide 110).
Personal agents that never sleep
The race for the personal agent: a real-time front end in front of an agent harness, as in MOSS-Live, or an always-on gateway with a heartbeat that wakes the agent to check its to-do list, as in OpenClaw (slide 2, slide 3, slide 102).
Models absorb the scaffolding
"Let's think step by step" became reasoning models, ReAct templates became native tool calling, and hand-written compaction became models trained to compact their own context. What stays in the harness: permissions, state, connectors and evals (slide 128, slide 129).
Benchmarks measure finishing the job
From "answer the question" to "finish the job": ARC-AGI-3 asks agents to learn unknown games by playing them; at launch humans solved every environment and frontier models scored under 1% (slide 147). RSI-Exam replays an agent's research artifact on a sealed split (slide 149).
5Quotes & reading
"What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle."
Agents are not new — and the multi-agent societies of Chapter 10 are Minsky's question with LLMs as the agents.
"Consistently, the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns."
Why Claude Code is a deliberately simple single loop (Chapter 7), and principle 1: start with one loop.
"Good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."
The principle behind slide 83: context is a finite attention budget.
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."
Why the best harness gets thinner as models get stronger — and principle 10: delete regularly.
Further reading
- Lilian Weng: LLM Powered Autonomous Agents (2023) — the picture behind Chapters 2–5
- Chain-of-Thought Prompting · Self-Consistency · Tree of Thoughts · Reflexion · ReAct
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (2024) — the "same model, different harness" experiment on slide 80
- Anthropic: Effective Harnesses for Long-Running Agents · OpenAI: Harness Engineering · Böckeler: Harness Engineering for Coding Agent Users
- Specs: Model Context Protocol · Agent2Agent (A2A) · Agent Skills · AGENTS.md
- Why Do Multi-Agent LLM Systems Fail? (MAST, 2025) · τ-bench (2024), where pass^k comes from · Evaluating LLMs Trained on Code (2021), the pass@k estimator