1The lecture map
Five acts, nine chapters. First who teaches the course and how it runs, then your project, the part that carries most of the grade. Then the content: what NLP is and how it evolved, the one idea behind LLMs, and finally what a university team builds with it and where the research is heading. Click a chapter to watch it.
Chapter by chapter what each chapter covers, and the question it leaves for the next
- IWho & howMeet the team; learn how the course runs and is graded
- 1
Welcome: Your Instructor and the Research Team
Benyou Wang's path through linguistics, industry and a PhD in Padua, the group's research on LLM-based agents, and its open-source philosophy: release models, data and code on Hugging Face and GitHub.
→ How is the course itself organized? - 2
How the Course Works: Logistics, Structure and Grading
Staff, time and place, the website and WeChat group, the prerequisites of an advanced graduate course, the arc from word vectors to agents, and a grade that is 40% assignments, 55% final project and 5% participation.
→ What exactly will you build? - IIYour projectPlan the assignments and a final project worth 55%
- 3
Three Assignments and the Final Project
Word vectors, SFT and RLHF, then prompting and agents through APIs. The final project: teams of 1–3, a poster and a paper of at most six pages, released on arXiv, GitHub and Hugging Face, and perhaps submitted to a conference.
→ Where does a good project idea come from? - 4
Finding Good Ideas and Succeeding in This Course
Ideas come from the literature and from you: your major, hobbies, data and users, as in the student startups 薯塔AI (FriesAI) and TennSync. Then the course design, what it does and does not cover, and how not to fail.
→ Now the content: what is NLP? - IIIWhat NLP isThe tasks, and how the field got here
- 5
What Is NLP? Understanding, Generation and Core Tasks
NLP sits between computational linguistics and machine learning; natural language is ambiguous where code is not; NLP = NLU + NLG. The classic tasks, from translation to question answering, show why keyword matching is not enough.
→ How were such systems built, and what replaced them? - 6
From Pipelines to Transformers: A Brief History of NLP
Stacks of separate components whose errors add up, then bag-of-words to GPT-4, and rules → statistics → deep learning → Transformers. And how close are today's assistants to Samantha in the film Her?
→ What single idea got us from old NLP to LLMs? - IVLLMsNext-word prediction, scaled up, then put to work as agents
- 7
Language Models, LLMs and the Step to Agents
A language model scores sequences or predicts the next word; neural networks compute that probability, and scalable architecture, objective and data made it large. LLMs then become agents that act on phones, in games and in robots.
→ What can a university team actually build with this? - VBuild & nextWhat a research group builds, and where it is heading
- 8
Research Showcase: Medical, Multilingual and Math LLMs
HuatuoGPT and Apollo bring medical LLMs to hospitals and to 50 languages, AceGPT aligns an LLM with Arabic culture, and small models fine-tuned for math compete with GPT-4 on GSM8K. Agents simulate a stock market in TwinMarket.
→ Where is this research heading? - 9
Recursive Self-Improvement, Oversight and Your Views
Recursive self-improvement in messy real-world environments, and the challenge it raises: oversight that scales with capability. Four class surveys on AI and jobs, and a summary of the whole lecture.
2Try it in Colab
Three short notebooks turn the lecture into experiments. Click Open in Colab, then Runtime → Run all. Each ends with exercises. All three run on a free CPU runtime; a T4 GPU only makes Notebook 3 faster.
The classic NLP pipeline
- Watch keyword search match Ginger the dog to a spice
- Tag, parse and find entities with spaCy
- Build a rule-based slot filler, and see "there" and typos break it
.ipynb · CPUWhat is a language model?
- Build a bigram model from counts and see it give zero to a good sentence
- Score the chimpanzee's typing vs. "ChatGPT is all you need" with GPT-2
- Multiply out the chain rule, and write the generation loop yourself
.ipynb · CPUOne model, many tasks
- Keyword lexicon vs. a 0.5B LLM on sarcasm and negation
- Translate, classify, answer and extract slots with one prompt each
- See that every answer is still a next-token distribution
.ipynb · CPU (T4 GPU optional)3Concept index
Key terms of the lecture. Each link opens the video at the slide where the idea is explained.
4What's new in 2026
Even an introduction moves with the field. Six trends to watch, each tied to a slide of this year's lecture:
From models to harnesses
The course now has a lecture on LLM agents and harness engineering: building the system of tools, checks and feedback loops around a model (slide 16).
Measuring AI by task length
Slide 118 asks how long a task AI can finish. METR measured it: the length of tasks frontier models complete with 50% success has doubled about every seven months since 2019 (slide 118).
Tens of trillions of tokens
A 13-year-old has heard fewer than 100 million words; Qwen3 was pre-trained on 36 trillion tokens in 119 languages and dialects (slide 87).
Recursive self-improvement, in the open
AI that helps build better AI is the group's research direction (slide 111), and in 2026 it got a dedicated workshop at ICLR.
Oversight must scale too
As capability grows, human supervision stays flat. The open problem is oversight where AI helps generate feedback and monitor, so supervision grows with the model (slide 114).
5Quotes & reading
"You shall know a word by the company it keeps!"
The distributional hypothesis behind word2vec and GPT (slide 76), and Assignment 1.
"The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point."
The 1948 paper where n-gram models of English begin, the first entry on the timeline of Chapter 7.
"I propose to consider the question, 'Can machines think?'"
Turing's test is a conversation in natural language; the surveys in Chapter 9 ask what machines can do today.
"The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."
Why scalable objectives and compute (Chapter 7) replaced hand-built pipelines (Chapter 6).
Further reading
- Dan Jurafsky and James H. Martin: Speech and Language Processing (3rd ed. draft) — the standard textbook, free online; its n-gram chapter goes with Chapter 7
- The course's recommended reading: Hugging Face NLP Course · LLM Course: roadmaps and Colab notebooks
- On the Opportunities and Risks of Foundation Models (2021) · Sparks of Artificial General Intelligence: Early experiments with GPT-4 (2023)
- The milestones of slides 69–84: A Neural Probabilistic Language Model (2003) · word2vec (2013) · Attention Is All You Need (2017) · GPT-3: Language Models are Few-Shot Learners (2020)
- Andrej Karpathy: nanoGPT — one of the small GPT repos the lecture suggests you train in your first month (slide 46)