1The lecture map
Five acts, eleven chapters. First why medicine is worth it, then the HuatuoGPT story of data, evaluation and knowledge, then the capabilities a generalist needs (perception, reasoning, generation, languages) and what blocks them, then the claim that the next leap comes from environments — simulated patients, hospitals and cells — and finally what is deployed today and where it goes next. The thesis (slide 3): generalist medical AI is the goal, trustworthy medical AI is the way, environments are the engine. Click a chapter to watch it.
Chapter by chapter what each chapter covers, and the question it leaves for the next
- IWhyWhy a domain, why medicine, why now
- 1
Why Medical AI? Domains, Giants and Timing
A focused domain is a cheaper sandbox for the big AGI questions. Medicine helps everyone, is hard enough, attracts funding and serves the public good — and with aging populations, stronger models and real adoption, the gap is no longer exam scores but trust and workflow.
→ So how do you actually build a model that behaves like a doctor? - IIA first modelBuild it, test it, then fix what the tests reveal
- 2
The Story of HuatuoGPT: Data, Hybrid SFT and RLMF
Start from Huatuo-26M, about 26 million medical QA pairs, most of them real online consultations. Mix doctors' answers polished by an LLM with LLM answers pushed to act like doctors, then run RL where ChatGPT judges but a real doctor's answer anchors every judgment.
→ The reward goes up — but is it a better doctor? - 3
How Do We Evaluate a Medical LLM?
Pairwise judging by GPT-4 and by doctors, single-turn and multi-turn with a simulated patient, then the CMB exam benchmark — which showed HuatuoGPT talked like a doctor but did not know enough medicine. The most trustworthy evaluation does not scale.
→ How do you get real knowledge into the model? - 4
HuatuoGPT-II and the Idea of Generalist Medical AI
Rewrite medical text into instruction format and train in one stage; a fresh pharmacist exam shows it worked — and a year later that general models catch up. That points to a bigger goal: one generalist medical AI, reached by climbing a ladder of earned trust.
→ What capabilities does a generalist doctor need? - IIICapabilitiesSee, reason, create, speak every language
- 5
Perception: Vision, Speech, Signals and Long Context
Better data makes better multimodal models (ALLaVA, VLFeedback, PubMedVision for HuatuoGPT-Vision). Then speech, motion, EEG and the pulse in ShizhenGPT — and long context, because medical images are huge.
→ The model can perceive — can it reason in a way we can check? - 6
Reasoning and Generation: HuatuoGPT-o1 to MedGen
Math taught us verifiers and RL with checkable answers. Turning exam questions into open-ended problems with one answer brings that recipe to medicine in HuatuoGPT-o1. Then the model learns to produce speech, images, medical video and micro-worlds.
→ Can all this reach patients who do not speak English or Chinese? - 7
Multilinguality and Challenges on the Way
Languages complement each other: one model trained on six languages beats single-language models, and language-family experts scale Apollo-2 to 50. Then the reality check: scattered data, specialists versus generalists, interpretability, and benchmarks that saturate.
→ If static data is running out, where will the next gains come from? - IVEnvironmentsWhere medical AI can act, fail safely and learn
- 8
From Models to Environments: Patients and Hospitals
Trust is the intermediate step, and the model alone may have hit a wall. Build environments instead: AI standardized patients, patient trajectories and clinical world models, and an agentic hospital that speaks the language of a real hospital information system.
→ What lies below the patient, and above the hospital? - 9
Virtual Cells, Evidence, Benchmarks and HuatuoGPT-III
Virtual cells and digital twins below; community-maintained, evidence-based medicine above; live benchmarks as infrastructure underneath. The loop closes in HuatuoGPT-III: RL-only domain adaptation with rubric rewards.
→ Where is any of this used today? - VPracticeWhat is deployed today, and what comes next
- 10
Applications and Ecosystem: From Triage to Open Source
Region-wide triage in Longgang, on-device and specialist products, medical skills for agents and an in-hospital record assistant, health education — all shared as open models, datasets and benchmarks.
→ What should medical AI — and the people who build it — bet on next? - 11
Outlook: Five Bets and AI Entrepreneurship
Five bets for medical AI, and how AI changes building things: smaller teams, building in public, speed, fast-fading advantages and living products. Then the five takeaways.
2Try it in Colab
Three short notebooks turn the lecture into experiments with a small open model (Qwen2.5-0.5B-Instruct) and public medical datasets. Click Open in Colab, then Runtime → Run all. Each ends with exercises. All three run on a free CPU runtime; a T4 GPU (Runtime → Change runtime type) makes them faster and lets you try bigger models. These notebooks are about evaluating models. Nothing a model writes in them is medical advice.
Evaluate a small model on medical exam questions
- Score real MedQA (USMLE-style) questions from letter probabilities
- Shuffle the options and watch the score move (position bias)
- Check whether confidence predicts correctness
- Remove the options and see why open answers need a verifier
.ipynb · CPU or GPURetrieval-augmented answering with citations
- Build an evidence base from 1,000 PubMed abstracts (PubMedQA)
- Write a BM25 retriever and measure recall@k
- Compare closed-book answers with retrieved and oracle evidence
- Check citations automatically, and abstain when no evidence is found
.ipynb · CPU or GPUAn LLM judge: position bias and order swapping
- Judge pairs of answers with a small LLM, in both orders
- Measure how often the first answer wins regardless of content
- See the order-agreement check (as in HuatuoGPT's reward model) flag a biased judge, and averaging both orders recover the signal
- Anchor the judge with a doctor's reference answer (RLMF)
.ipynb · CPU or GPU3Concept index
Key terms of the lecture. Each link opens the video at the slide where the idea is explained.
4What's new in 2026
The lecture's own thesis is that the next gains come from environments, rubrics and trust rather than from more text (slide 167). Six things that changed recently:
Rubrics became rewards
Physician-written rubrics, first used to grade answers, now also train models. In the lecture, an 8B HuatuoGPT-III adapted to medicine with RL only (no SFT stage) scores 67.5 on HealthBench, above the proprietary models listed — while frontier models stay ahead on hard exam questions (slide 147).
Benchmarks that stay fresh
To beat saturation and leakage, benchmarks keep adding new cases and follow one patient over several stages and images, and experts write questions until every model fails them (slide 140, slide 142).
From datasets to environments
Simulated patients, clinical world models and digital hospitals let agents act, get outcome feedback and improve — in medicine, the “era of experience” argument (slide 109, slide 111).
Medical skills for general agents
Expertise is packaged as skills a general agent loads on demand, and the agent runs on a desktop box inside the hospital, so patient data stay in the building (slide 156, slide 157).
Virtual cells go zero-shot
Arc Institute's first Virtual Cell Challenge (2025) drew more than 5,000 participants from 114 countries; the 2026 edition asks for perturbation responses in cell lines the model has never seen. Beating simple baselines is still hard (slide 129).
Evidence at the point of care
Evidence-grounded assistants are part of daily practice: the lecture cites OpenEvidence's report that about 65% of US physicians use it. The open question is who maintains the evidence — the lecture proposes an open, community-edited “AI-native UpToDate” (slide 132, slide 134).
5Quotes & reading
"We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI)."
The goal named in the lecture's title (slide 47).
"Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients."
What “evidence-grounded” medical AI has to live up to (Chapter 9).
"Pre-training as we know it will unquestionably end."
Why adding more medical text stopped being enough, and reasoning came next.
"A new generation of agents will acquire superhuman capabilities by learning predominantly from experience."
The case for environments over datasets (Chapter 8).
Further reading
- Moor et al.: Foundation models for generalist medical AI (Nature, 2023) — the GMAI vision of Chapter 4
- HuatuoGPT · HuatuoGPT-II · HuatuoGPT-Vision · HuatuoGPT-o1 — the HuatuoGPT line, Chapters 2–6
- CMB: a comprehensive medical benchmark in Chinese · HealthBench — exams and rubrics (Chapter 3)
- Med-PaLM: LLMs encode clinical knowledge · AMIE: towards conversational diagnostic AI
- Apollo: multilingual medical LLMs · ShizhenGPT · Agent Hospital
- Code, models and data: FreedomIntelligence on GitHub and on Hugging Face