In crisis or need support? Call or text 988 - 988 Suicide & Crisis Lifeline (24/7)

Voice Up®

Back to Video Modules
AI FOUNDATIONS · KARPATHY

Andrej Karpathy — Intro to Large Language Models

A One-Hour General-Audience Talk · Recorded November 2023 · Released Freely for Education

Andrej Karpathy · Intro to Large Language Models (2023)

Page Narrator

Key Themes from the Talk

An LLM Is Two Files

Karpathy's famous simplification: a large language model is a parameters file — the learned numbers, which for a big open model run to hundreds of gigabytes — plus a comparatively tiny program that runs them. The mystery has a file size, and he gives it.

✦ Precision

Training Compresses the Internet

Pretraining squeezes a vast slice of the internet into the parameters by teaching the model to predict the next word, billions of times. What results is a lossy compression of human text — which is why the base model dreams plausible continuations instead of giving reliable answers.

✦ Precision

Fine-Tuning Directs the Dream

An assistant is made, not born: supervised fine-tuning on labeled conversations, then reinforcement learning from human feedback. Karpathy's own caution is the point — fine-tuning directs the dreaming into helpful shapes; it does not remove the dreaming. Verify what matters.

✦ Humility

The LLM Operating System

His forward look: models that use tools, see and hear, reason more slowly in a System 2 style, improve themselves, and sit at the center of computing the way an operating system does — coordinating processes, memory, and tools. A new paradigm, arriving in pieces.

✦ Collaboration

A New Platform, A New Attack Surface

The final part covers jailbreaks, prompt injection, and data poisoning — attacks with no real precedent in earlier software, aimed at a system that reads instructions and data in the same stream. His conclusion is a planner's, not a panicker's: treat model output the way you would treat any untrusted source.

✦ Patience

Connected to the Voice Up Trilogy

The Naming Gap — Naming What Matters

Parameters, inference, fine-tuning, RLHF, prompt injection: this talk is a naming ceremony for the whole stack. Karpathy believes a general audience can hold the real vocabulary, and an hour later, his audience can.

A Time for Purpose — Living What You Name

Karpathy left the center of the industry and became one of its teachers. Purpose, in his case, looks like mastery turned outward: the person who helped build the models now spends his time making them legible to everyone else.

Miracle Has a Monster — Facing What Stands in the Way

The monster named here is unearned trust — fluent answers consumed as facts. Karpathy faces it in the open, from inside the field: the dreams are a feature of the compression, and the defense is a habit of verification.

⚡ 7-DAY MINI SPRINT

The New Operating System — 7-Day Sprint

A 7-day sprint built from the talk: the two files, training as compression, the making of an assistant, scale and tools, the LLM operating system, the security chapter, and a mental model of your own — tested by teaching it.

Take the Next Step

The 7-day sprint turns this hour into a durable mental model — including the habit Karpathy himself recommends: enjoy the fluency, verify the facts. Watch the talk, walk the sprint, and use these systems with your eyes open.

Voice Up® is a federally registered service mark (U.S. Reg. No. 8,148,220) owned by J Ellington Publishing LLC and operated by Voice Up Publishing Incorporated under permanent license.