Release: 2026/11/01 22:00 Reading: 0
Original author:Stack Zero
Original source:https://www.youtube.com/embed/oT6WzDMksgI
How does AI actually work? Every answer from a chatbot like ChatGPT, Claude or Gemini is written one token at a time, by a large language model guessing the next word. This visual explainer follows one prompt through the whole machine. It starts in 1948, when Claude Shannon built a sentence by flipping through a book and guessing word after word. It almost worked, then fell apart. Today the goal is the same, but the machine can look back at everything it has read. We follow "The first person to walk on the Moon was…" from tokens to vectors, through attention and 96 layers, to a list of probabilities and the one word that gets picked. Then we look at training, why models sound confident when they're wrong, and how far prediction can go. In this video: • What a large language model (LLM) is, and what "parameters" are • Tokens: how text is cut into pieces and turned into numbers • Vectors and word embeddings (Bengio, 2003) • Attention: queries, keys and values, explained visually (Vaswani et al., 2017) • Softmax, probabilities and temperature: how the next token is chosen • Training, fine-tuning and human feedback • Why AI hallucinates, context windows and knowledge cutoffs CHAPTERS How next-word prediction models actually generate text. See the logic behind probabilistic language models and attention. Most people assume AI models think like humans, but they are actually statistical engines. This film breaks down the mechanics of how next-word prediction models calculate the probability of the next token based on training data. We explore the original experiments by Claude Shannon and how they evolved into the architecture powering modern systems. Beyond simple probability, we examine word vectors and attention mechanisms to show how machines process context. You will learn why these systems sometimes produce nonsensical outputs and how they map language into mathematical space. Understanding these core principles is essential for anyone building with or researching large language models. Stack Zero. Complex Systems. Clear Explanations. Real Impact. Subscribe for weekly breakdowns of the technology behind modern software. 0:00 A 1948 experiment in guessing words 0:40 Today's chatbots: still guessing the next word 1:41 What a large language model is 2:02 Tokens and the prediction loop 2:41 Why Shannon's sentence fell apart 2:56 Shannon's human guessing game (1951) 3:32 Counting words: n-grams 3:57 Word vectors (2003) 4:19 Attention arrives (2014) 4:41 Transformers: "Attention Is All You Need" (2017) 5:07 Inside the model: one prompt, step by step 5:15 Step 1: Tokens 5:38 Step 2: Vectors 6:01 Step 3: Attention (query, key, value) 6:49 Attention heads 7:04 96 layers and feed-forward networks 7:33 Step 4: Probabilities and softmax 7:48 Step 5: Choosing the next token (greedy, sampling, temperature) 8:14 The loop: one token at a time 8:33 Training: Shannon's game at scale 9:16 From predictor to assistant 9:34 One loop, many products 9:45 Why AI hallucinates 10:18 Context windows, cutoffs and sources 10:37 Recap: the whole machine 10:59 How far can prediction go? Numbers on screen are illustrative unless a source is shown. SOURCES • Shannon (1948), A Mathematical Theory of Communication: https://people.csail.mit.edu/dmoshkov/courses/codes/shannon1948.pdf • Shannon (1951), Prediction and Entropy of Printed English: https://sites.socsci.uci.edu/~rfutrell/teaching/itl-davis/readings/shannon1951prediction.pdf • Bengio et al. (2003), A Neural Probabilistic Language Model: https://www.jmlr.org/papers/v3/bengio03a.html • Bahdanau, Cho & Bengio (2014), Neural Machine Translation by Jointly Learning to Align and Translate: https://arxiv.org/abs/1409.0473 • Vaswani et al. (2017), Attention Is All You Need: https://arxiv.org/abs/1706.03762 • Brown et al. (2020), Language Models are Few-Shot Learners (GPT-3): https://arxiv.org/abs/2005.14165 • Meta (2024), Introducing Meta Llama 3: https://ai.meta.com/blog/meta-llama-3/ • OpenAI Help, What are tokens and how to count them: https://help.openai.com/en/articles/4936856 • Geva et al. (2021), Transformer Feed-Forward Layers Are Key-Value Memories: https://arxiv.org/abs/2012.14913 • Ouyang et al. (2022), Training language models to follow instructions with human feedback: https://arxiv.org/abs/2203.02155 • OpenAI (2025), Why language models hallucinate: https://openai.com/index/why-language-models-hallucinate/ • Olsson et al. (2022), In-context Learning and Induction Heads: https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html Stack Zero explains the systems behind everyday technology, from the ground up. #AI #LLM #HowAIWorks
kumis patiler
2026-11-23 06:22
Ha Ms
2026-11-23 06:22
CRYPTO_BEN
2026-11-23 06:22
子敬.轮回珠宝
2026-11-23 06:22
CryptoMetric
2026-11-23 06:22
劇翻天 Drama Go
2026-11-22 16:36
CryptoMason
2026-11-22 16:22
Lý Tử Ly
2026-11-22 14:42
Pidoku
2026-11-22 14:15
Select Currency
US Dollar
USD
Chinese Yuan
CNY
Japanese Yen
JPY
South Korean Won
KRW
New Taiwan Dollar
TWD
Canadian Dollar
CAD
Euro
EUR
Pound Sterling
GBP
Danish Krone
DKK
Hong Kong Dollar
HKD
Australian Dollar
AUD
Brazilian Real
BRL
Swiss Franc
CHF
Chilean Peso
CLP
Czech Koruna KČ
CZK
Singapore Dollar
SGD
Indian Rupee
INR
Saudi Riyal
SAR
Vietnamese Dong
VND
Thai Baht
THB
Select Currency
US Dollar
USD-$
Chinese Yuan
CNY-¥
Japanese Yen
JPY-¥
South Korean Won
KRW -₩
New Taiwan Dollar
TWD-NT$
Canadian Dollar
CAD-$
Euro
EUR - €
Pound Sterling
GBP-£
Danish Krone
DKK-KR
Hong Kong Dollar
HKD- $
Australian Dollar
AUD-$
Brazilian Real
BRL -R$
Swiss Franc
CHF -FR
Chilean Peso
CLP-$
Czech Koruna KČ
CZK -KČ
Singapore Dollar
SGD-S$
Indian Rupee
INR -₹
Saudi Riyal
SAR -SAR
Vietnamese Dong
VND-₫
Thai Baht
THB -฿