Release: 2026/11/04 13:16 Reading: 0
Original author:Curiosity, Clearly
Original source:https://www.youtube.com/embed/tR12UfyJvTw
Mathematics for AI 02 Why does a fair coin give one bit of information, while one of eight equally likely cards gives three? Follow the choices, the binary tree, and the logarithm to see how probability becomes a measure of information. In episode 2 of Mathematics for AI, Emma, Aoi, and Norm explain self-information (surprisal), why the unit uses base two, and why logarithms let information from successive steps add up. Then we use a small language-model example to distinguish scoring an observed token during training from choosing the next token during generation. 00:00 A measure for surprise 00:57 One balanced choice, one bit 01:59 Why a logarithm? 05:28 The model matters 06:21 Scoring an AI prediction 09:21 One outcome, then an average The central definition is I = −log₂(p), measured in bits. Here, surprise is a mathematical quantity under a specified probability model, not a measurement of emotion, importance, or truth. The language-model probabilities are illustrative examples, not measurements of a particular AI system. Series: Mathematics for AI 01 — Bayes' rule: updating beliefs 02 — Information, surprise, and bits (this episode) 03 — Entropy: measuring uncertainty (next) 04 — Cross-entropy and prediction loss 05 — Probability calibration 06 — Gradient descent and learning Future topics and order are provisional. References and further reading: Claude E. Shannon, A Mathematical Theory of Communication (1948): https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.1538-7305.1948.tb01338.x Stanford EE276, Information Theory lecture notes: https://web.stanford.edu/class/ee276/files/previous_notes/lecture_2.pdf Hugging Face Transformers, Perplexity: https://huggingface.co/docs/transformers/perplexity Hugging Face Transformers, Generation strategies: https://huggingface.co/docs/transformers/generation_strategies Original diagrams and animation: Curiosity, Clearly. Fictional illustrated presenters and synthetic character voices. If the explanation helped, like the video and subscribe. What everyday prediction would you measure in bits? Tell us in the comments. #InformationTheory #MathematicsForAI #Bits Series playlist: https://www.youtube.com/playlist?list=PLGi5Uwmf0LpY Previous episode: https://youtu.be/kS9gsyHW0yk
kumis patiler
2026-11-23 02:00
Ha Ms
2026-11-23 02:00
CRYPTO_BEN
2026-11-23 02:00
子敬.轮回珠宝
2026-11-23 02:00
CryptoMetric
2026-11-23 02:00
劇翻天 Drama Go
2026-11-22 16:36
CryptoMason
2026-11-22 16:22
Lý Tử Ly
2026-11-22 14:42
Pidoku
2026-11-22 14:15
Select Currency
US Dollar
USD
Chinese Yuan
CNY
Japanese Yen
JPY
South Korean Won
KRW
New Taiwan Dollar
TWD
Canadian Dollar
CAD
Euro
EUR
Pound Sterling
GBP
Danish Krone
DKK
Hong Kong Dollar
HKD
Australian Dollar
AUD
Brazilian Real
BRL
Swiss Franc
CHF
Chilean Peso
CLP
Czech Koruna KČ
CZK
Singapore Dollar
SGD
Indian Rupee
INR
Saudi Riyal
SAR
Vietnamese Dong
VND
Thai Baht
THB
Select Currency
US Dollar
USD-$
Chinese Yuan
CNY-¥
Japanese Yen
JPY-¥
South Korean Won
KRW -₩
New Taiwan Dollar
TWD-NT$
Canadian Dollar
CAD-$
Euro
EUR - €
Pound Sterling
GBP-£
Danish Krone
DKK-KR
Hong Kong Dollar
HKD- $
Australian Dollar
AUD-$
Brazilian Real
BRL -R$
Swiss Franc
CHF -FR
Chilean Peso
CLP-$
Czech Koruna KČ
CZK -KČ
Singapore Dollar
SGD-S$
Indian Rupee
INR -₹
Saudi Riyal
SAR -SAR
Vietnamese Dong
VND-₫
Thai Baht
THB -฿