Price: $0.05520 -0.0059%
Market Cap: $8.66B 0.4163%
Volume (24h): 398.97M 0%
Dominance: 0.4163%
Price: $0.05520 -0.0059%
Market Cap: $8.66B 0.4163%
Volume (24h): 398.97M 0%
Dominance: 0.4163% 0.4163%
  • Price: $0.05520 -0.0059%
  • Market Cap: 8.66B 0.4163%
  • Volume (24h): 398.97M 0%
  • Dominance: 0.4163% 0.4163%
  • Price: $0.05520 -0.0059%
Home > 视频 > Mathematics for AI 02 | Why Information Is Measured in Bits

Mathematics for AI 02 | Why Information Is Measured in Bits

Release: 2026/11/04 13:16 Reading: 0

Original author:Curiosity, Clearly

Original source:https://www.youtube.com/embed/tR12UfyJvTw

Mathematics for AI 02 Why does a fair coin give one bit of information, while one of eight equally likely cards gives three? Follow the choices, the binary tree, and the logarithm to see how probability becomes a measure of information. In episode 2 of Mathematics for AI, Emma, Aoi, and Norm explain self-information (surprisal), why the unit uses base two, and why logarithms let information from successive steps add up. Then we use a small language-model example to distinguish scoring an observed token during training from choosing the next token during generation. 00:00 A measure for surprise 00:57 One balanced choice, one bit 01:59 Why a logarithm? 05:28 The model matters 06:21 Scoring an AI prediction 09:21 One outcome, then an average The central definition is I = −log₂(p), measured in bits. Here, surprise is a mathematical quantity under a specified probability model, not a measurement of emotion, importance, or truth. The language-model probabilities are illustrative examples, not measurements of a particular AI system. Series: Mathematics for AI 01 — Bayes' rule: updating beliefs 02 — Information, surprise, and bits (this episode) 03 — Entropy: measuring uncertainty (next) 04 — Cross-entropy and prediction loss 05 — Probability calibration 06 — Gradient descent and learning Future topics and order are provisional. References and further reading: Claude E. Shannon, A Mathematical Theory of Communication (1948): https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.1538-7305.1948.tb01338.x Stanford EE276, Information Theory lecture notes: https://web.stanford.edu/class/ee276/files/previous_notes/lecture_2.pdf Hugging Face Transformers, Perplexity: https://huggingface.co/docs/transformers/perplexity Hugging Face Transformers, Generation strategies: https://huggingface.co/docs/transformers/generation_strategies Original diagrams and animation: Curiosity, Clearly. Fictional illustrated presenters and synthetic character voices. If the explanation helped, like the video and subscribe. What everyday prediction would you measure in bits? Tell us in the comments. #InformationTheory #MathematicsForAI #Bits Series playlist: https://www.youtube.com/playlist?list=PLGi5Uwmf0LpY Previous episode: https://youtu.be/kS9gsyHW0yk

Selected Topics

  • Dogecoin whale activity
    Dogecoin whale activity
    Get the latest insights into Dogecoin whale activities with our comprehensive analysis. Discover trends, patterns, and the impact of these whales on the Dogecoin market. Stay informed with our expert analysis and stay ahead in your cryptocurrency journey.
  • Dogecoin Mining
    Dogecoin Mining
    Dogecoin mining is the process of adding new blocks of transactions to the Dogecoin blockchain. Miners are rewarded with new Dogecoin for their work. This topic provides articles related to Dogecoin mining, including how to mine Dogecoin, the best mining hardware and software, and the profitability of Dogecoin mining.
  • Spacex Starship Launch
    Spacex Starship Launch
    This topic provides articles related to SpaceX Starship launches, including launch dates, mission details, and launch status. Stay up to date on the latest SpaceX Starship launches with this informative and comprehensive resource.
  • King of Memes: Dogecoin
    King of Memes: Dogecoin
    This topic provides articles related to the most popular memes, including "The King of Memes: Dogecoin." Memecoin has become a dominant player in the crypto space. These digital assets are popular for a variety of reasons. They drive the most innovative aspects of blockchain.