Release: 2026/10/31 10:53 Reading: 0
Original author:Ignis Inference
Original source:https://www.youtube.com/embed/DrSSAkaLQqM
Mixture of experts explained: how a 125B-parameter LLM runs on a 32 GB GPU at about 100 tokens a second. Each token wakes 10 of 512 experts per layer; the experts wait in host RAM, a VRAM expert cache holds the ones in use, and the price is paid on PCIe. Full write-up: https://gpillon.github.io/ignis/why-ignis/flash-next/ ignis on GitHub: https://github.com/gpillon/ignis (Apache-2.0) Chapters 0:00 125B on a 32 GB card 0:20 512 experts, 10 per token 0:37 Read only what you use 0:52 Make them small 1:11 Where the experts live 1:30 How often the cache hits 1:47 The price: the link 2:08 The right question The numbers in this video Qwen3.8-Flash-Next (125B mixture of experts, 48 layers, 512 routed experts per layer, 10 per token, about 6B active), served by ignis on one RTX 5090 (32 GB) on a PCIe 3.0 x16 host (about 12 GB/s): - Experts stored in a trellis code at 2, 2.5, 3 or 4 bits per expert, 2.5 bits a weight on average with scales: 120.8B expert parameters in 37.7 GB, pinned in host RAM. - One lane, greedy, one prompt repeated, expert cache 15.70 GB: 101.5 tok/s (9.85 ms a token), a 96.59% expert hit rate, 51.6 MB a token over the link, 1.84 ms of each token waiting for copies (19%). - Three lanes, three different prompts, same server: hit rate 91.05%, the link busy 23.8 ms of a 27.5 ms round (87%), 15.0 ms of the round waiting for copies (55%). Sources: docs/findings/ in the repo (flash-next-decode-headroom), docs/adr/0044 (experts in trellis with a K per expert) and docs/user/concepts/models-and-artifacts.md. How this video was made Motion graphics only, every figure from the measurements above. Narration: Qwen3-TTS (AI voice). Music: ACE-Step 1.5 (AI-generated). Built with HyperFrames. #LLM #MixtureOfExperts #MoE
CRYPTO_BEN
2026-11-23 06:53
子敬.轮回珠宝
2026-11-23 06:53
kumis patiler
2026-11-23 06:53
Ha Ms
2026-11-23 06:53
CryptoMetric
2026-11-23 06:53
劇翻天 Drama Go
2026-11-22 16:36
CryptoMason
2026-11-22 16:22
Lý Tử Ly
2026-11-22 14:42
Pidoku
2026-11-22 14:15
Select Currency
US Dollar
USD
Chinese Yuan
CNY
Japanese Yen
JPY
South Korean Won
KRW
New Taiwan Dollar
TWD
Canadian Dollar
CAD
Euro
EUR
Pound Sterling
GBP
Danish Krone
DKK
Hong Kong Dollar
HKD
Australian Dollar
AUD
Brazilian Real
BRL
Swiss Franc
CHF
Chilean Peso
CLP
Czech Koruna KČ
CZK
Singapore Dollar
SGD
Indian Rupee
INR
Saudi Riyal
SAR
Vietnamese Dong
VND
Thai Baht
THB
Select Currency
US Dollar
USD-$
Chinese Yuan
CNY-¥
Japanese Yen
JPY-¥
South Korean Won
KRW -₩
New Taiwan Dollar
TWD-NT$
Canadian Dollar
CAD-$
Euro
EUR - €
Pound Sterling
GBP-£
Danish Krone
DKK-KR
Hong Kong Dollar
HKD- $
Australian Dollar
AUD-$
Brazilian Real
BRL -R$
Swiss Franc
CHF -FR
Chilean Peso
CLP-$
Czech Koruna KČ
CZK -KČ
Singapore Dollar
SGD-S$
Indian Rupee
INR -₹
Saudi Riyal
SAR -SAR
Vietnamese Dong
VND-₫
Thai Baht
THB -฿