Interactive demo of per-layer embedding costs: step one token through eight decoder layers while an arithmetic meter fills and a fetch counter adds one tiny row per layer, then use a slider to reallocate the 5B parameters between matmuls and lookup tables and watch compute, fast memory and storage change.

One token's bill, layer by layer
Token walking the layer stack
Arithmetic done0%
Rows fetched0
Press advance: every layer multiplies with all of its weights, and fetches one small row.
8 layers drawn; the real stack is deeper. The ratio is the point.
The allocation dial
Share of the 5B living in PLE tables: 54%
plain dense model the real E2B split
Per-token compute2.3B
Fast memory needed2.3B
Ordinary storage (tables)2.7B
Runs like a 2.3B model. Storage stays 5B total; only the matmul share costs compute and fast memory.