Interactive demo of PLE trade-offs: the embedding row for the word bank stays identical across two different sentences while the attention output changes, plus a map of which Gemma 4 models use PLE, dense weights, or Mixture-of-Experts.

A lookup cannot read the sentence
Context-blind lookup the river bank PLE row for "bank" Fetched by ID alone Identical every time Attention output for "bank" Blended with "river" Changes with the sentence
the word we ask aboutcontext word
Where Google itself uses each trick
Gemma 4 family design choices E2B and E4BPLE, phone-sized 12B and 31BDense, no trick 26B A4BMoE, server trick
Click a group to see why that design fits its home.