Interactive diagram of the Gemma 4 E2B model: inputs flow through encoders into a stack of decoder layers that predict the next token, with panels on attention windows and effective versus total parameters.

The whole model, input to output
Gemma 4 E2B pipeline Text Image Video Audio TokenizerText into tokens Vision encoderPixels into vectors Audio encoderSound into vectors Decoder stackSame layer, repeated Next tokenLoops back as input
Click any block above for a plain-English note.
Inside one decoder layer
One Gemma 4 decoder layer Token vector in RMSNorm (pre) Self-attentionMixes info between tokens PLE lookupLayer's own memory RMSNorm (post) MLPFeed-forward thinking step Updated vector out
Click a stage above to see what it does.
Attention window
Local versus global attention now Local layer: the current token sees only the last 512 tokens
Effective vs total parameters
2.3B effective, does the thinking5B total stored
2.3B

The gap is the PLE tables. They are big to store but only used for cheap lookups, so the compute cost matches a 2.3B model. Total footprint runs under 1.5 GB, small enough for a phone.