Mechanistic Interpretability
Techniques for tracing what happens inside a model — logging activations and patching them — to see how and where it forms an output.
Mechanistic interpretability is the practice of looking inside a model to understand how it reaches an output, rather than just observing what it produces. It treats the network's internal activations as something you can record, probe and even intervene on.
Two core techniques sit at its heart. Activation logging records the internal states — attention patterns, layer outputs, residual-stream values — during a forward pass, so you can see where in the network a fact or entity (say a brand name) first appears; a "logit lens" projects a mid-layer state onto the vocabulary to watch a token become probable. Activation patching goes further, swapping activations between runs to establish causation, not just correlation.
For AI visibility this is powerful: if you can see at which layer a model "decides" to associate a term with your brand, you can reason about how to strengthen it. It connects the attention mechanism to observable behaviour and underpins tools like our Tree Walker.
Related concepts
Mentioned in
-
We Tested an AI for Signs of Inner Experience. Nobody's Home.
Eight behavioural tests on Gemini found no evidence of AI consciousness, though results reveal how the model categorises and judges its own self-concept.
-
Mechanistic Interpretability of Google's Gemma - Safety and Personality Insights
Modifying specific self-attention layers in the Gemma language model alters its self-identity and safety responses, blurring real-world harm with video games.
-
Neural Circuit Analysis Framework for Brand Mention Optimization
This framework uses open-weight models like Gemma 3 Instruct to perform mechanistic brand positioning through direct neural circuit and activation analysis.
