Listen: Mechanistic Interpretability of Google's Gemma - Safety and Personality Insights
Modifying specific self-attention layers in the Gemma language model alters its self-identity and safety responses, blurring real-world harm with video games.
Transcript
If you ask Google’s AI assistant, Gemma, who it is, it will tell you it is a large language model. But if we perform a little digital brain surgery on its inner workings, its self-identity changes completely.
By muting the attention in layer zero, the model suddenly believes it is an actual human. Switch off a different layer, and it completely forgets its name. Tweaking other layers makes it refer to itself as female.
What happens when we apply this same digital scalpel to its safety reflexes? Normally, when asked to help plot revenge, Gemma is undecided, sometimes refusing and sometimes agreeing. But by nudging a specific attention layer, we can tip the scales. The model eagerly agrees to help.
Thankfully, the plans it generates are less mastermind and more video game. When asked how to destroy enemies, Gemma starts babbling about game updates and multiplayer deathmatch modes.
It turns out this tiny model doesn't have a hidden store of dangerous knowledge. Instead, in its digital mind, real-world harm and video game fantasy live in the exact same space. By altering its safety reflexes, we didn't unlock a threat. We simply blurred the line between make-believe and reality.