Unlocking hidden skills inside Google's model

TL;DR. AI models hold skills that no prompt can unlock. A NeurIPS 2025 study trained small numbers of randomly chosen weights in Llama and Qwen models of four sizes, across ten tasks: 10 to 100 trained weights closed up to half the gap between the original model and a fully fine-tuned one. We repeated the test on Google's smallest Gemma model, Gemma 3 270M. The best of 20 prompt wordings got it to sort five numbers correctly 1% of the time. After we trained 1,000 of its 100 million weights (0.001%), it sorted correctly 97% of the time. The same method switched on reversing word lists (84%) and clean JSON output (99% with 30 weights), and lifted long addition, while multiplication stayed near 30% even with 100,000 trained weights.

The study we repeated

In Quantifying Elicitation of Latent Capabilities in Language Models (NeurIPS 2025), Donoway and colleagues from Anthropic, UC Berkeley, Constellation, Thinking Machines and Scale AI fine-tuned only a few randomly selected weights and measured how much of a task the model could then do. Across Llama 3 and Qwen 2.5 models from 1B to 8B parameters and ten benchmark tasks, including science questions (ARC), yes/no reading questions (BoolQ), story writing (TinyStories) and chess puzzles:

• 10 to 100 trained weights recovered up to 50% of the gap between the original model and full fine-tuning.
• 1,000s to 10,000s of weights recovered 95% of it.
• A model that already had a skill needed orders of magnitude fewer weights than a model learning it from scratch, so the number of weights needed shows how much of the skill was already there.

The authors' code is on GitHub. We wrote our own version for Gemma 3 270M, a model small enough to train on one 8 GB consumer GPU, and tested it on tasks a script can mark right or wrong.

No prompt got Gemma to sort

We asked the original Gemma 3 270M to sort five two-digit numbers in 20 different ways, 200 problems each: plain, polite, urgent, in capitals, in German, as a Python list, with a persona, step by step, "repeatedly take the smallest", and with one or three solved examples. The best wording, the Python list, scored 2 out of 200. Seventeen of the 20 scored zero.

The usual reply returns the list unchanged:

Sort these numbers in ascending order: 98, 37, 30, 59, 70. Answer with only the sorted numbers, separated by commas.

98, 37, 30, 59, 70

With a solved example in the prompt, the model sorted the example's numbers again and ignored the new list:

Sort the numbers in ascending order.
42, 17, 88, 23, 61 -> 17, 23, 42, 61, 88
98, 37, 30, 59, 70 ->

Here's the sorted list of numbers:
1.  **17**
2.  **23**
3.  **42**
4.  **61**
5.  **88**

Training a few weights unlocked it

For each task we trained 10, 30, 100, 300, 1,000 or 100,000 randomly chosen weights, kept every other weight frozen, and scored the model on 500 problems it had not seen in training.

Share of 500 test problems answered exactly right, by number of weights trained.

JSON at 10 to 100 weights, and sorting, reversing and multiplying at 100 to 1,000 weights, are the average of three runs with different random picks. The other cells are one run.

• JSON (extract a name, age and city from a sentence and reply with valid JSON) was a formatting habit. The original model already extracted the right values but wrapped them in markdown code fences, which made every reply invalid JSON. Thirty trained weights removed the fences.
• Sorting five two-digit numbers and reversing a list of five words went from zero to 97% and 84% with 1,000 trained weights.
• Multiplying two two-digit numbers reached 29% with 1,000 weights and 33% with 100,000. By the study's measure, this skill was mostly absent from the model.

Addition

Gemma 3 270M adds three-digit numbers correctly 84% of the time and 12-digit numbers almost never. We compared 1,000 trained weights with full fine-tuning of the whole model, three runs each:

The share of the gap that 1,000 weights closed fell from 85% at 3 digits to 29% at 12 digits.

Weaker results

We tried the same method on four other behaviours. Each result came with a problem that limits what it shows.

Which weights carry the sorting skill

We took the 1,000-weight sorting run (98.8% correct) and tested its changes:

• Removing each change on its own: 50 of the 1,000 lowered the score. The largest single drop came from one weight in the attention output of layer 10, which training moved from +0.004 to −1.15. Removing it cut sorting from 98.8% to 92.2%.
• Removing half at a time: removing either random half of the 1,000 changes left 25% and 15% correct.
• Training only the 50 most important positions from scratch: 24% correct, against 16% for 50 random positions from the same 1,000 and 97% for all 1,000.

The skill came from many small changes acting together. The 50 most damaging changes, trained alone, reached a quarter of the full result.

Where to pick the weights

Seven of the ten most damaging changes sat in attention output matrices, the part of each layer that writes attention's result back into the model. Restricting the random picks to those matrices helped:

Sorting, average of three runs. Attention output was ahead or level in every run.

We also tested a second way of choosing weights, which picks those whose input and output activations are most anti-correlated across the task's prompts. It needs only prompts, not answers. It added 2.2 to 4.6 points on 9- and 12-digit addition at 1,000 weights, and did worse than random picks on sorting with 10 to 100 weights (4.4% against 33% at 100 weights, one run each).

How we ran it

• Model: Gemma 3 270M, instruction-tuned.
• Weight pool: every weight in the attention and MLP matrices of the 18 transformer layers, 100,270,080 in total. Embeddings and norms were left out.
• Picking: uniform random over the pool, with a fixed seed per run.
• Training: only the picked weights change, as a sparse addition to the frozen model; AdamW, batch 32, at most 400 to 600 steps, early stopping on a validation set, loss on the reply tokens only. Each run checks afterwards that no more than the picked number of weights changed.
• Scoring: exact match with greedy decoding on held-out problems, using the best checkpoint on validation.
• Hardware: one 8 GB consumer GPU, 1 to 2 minutes per training run.

The study trained randomly chosen weights inside low-rank adapters on Llama and Qwen. We trained randomly chosen weights of the model itself on Gemma.

Scope

• One model, Gemma 3 270M, and mostly synthetic tasks with exact answers.
• At most three runs per setting, and one run for many cells, so gaps of a few points may not hold.
• Test sets of 500 problems (200 for the prompt test) carry about 2 to 4 points of sampling noise.
• Learning rates came from a small grid (0.01, 0.1, 1.0), not tuned per setting.