Listen: Teaching a Model to Reason Before It Learns to Talk

Almost every artificial intelligence you hear about today is a massive transformer trained on a firehose of text. They learn language first, and reasoning comes along for the ride. But what if we did the opposite? What if we built a tiny model, under ten megabytes, that learns logic and reasoning first, and leaves language for later?

This experiment started with the Abstraction and Reasoning Corpus, a benchmark of grid puzzles designed to resist memorization. To solve them, you have to actually generalize from a few examples. The first model tested was a cellular automaton, where every cell only talks to its immediate neighbors. It worked beautifully for local tasks like recoloring and filling holes, but it failed completely on global tasks like rotation. That failure was incredibly useful. It showed exactly where local reasoning ends, and what the next building block needs to be.

Transformers did not win because they are the only road to intelligence. They won because they scale well and map beautifully onto modern hardware. But the human brain proves there are other, more efficient ways. The brain learns without backpropagation, runs continuously, and keeps memory and computation in the exact same place.

By building tiny, we can test these basic ingredients of intelligence one by one, watching which abilities switch on. We might not beat the giant models today, but we are drawing a map of what reasoning actually requires.