Listen: Generative self-retrieval
A reasoning model surfacing facts from its own parameters by generating them as text, where those generated facts then condition and improve its final answer.
Transcript
How do reasoning language models pull deep facts out of their own memory? A concept called generative self-retrieval explains this process.
When a model works through a tough question, it writes down related facts as part of its chain of thought. By generating these facts, it brings them into its immediate workspace, which helps it recall the correct answer.
This is a lot like how human memory works, through a process called spreading activation. When you think of one concept, it naturally triggers related thoughts. For a language model, recalling neighboring facts builds a contextual bridge to an answer that would otherwise be out of reach.
For example, to name the tenth king of Nepal, a model might first list the nine kings who came before him. That list acts as scaffolding to help it reach the tenth. Researchers proved this by taking those intermediate facts and feeding them directly to a model with its reasoning turned off. The model still got the answer right, showing that the written facts themselves carry the benefit.
But this mechanism is as fragile as it is powerful. Because the model generates these facts from its own memory, it can easily make things up. If it hallucinates even one stepping-stone fact along the way, the final answer is highly likely to be wrong.