Watch: Fanout Vectors
An analysis and reproduction of Google's query fanout research, demonstrating how vector-based diffusion models could replace text fanouts for faster search.
Transcript
Google's new research on query fanout could completely change how we search, potentially eliminating traditional text fanouts entirely. While many people skimming the research think it is about text, the model actually skips text generation altogether. Instead, it works directly with vectors. A small diffusion model is trained on vectors and outputs vectors, allowing for direct, highly efficient semantic matching. To test this, I reproduced the paper and trained the model myself. I simplified the process by skipping reinforcement learning, using a powerful Gemma teacher model to generate high-quality synthetic training data instead. The resulting model uses embeddings for document retrieval rather than traditional text queries. By using a one-bit model architecture and custom kernels, I achieved blazing speeds. While this is still just a research piece with no guarantee of a production release, it shows how Google could realistically make search much cheaper, faster, and more efficient.