Watch: Generative Reranking
Re-ranking where an LLM writes the ranked order itself, emitting candidate identifiers as tokens instead of scoring each candidate with a separate model.
Transcript
Generative reranking is a powerful way to order search results using a Large Language Model, or LLM. Instead of using a separate scoring system, the model reads a query and a list of candidate pages, then writes out the ranked order directly.
While this approach is highly accurate, traditional generative reranking is slow. Because the model has to generate the list token by token, the process takes longer as the list grows. Even worse, because the output is free text, the model can make mistakes, repeating some candidates or leaving others out entirely.
To solve this, researchers are changing how these rankings are decoded. A new method called hLLM reads the ranking from the model's hidden states using a small attention head. It guarantees a valid ranking using a classic mathematical algorithm, running over sixty times faster than older methods.
This technology is critical for search engine optimization, or SEO, because it is how generative AI systems decide which information is most relevant. But these models have strong position biases. For example, testing shows that Gemini picks the first page it reads ninety-two percent of the time. Because models like hLLM form their preferences while reading the candidates, before generating a single word, understanding these hidden biases is the key to mastering AI search.