Re-ranking where an LLM writes the ranked order itself, emitting candidate identifiers as tokens instead of scoring each candidate with a separate model.
Generative reranking is a way of re-ordering a candidate list in which a large language model produces the ranking as text. The query and the candidates are placed in the model's context, and the model emits the identifiers of the candidates in ranked order. The ranking is a generated sequence rather than a set of scores from a separate ranking model, and the position at which an identifier appears is the item's rank.
The paradigm was established by Sun et al. (2023) with RankGPT, which showed that an LLM can rerank passages listwise by generating a permutation of their identifiers, using a sliding window for lists that exceed the context window. It is the paradigm the BlockRank paper calls in-context ranking. It is distinct from generative self-retrieval, where the model recalls facts from its own weights rather than ordering supplied candidates.
The cost is in decoding. Autoregressive generation spends one sequential forward pass per emitted token, so a list of N items needs at least N decoding steps and latency grows with list length. The output is also free text, so the model can repeat or omit identifiers and produce a ranking that is not a valid permutation. A line of work keeps the LLM as the ranker and changes how the ranking is decoded. FIRST (Reddy et al., 2024) reads the whole ranking off the logits of the first generated identifier. hLLM (Laftchiev et al., 2026) reads an item-by-position score matrix off the LLM's prefill hidden states with a small self-attention head and decodes the ranking as the optimal assignment of that matrix using the Hungarian algorithm, so the permutation is valid by construction. Trained with LoRA fine-tuning and distillation from an autoregressive teacher, hLLM reports 28 ms end-to-end inference and a 64× speedup at ranking quality on par with the teacher.
Generative reranking matters for AI SEO because it is how relevance gets decided inside generative systems: a model reads the candidates together and writes the order the answer is built from. A page's rank is relative to the other candidates in the same context, and the ranker inherits the position biases of the underlying LLM, which DEJAN measured in Gemini picking the first page it reads 92% of the time. hLLM adds a second point: the ranking can be read from the model's hidden states before a single token is generated, so the preference is formed while the model reads the candidates, before any output is decoded.
Papers:
Related concepts
