In-Context Ranking
Re-ranking candidate documents for a query by feeding them all into an LLM's context and letting it pick the most relevant.
In-context ranking is a powerful new way generative AI systems decide which information is most relevant. Instead of relying on traditional keyword scores, a large language model is given a query, a list of candidate documents, and a task description all at once. The model reads everything in its context window and decides on the best order. This is a core part of modern relevance engineering, and it is reshaping how search engine optimization works for AI.
The major drawback of this approach is the cost. Because the attention mechanism in language models scales quadratically, doubling the number of documents can quadruple the required computing power. This bottleneck makes ranking large lists of candidates highly impractical.
To solve this problem, a new technique called BlockRank is being used to exploit the block-sparse structure of attention. This allows systems to leverage the deep contextual understanding of language models for ranking, without the massive computational bottleneck.
In-context ranking (ICR) is a way of re-ordering candidate documents for a query by placing the query, the document list and a task description directly in a language model's context window and letting the model identify the most relevant items. It leans on the LLM's contextual understanding instead of a separate scoring model.
The catch is cost. The attention mechanism scales quadratically with input length, so doubling the number of documents can quadruple the compute — impractical for large candidate lists. This is the bottleneck that BlockRank tackles by exploiting the block-sparse structure of attention.
ICR matters for AI SEO because it's increasingly how relevance gets decided inside generative systems: not classic keyword scoring, but a model reading candidates and choosing. It's a core part of modern relevance engineering.