The field behind search: matching a query to documents in a collection and ranking them by estimated relevance. It supplies the retrieval half of RAG and of grounding in AI answers.
Information Retrieval (IR) is the field concerned with finding the documents in a collection that answer a query, and ranking them by estimated relevance. It predates the web, and it supplies the retrieval half of retrieval-augmented generation and of grounding in AI search.
Lexical retrieval matches terms. An inverted index maps each term to the documents containing it, and a scoring function such as BM25 weights matches by term frequency, inverse document frequency and document length. Dense retrieval instead maps query and document into one embedding space and scores by cosine similarity, which matches meaning where the words differ. Production systems run both and fuse the results, because lexical matching still wins on rare terms, names and product codes.
IR is where precision and recall come from, alongside ranked measures: precision at k, mean reciprocal rank, and normalised discounted cumulative gain, which rewards putting relevant documents near the top. Retrieve-then-rerank is the standard pipeline, with a cheap retriever pulling hundreds of candidates and an expensive cross-encoder reordering the head of that list.
An answer engine runs an IR system first and a language model second. Retrieved passages become grounding snippets in the model's context window, so a page the retriever never returns cannot be cited however well it is written.
