← all concepts

Information Gain

How much new information a document adds beyond what is already known; defined in information theory as a drop in entropy, and applied in search to reward content that adds rather than repeats.

Information Gain measures how much new information a piece of content adds about a topic, beyond what is already known. In information theory it has a precise definition: the reduction in entropy that comes from learning something, written \(\text{IG} = H_{\text{before}} - H_{\text{after}}\) and equal to the Kullback-Leibler divergence between the distributions before and after. It is the standard criterion for choosing splits in decision trees and for feature selection in machine learning, and it is related to Kolmogorov complexity, which counts the information in a single object rather than a distribution.

In search and AI the term names the same idea applied to documents: a page scores well on information gain when it contributes something the already-ranked pages do not, rather than restating the consensus. Google holds a patent on contextual estimation of link information gain that describes ranking documents by how much they add relative to what a reader has already seen. This document-level score is not a confirmed live ranking signal, so treat it as a working hypothesis drawn from that patent, not as a measured factor.

The idea matters for AI visibility because systems that retrieve and cite sources have reason to prefer pages that carry information not found elsewhere. Content that only echoes what a model already holds, or what other pages already say, gives an answer engine little reason to pull it in. Original data, first-hand testing, and specific findings are what raise a page's information gain, and with it the odds of being grounded and cited. This is a lever we use directly in our own AI Visibility work.

Concept