Otsu's Thresholding (for Queries)
An unsupervised method borrowed from computer vision that sets optimal per-label confidence cut-offs in query classification by maximizing the variance between positive and negative score clusters.
In search systems, classifying queries accurately is a constant challenge, especially because different types of queries produce very different score distributions. Instead of relying on a single, rigid confidence cutoff set by hand, search systems can use a technique called Otsu's Thresholding for Queries.
Originally borrowed from computer vision, this is an unsupervised, dynamic method that automatically calculates the optimal threshold for each individual label. It works by looking at the scores and finding the exact split point that maximizes the variance between positive and negative clusters. In simple terms, it separates "this label applies" from "it does not" as cleanly as the data allows.
Because this cutoff adapts to each label, it is highly effective for query classification and crucial decision points, like determining whether a query deserves grounding. It is precisely the kind of smart calibration a grounding classifier relies on to stay accurate and responsive to real-world search behavior.
What it is
Otsu's Thresholding (for Queries) is a dynamic, unsupervised method borrowed from computer vision and applied to search query classification. Rather than fixing one confidence cut-off by hand, it sets an optimal per-label threshold automatically by maximizing the variance between the positive and negative score clusters, in effect finding the split point that separates "this label applies" from "it does not" as cleanly as the data allows.
Because the cut-off adapts per label, it handles the reality that different query classes carry different score distributions. It is directly relevant to Query Classification and to decision points like Query Deserves Grounding, and it is the kind of calibration a system such as the Grounding Classifier relies on to stay accurate.