Multi-label Classification
A classification task where each input can be assigned more than one label simultaneously — a query can be both informational and local; a sentence can be both positive and sarcastic.
In machine learning, we often need to categorize data that does not fit into a single, neat box. While traditional classification forces us to choose just one label, multi-label classification allows us to assign any number of labels to a single input. A search query can be both informational and local. A product review can praise a fast delivery while simultaneously criticizing the item's quality.
Instead of distributing a single probability score across all categories, a multi-label classifier calculates an independent score for each individual label. If a score passes a certain threshold, that label is applied. While many systems use a flat threshold of point-five, advanced models use techniques like Otsu's thresholding to automatically calculate a unique, optimal cutoff for each specific class.
This approach is central to several proprietary models developed by DEJAN. Their Query Intent Classifier can tag a search term with any combination of intents, from commercial and transactional to local and entertainment. Their Sentiment model evaluates text across a seven-point scale, capturing mixed emotions in a single excerpt. Additionally, their Universal Query Classifier uses multi-label assignment as its default mode to support complex, custom taxonomies.
Measuring the success of these models is unique. Because inputs can have multiple tags, standard metrics must be adapted. Teams use micro-averaging to treat every single label decision equally, or macro-averaging to give equal weight to each category, ensuring that rare labels are not overshadowed by common ones.
What multi-label classification is
Multi-label classification is the task of assigning any number of labels from a predefined set to a single input. Unlike binary classification (two mutually exclusive states) or standard multi-class classification (one label from many), multi-label classification allows the model to output zero, one, or several labels at once. A search query can simultaneously be informational, local, and commercial. A product review can be both positive about delivery and negative about quality.
How it works
A multi-label classifier typically produces one independent probability score per label rather than a single probability distribution across all labels. Each score is compared to a threshold — often 0.5, but tunable per label — and any label whose score exceeds the threshold is assigned. Otsu's thresholding is one method DEJAN uses to set per-label thresholds automatically, rather than applying a single global cutoff to every class.
DEJAN's multi-label models
Several DEJAN models are multi-label classifiers. The Query Intent Classifier assigns queries to any combination of commercial, non-commercial, informational, navigational, transactional, commercial investigation, local, and entertainment labels — a query can qualify for multiple simultaneously. The Sentiment model outputs scores across a 7-point scale (very positive through very negative), and a text excerpt can register on multiple levels. The Universal Query Classifier explicitly supports arbitrary label taxonomies with multi-label assignment as its default mode.
Evaluation
Evaluating multi-label classifiers is more complex than single-label evaluation. Micro-averaged precision, recall, and F1 treat every label-instance pair equally; macro-averaged metrics compute per-label scores and average them, giving equal weight to rare and common labels. Which to report depends on whether rare labels matter as much as common ones for the use case.