Multi-label Classification
A classification task where each input can be assigned more than one label simultaneously — a query can be both informational and local; a sentence can be both positive and sarcastic.
What multi-label classification is
Multi-label classification is the task of assigning any number of labels from a predefined set to a single input. Unlike binary classification (two mutually exclusive states) or standard multi-class classification (one label from many), multi-label classification allows the model to output zero, one, or several labels at once. A search query can simultaneously be informational, local, and commercial. A product review can be both positive about delivery and negative about quality.
How it works
A multi-label classifier typically produces one independent probability score per label rather than a single probability distribution across all labels. Each score is compared to a threshold — often 0.5, but tunable per label — and any label whose score exceeds the threshold is assigned. Otsu's thresholding is one method DEJAN uses to set per-label thresholds automatically, rather than applying a single global cutoff to every class.
DEJAN's multi-label models
Several DEJAN models are multi-label classifiers. The Query Intent Classifier assigns queries to any combination of commercial, non-commercial, informational, navigational, transactional, commercial investigation, local, and entertainment labels — a query can qualify for multiple simultaneously. The Sentiment model outputs scores across a 7-point scale (very positive through very negative), and a text excerpt can register on multiple levels. The Universal Query Classifier explicitly supports arbitrary label taxonomies with multi-label assignment as its default mode.
Evaluation
Evaluating multi-label classifiers is more complex than single-label evaluation. Micro-averaged precision, recall, and F1 treat every label-instance pair equally; macro-averaged metrics compute per-label scores and average them, giving equal weight to rare and common labels. Which to report depends on whether rare labels matter as much as common ones for the use case.
Related concepts
Mentioned in
-
Releasing a new sentiment classification model.
Our new sentiment model is free on Hugging Face. It scores every heading, list item and sentence from very positive to very negative, gets 84% of labels exactly right and 95% within one point, and is built into DEJAN App and a public demo.
