When one class in a training dataset vastly outnumbers another, causing a naive model to ignore the minority class — requires deliberate handling through weighted loss, resampling, or specialised loss functions.
Class imbalance occurs when the training dataset contains far more examples of one class than another. A dataset of web content where 95% of text tokens are ordinary text and 5% are anchor text is severely imbalanced. A dataset where 90% of queries don't need live grounding and 10% do is moderately imbalanced. In both cases a model trained naively — minimising average loss across all examples — will learn to mostly predict the majority class, because that's the path to low average loss.
A model that always predicts "not a link" for every token scores 95% accuracy on a dataset where 95% of tokens are non-links. But its recall for the link class is 0% — it never finds what it was built to find. Standard accuracy is a misleading metric under class imbalance; precision, recall, and F1 for the minority class are what matter.
Class-weighted loss — multiplying the loss from minority-class examples by a higher weight, so the model is penalised more heavily for getting them wrong. DEJAN's AI Content Detection model uses class-weighted training to handle the imbalance between human and AI-generated text in its training set.
Focal loss — a more principled approach that down-weights easy examples regardless of class, naturally causing hard minority examples to dominate training. Used in LinkjeBERT for the severe imbalance between link and non-link tokens.
Resampling — oversampling the minority class (creating duplicates or synthetic variants) or undersampling the majority class. DEJAN used oversampling with synthetic data generation to address class imbalance in the grounding classifier training set.
