The mathematical measure of how wrong a model's predictions are — the quantity training minimises, and the choice of which shapes everything about how the model learns.
A loss function measures the gap between what the model predicted and what the correct answer was. For each training example, the model makes a prediction, the loss function compares it to the true label, and produces a scalar number — the loss. Lower loss means better predictions. Training is the process of adjusting the model's weights to reduce this number over time.
Binary cross-entropy — the standard loss for binary classifiers. It penalises confident wrong predictions very heavily and tolerates uncertain correct ones. Used in the DEJAN grounding classifier and AI content detection model.
Categorical cross-entropy — the multi-class equivalent, used when outputs are mutually exclusive categories.
Focal loss — a variant of cross-entropy that down-weights easy, well-classified examples so the model focuses its learning on hard cases. LinkjeBERT uses focal loss (γ=2.0) because link tokens are rare — most tokens in any document are ordinary text, not anchor text — making the positive class very sparse. Focal loss prevents the model from ignoring rare positives by coasting on easy negatives.
Loss is what training optimises; metrics are what you actually care about. A model trained to minimise binary cross-entropy is evaluated using precision, recall, and F1 — because those tell you what the model is actually doing in the real task, not just how small its gradient signals are. The two are correlated but not identical.
