Watch: Loss Function
The mathematical measure of how wrong a model's predictions are — the quantity training minimises, and the choice of which shapes everything about how the model learns.
Transcript
In machine learning, a loss function is the ultimate reality check. It measures the gap between what a model predicts and the actual, correct answer. For every single training example, the loss function compares the prediction to the true label and spits out a single number. The lower this number, the better the model is performing. Training is simply the process of adjusting the model's weights over time to drive this loss as close to zero as possible.
Different tasks require different loss functions. For binary classifiers, the standard choice is binary cross-entropy, which heavily penalizes confident but incorrect guesses. If you are dealing with multiple, mutually exclusive categories, you use categorical cross-entropy.
But sometimes, the data is highly unbalanced. When one class is extremely rare, a model can get lazy by simply predicting the common class every time. In these cases, focal loss is used. Focal loss down-weights easy, well-classified examples, forcing the model to focus its learning on the hard, rare cases.
It is also important to separate loss from evaluation metrics. Loss is what the training process optimizes under the hood to calculate gradients. Metrics, on the other hand, are what humans actually care about, like precision, recall, and the F-one score. While loss and metrics are closely correlated, loss is what guides the training, while metrics tell us how the model will perform in the real world.