Early Stopping
Halting training when validation performance stops improving rather than running for a fixed number of epochs — the primary defence against overfitting in practice.
When you train a machine learning model, deciding exactly when to stop is often a guessing game. If you stop too early, the model underperforms. If you train too long, it overfits, memorizing the training data instead of learning to generalize.
Early stopping solves this problem by turning that guess into an empirical decision. Instead of running for a fixed number of epochs, the training process constantly monitors a validation metric, like accuracy or loss, at regular intervals. When that metric stops improving, the training halts automatically.
This process relies on a setting called the patience parameter. Patience defines how many validation intervals the system will tolerate without any improvement before it officially shuts down. For example, a patience of three means the model gets three more chances to beat its best score. If it fails to do so, training stops.
Crucially, the final model deployed is not the one from the very last step, which has likely already begun to degrade. Instead, the system saves and loads the best checkpoint recorded during the entire run.
By letting the data dictate when to stop, you avoid the waste of training a model that has already peaked. This not only prevents overfitting, but it also saves valuable time and compute power.
What early stopping is
Early stopping is a training strategy that monitors a validation metric — typically F1, accuracy, or validation loss — after each evaluation interval and halts training automatically when the metric stops improving. Instead of training for a predetermined number of epochs, the model trains until further training provably makes things worse.
How it works in practice
A patience parameter defines how many evaluation intervals without improvement to tolerate before stopping. Patience of 3 means training continues for 3 more evaluations after the best-seen metric, and stops if nothing beats that best. The model saved at the best checkpoint is the one that gets deployed — not the final state at stopping time, which may already have degraded.
DEJAN's grounding classifier training validated every 500 steps and loaded the best model at end of training. LinkjeBERT trained for up to 10 epochs with early stopping to catch the optimal checkpoint before overfitting set in.
Why it is preferred over fixed epochs
Deciding in advance how many epochs to train requires guessing when the model will saturate. Early stopping replaces that guess with an empirical observation: keep training until the data tell you to stop. It also saves compute — if the model peaks at epoch 4 out of a planned 20, early stopping saves 16 epochs of wasted training.