F1 Score
The harmonic mean of precision and recall — the standard single-number summary of classifier performance when both false positives and false negatives matter.
When you are evaluating a machine learning model, relying on simple accuracy can be highly misleading. If ninety-five percent of your dataset is human-written text, a model that simply labels everything as human-written will be ninety-five percent accurate, even though it is completely useless at detecting AI.
That is why machine learning pipelines often rely on the F1 score.
The F1 score is a single metric that balances precision and recall. Precision is about how many of the model's positive predictions were actually correct, while recall is about how many of the actual positive cases the model managed to find.
Instead of a simple average, the F1 score uses what is called a harmonic mean. This math is crucial because it heavily penalizes extreme imbalances. If a model has perfect precision but zero recall, a regular average would give it a passing grade, but the F1 score drops it straight to zero.
You can also adjust this metric depending on your goals. An F-two score weights recall more heavily, which is useful when missing a positive case is a major problem. An F-point-five score does the opposite, prioritizing precision. For complex models with multiple labels, you can use macro-averages to treat every category equally, or micro-averages to pool all the data together.
Ultimately, because the F1 score focuses on how well a model actually handles the cases that matter, it is the industry standard for selecting and shipping the best possible model checkpoints.
What the F1 score is
The F1 score is the harmonic mean of precision and recall. It combines both metrics into one number, penalising models that sacrifice one for the other. A model with perfect precision but zero recall scores F1 = 0. A model with high precision and high recall scores close to 1. The harmonic mean is used rather than the arithmetic mean because it punishes extreme imbalances more heavily.
Formula: F1 = 2 × (precision × recall) / (precision + recall)
Why it is used as the primary evaluation metric
Accuracy — the fraction of all predictions that are correct — is misleading when classes are imbalanced. A model that predicts "not AI-generated" for every input will score 95% accuracy if 95% of the dataset is human-written, while completely failing its actual purpose. F1 is immune to this: it focuses only on how well the model handles positive cases, making it the standard metric for binary and multi-label classification tasks where the positive class is what matters.
DEJAN's model training pipelines use F1 as the primary metric for checkpoint selection — the checkpoint with the highest validation F1 is the one that ships.
F1 variants
The standard F1 weights precision and recall equally. F-beta scores shift the balance: F2 weights recall twice as heavily as precision (useful when missing positives is worse than false alarms); F0.5 weights precision more heavily. For multi-label models, macro F1 averages per-label scores equally; micro F1 aggregates across all label-instance pairs before computing the score.