Watch: Hyperparameters
Settings configured before training begins that control how a model learns — learning rate, batch size, epoch count, dropout rate — as distinct from the weights the model learns itself.
Transcript
When we train a machine learning model, we deal with two types of settings: parameters and hyperparameters. While the model learns its own parameters, like weights, from the training data, we have to set the hyperparameters ourselves before the process even begins. These settings control exactly how the training runs.
A few key hyperparameters make a massive difference. First is the learning rate, which controls the size of the steps the model takes to improve. Step too fast, and the model misses the mark; step too slow, and training takes forever. Then there is the batch size, which determines how many data points the model looks at before updating itself. We also set the number of epochs, which is how many times the model scans the entire dataset. To keep the model from memorizing the data too perfectly—a problem called overfitting—we use regularization techniques like dropout rates and weight decay.
Finding the perfect combination of these settings is a mix of intuition and systematic search. Practitioners often use grid search to try every combination, random search to test random mixes, or Bayesian optimization to make smarter, guided guesses. For most standard tasks, however, start with reliable default settings, and focus your energy on fine-tuning just the learning rate and the number of epochs.