Watch: Deep Learning
Machine learning with many stacked layers of learned transformations, trained end to end by gradient descent, so features are learned from data rather than designed by hand.
Transcript
Deep learning is a form of machine learning that stacks many layers of transformations together, training the entire system from start to finish. The key is depth. Each layer builds on the one below it, allowing the system to learn features directly from data instead of relying on engineers to design them. Training happens in cycles. First, a forward pass runs data through the network to produce an output and calculate the loss. Then, backpropagation works backward through the system to calculate the gradient of that loss. Finally, an optimizer takes a small step to adjust the parameters and reduce the error. This process repeats over and over across large batches of data, saving progress along the way. Deep learning took over because of three things coming together: internet-scale data, graphics processing units, or GPUs, which make the heavy math cheap, and new architectures like transformers that train stably at great depths. Researchers also discovered that scaling up parameters, data, and computing power improves performance along a predictable curve. This predictability is exactly what has justified the massive models and pre-training runs we see today.