Watch: Transfer Learning
Applying knowledge learned on one task or dataset to a different but related task — the principle that makes fine-tuning a pre-trained model far more efficient than training from scratch.
Transcript
Transfer learning is the practice of taking the knowledge a machine learning model gained on one task and applying it to a different, more specific task.
In natural language processing, this usually means starting with a large model that was pre-trained on a massive corpus of text. This pre-trained model has already done the heavy lifting of learning grammar, syntax, and broad world knowledge. From there, you fine-tune the model on a much smaller, labeled dataset for your specific job.
This approach works because language understanding is largely transferable. The patterns a model learns to predict missing words in general text are the very same patterns useful for classifying search intent or detecting AI-generated writing. Because the model doesn't start from scratch, you can train a production-quality classifier with only thousands of labeled examples, rather than billions. Fine-tuning is incredibly efficient.
While general language understanding transfers beautifully, highly specialized vocabulary sometimes does not. A model trained on general web text might struggle with niche technical jargon or legal documents. To solve this, models can be pre-trained on domain-specific content to create a better starting point.
You can also use advanced techniques like Low-Rank Adaptation, or LoRA. Instead of updating all of a model's weights during fine-tuning, LoRA freezes the pre-trained knowledge and trains only small, adapter layers. This preserves what the model already knows while efficiently adapting it to new tasks.