Listen: Natural Language Processing (NLP)
The field concerned with computers reading and producing human language. Its pipeline of tokenising, tagging and parsing has largely been absorbed into general-purpose language models.
Transcript
Natural language processing, or N-L-P, is how we get computers to read, interpret, and generate human language.
For a long time, N-L-P worked like an assembly line. To understand a sentence, a system had to pass the text through a series of separate steps. First came tokenization to break down the words, then tagging, then parsing, and finally a classifier to get the result. Every single piece of this pipeline had to be trained and evaluated on its own.
But in the late twenty-tens, pre-trained transformers changed everything. They collapsed the traditional pipeline. Today, a single large language model can handle almost all of these tasks from a simple prompt, learning how to represent language internally instead of relying on step-by-step human annotations.
So, what survived this shift? Mostly the processes that are cheap, predictable, or easy to measure. Tokenization is still a necessary first step before any model runs. Named entity recognition is still used on its own when we need exact, precise categories. And classic evaluation metrics, like precision, recall, and perplexity, remain the standard for measuring success. The pipeline may have changed, but the foundation of how we prepare and measure language is still very much alive.