The field concerned with computers reading and producing human language. Its pipeline of tokenising, tagging and parsing has largely been absorbed into general-purpose language models.
Natural Language Processing (NLP) is the field concerned with computers reading, interpreting and producing human language. Its classic tasks are tokenisation, part-of-speech tagging, parsing, named entity recognition, sentiment analysis, translation and summarisation.
Until the late 2010s an NLP system was a pipeline: a tokeniser fed a tagger, which fed a parser, which fed a task-specific classifier, each component trained and evaluated on its own. Pre-trained transformers collapsed that pipeline. One large language model now performs most of those tasks from a prompt, and the intermediate representations are learned rather than annotated.
The parts that remain are the cheap, deterministic or measurable ones. Tokenisation is still a discrete step in front of every model. Named entity recognition still runs standalone wherever entity spans have to be exact. The field's evaluation habits carry over intact: precision, recall, F1 and perplexity are all NLP measures.
