Training Data
The labelled or unlabelled examples a model learns from — the single biggest determinant of what a model knows, what it believes, and which brands and entities it associates with which topics.
Training data is the foundation of any artificial intelligence model. It is the collection of examples a model learns from, and it ultimately determines what the AI knows, what it believes, and how it behaves.
For a large language model, this begins with a massive pre-training corpus. This dataset contains trillions of words scraped from the internet, including web pages, books, and academic papers. What makes it into this corpus shapes the model's primary bias. Brands, topics, and ideas that appear frequently are remembered reliably, while those that rarely appear are forgotten.
After this initial phase, models are refined using a smaller, task-specific dataset in a process called fine-tuning. Here, quality and balance matter far more than raw size. Sometimes, developers even use synthetic training data—which is generated by another AI—to fill in gaps or correct imbalances.
In the world of search engine optimization, training data is the root cause of every AI output. A model cannot recommend a brand it has never heard of, nor can it associate a company with a topic if they never appeared together in the training data. For businesses looking to be visible in AI-generated answers, the most powerful strategy is to influence this training data upstream. This means focusing on high-quality publications, citations, and clear associations with the right topics. In the end, curation and quality matter far more than sheer volume.
What training data is
Training data is the collection of examples a model is trained on. For a language model, this is typically billions of documents — web pages, books, code, academic papers — from which the model learns the statistical patterns of language. For a classifier, it is a labelled dataset: pairs of inputs and correct outputs the model learns to replicate. The quality, scale, and composition of training data determines what the model knows, what it believes, and how it behaves.
Types of training data
Pre-training corpus — the large, mostly unlabelled dataset used to build a base language model. For frontier models this runs to trillions of tokens scraped from the open web, filtered for quality. What makes it into this corpus shapes the model's primary bias: entities, brands, and topics well-represented in the corpus are recalled more reliably than those that appear rarely or not at all.
Fine-tuning dataset — a smaller, task-specific labelled dataset used during fine-tuning. DEJAN's grounding classifier was fine-tuned on 10,000 labelled prompt examples; the AI content detection model on 20 million labelled sentences. Quality and balance matter far more than raw size at this stage.
Synthetic training data — examples generated by an AI model rather than collected from human sources. Used to augment scarce real examples, address class imbalance, or target specific behaviours.
Why training data is the root cause of AI SEO outcomes
A model cannot recall a brand it was never trained on. It cannot associate a brand with a topic if those two things never co-occurred meaningfully in its training corpus. Every behaviour observable through model probing — which brands surface, which associations exist, which claims feel confident — traces back to the training data. This is why influencing training data, through publication, citation, and co-occurrence with the right entities, is the upstream lever in AI visibility strategy.
Data quality over quantity
More data is not always better. A model trained on low-quality, repetitive, or biased text inherits those properties. DEJAN's DEJAN-LM was pre-trained on a curated corpus of high-quality editorial content specifically to make it a better judge of content quality than a model trained on the raw web. Curation — what is included and excluded — is as important as scale.