Context Window
The maximum number of tokens a model can read at once — the total working memory available for a prompt, grounding documents, conversation history, and the generated response combined.
An artificial intelligence model's context window is its memory limit for a single conversation. Everything the model needs to understand—including your prompt, past messages, and any documents you upload—must fit inside this window. If information falls outside of it, the model simply cannot see it.
The size of this window is crucial for grounding, which is how a model uses real-world data to answer questions. While older models were limited to just a few thousand tokens, newer ones have massive capacities. For example, OpenAI's GPT-4o can handle one hundred and twenty-eight thousand tokens, while Google's Gemini 1.5 Pro can process a staggering one million tokens. This is enough to hold roughly three-quarters of a million words, or hundreds of web pages.
However, a massive context window doesn't mean a model reads everything perfectly. Research shows that language models suffer from a "lost-in-the-middle" problem. They pay close attention to the very beginning and the very end of a long document, but often overlook details buried in the middle.
Furthermore, real-world constraints like latency and cost mean search engines often use a fixed word budget for grounding, regardless of how large the model's theoretical window is. For anyone looking to optimize content for AI search, this means the position of your information matters just as much as its presence. To be remembered, your key points and brand mentions need to be right at the top or at the very end.
What the context window is
The context window is the maximum number of tokens a language model can attend to in a single forward pass. Everything the model "knows" about a conversation — the system prompt, previous messages, any documents provided for grounding, and the response it is generating — must fit within this limit. Tokens outside the window are simply not visible to the model; they might as well not exist.
Why size matters
A larger context window means more information can be fed to the model at once. This directly affects grounding: the more pages a model can read in a single request, the richer its answer can be. Gemini 1.5 Pro has a 1 million token context window, which can hold approximately 750,000 words — roughly 700 average web pages. GPT-4o supports 128,000 tokens. Earlier models were limited to 4,096 tokens, making deep document grounding impossible.
Context window and grounding
When AI search systems ground a query in web results, the retrieved grounding snippets are placed into the model's context window alongside the user's question. The model only sees what fits. DEJAN research has established that Google's AI Mode operates with a fixed per-query word budget for grounding content — a practical ceiling imposed by context window economics and latency constraints, not by the theoretical window size of the underlying model.
The lost-in-the-middle problem
Having a large context window doesn't mean the model uses all of it equally well. Research has shown that LLMs tend to attend more strongly to the beginning and end of long contexts, with information in the middle receiving less weight. For AI SEO, this means position within a grounding document — not just presence — may affect how reliably a model incorporates a brand mention.