The property that standard transformer attention grows with the square of the input length — doubling the context roughly quadruples the compute and memory — which is why long context windows are expensive and why grounding budgets exist.
In a standard AI transformer, every word or token compares itself to every other token. This means if you double the length of your input, you quadruple the computational work and memory required. This is known as quadratic attention cost, and it is the main reason why processing long text is so expensive.
This math shapes the AI products we use every day. It is why longer prompts cost more and take longer to process, and why systems impose strict limits on length. Even when a model theoretically has a massive context window, actually filling it is too costly. For example, AI search engines do not feed entire websites into a model. Instead, they operate on a strict, fixed word budget driven by these economics.
To get around this bottleneck, developers use several clever workarounds. Retrieval-augmented generation keeps contexts short by only pulling in the most relevant passages. Other techniques use mathematical shortcuts and approximations to avoid comparing every single token.
For anyone creating content, this quadratic cost is the driving force behind what AI actually sees. Because these models cannot afford to read everything, they only read a highly curated, budgeted slice. To survive the cut, content must be concise, self-contained, and easy for these algorithms to quickly rank and select.
In a standard transformer, attention lets every token compare itself against every other token. For a sequence of n tokens that is n × n comparisons, so the cost scales with n². Double the input and you roughly quadruple the work and the memory needed to hold the attention scores. This is the quadratic attention cost, and it is the central reason long contexts are expensive rather than free.
The quadratic term is what makes a large context window costly to actually fill. It explains why longer prompts are slower and priced higher, why providers impose per-request length limits, and why AI search systems ground answers against a fixed per-query word budget instead of dumping every candidate page into the model. DEJAN research has established that Google's AI Mode works within such a fixed budget — a constraint driven by attention economics and latency, not by the theoretical size of the underlying model's window.
Because the naive cost is punishing, modern systems avoid paying it in full. Retrieval-augmented generation keeps the context short by fetching only the most relevant passages. Ranking methods like BlockRank exploit the block-sparse structure of attention so in-context ranking scales to large candidate sets. Architectural variants use sparse, windowed, or linear-attention approximations to break the n² wall. All of these are, at heart, ways to spend the attention budget where it matters.
The quadratic cost is the economic force behind selection. Because a model cannot afford to read everything, it reads a budgeted slice — which turns visibility into a competition for that slice and makes the needle-in-a-haystack problem unavoidable. Content that is concise, self-contained, and easy to rank into the budget is content that survives the cut.
