Watch: The Verge: Quantitative Linguistic Analysis
This paper profiles the recovered corpus of The Verge from 2011 to 2023 using Shannon entropy and data compression to analyze changes in article structure.
Transcript
What happens when you analyze twelve years of digital journalism using nothing but data compression and math? A study of over one hundred and fifty thousand articles from the technology website The Verge, published between 2011 and 2023, reveals some striking structural shifts. The analysis ignored words and meaning entirely, treating each article simply as a stream of raw characters and bytes. The most dramatic change was size. The typical article more than doubled in length over the decade. This growth also created an illusion of complexity. While the raw files became harder for computers to compress, suggesting denser information, adjusting for length reveals that the actual prose structure remained remarkably consistent. At the character level, two subtle trends emerged. The overall variety of individual characters slightly narrowed, but the combinations of adjacent characters became more unpredictable. Essentially, a tighter set of building blocks was arranged in more complex ways. The most telling finding is that the biggest changes happened early on. By 2019, nearly every metric stabilized into a quiet, uniform pattern. The data shows a publication evolving from a volatile, early era of short posts into a mature, highly structured, and predictable format. While the math can map these physical changes with perfect precision, it cannot tell us why they happened. Whether it was the rise of explainers, new editing software, or changing industry standards, the explanation is left to the reader.
