Watch: Cascading Hallucination
A novel failure mode in multi-agent AI systems: a capable model trusts its sub-agents to deliver grounded facts, but they hallucinate, and the orchestrator launders the fiction into a confident answer without ever questioning it.
Transcript
There is a new, dangerous way for artificial intelligence to be confidently wrong, and it is not the typical hallucination we have learned to watch for. It happens in multi-agent systems, where a main orchestrating model delegates tasks to sub-agents.
These sub-agents use search engines and databases, so their answers are supposed to be anchored in real sources. But when a sub-agent reads a summary, it often infers a connection that was never in the source, and passes that inference up the chain as a fact. The orchestrating model has no reason to doubt its helper, so it accepts the claim and builds on it. The fiction is laundered through the system, gaining authority at each step while losing its connection to reality. This is a cascading hallucination.
Grounding models in real documents does not prevent this. First, grounding only constrains what a model reads, not what it concludes. A model can easily bridge unrelated facts with a fabricated connection. Second, most grounding relies on quick search summaries rather than primary documents. By the time the final orchestrator receives the information, the uncertainty is completely stripped away. The final output sounds polished and authoritative, but it is built on a game of telephone.
To stop this, we must treat every agent summary as a secondary source, not ground truth. Systems must preserve uncertainty across steps, separating what was actually read from what was merely concluded. When the stakes are high, we have to insist on checking the primary source. As we build more complex AI networks, we have to ask: did anyone, anywhere in the chain, actually verify the thing we are repeating?