Why we don't do prompt tracking.

TL;DR: We don't do prompt tracking because prompts are infinite variants of canonical core intent entities and we monitor those instead.

Read on though. It's worth going beyond surface claims.

Let's start with an example:

Core Entity:

1. running shoes

Queries:

1. buy running shoes
2. running shoes online store
3. where to buy running shoes

Prompts:

1. I want to buy a new pair of running shoes. Where should I buy them?
2. Can you recommend a good online store for running shoes?
3. Where is the best place to buy running shoes?

So I see people do this:

I'm a 27 year old female from Sydney looking to buy some running shoes...

My expert opinion and comment on this approach is: LOL!!!!

And clickstream?

Unethical, creepy and bundled with a looooooot of mathematical fudge, formulas and extrapolations between sparse data points. At this point you might as well just give up and make up synthetic prompts.

Even in cases where people willingly volunteer their chat history you're getting *that* demographic only. I bet you're not *it* yourself and don't know many people who are willing to surrender their most personal conversations for a small benefit or a tiny fee.

What we track instead

app.dejan.ai tracks the core entity. Each tracked entity, such as "running shoes", goes to Google, OpenAI and Anthropic models with web search on, using one fixed prompt:

Recommend brands for a user searching for the supplied query. When mentioning the brand in your response wrap each brand name like this:

[[brand]] Text goes here.

Example: [[Microsoft]] A short blurb relevant to user query goes here. [[Google]] A short blurb relevant to user query goes here.

Don't enumerate.

The markers let us record which brands the model names, in what order, and which pages it cites. We run the same probe for each target location on a daily, weekly or monthly cadence, and the results become visibility scores per entity, per location and per brand over time.

The wording never changes between runs, so when a brand's share moves, the cause is a change in the model or in the web results it retrieved.

We also ask the same question with web search off. That answer shows what the model knows about the market from its training data, separate from what it finds when it searches.

Answers vary from run to run even when the input is identical, so one answer is one sample. Our association probes repeat each entity up to 100 times, and our relevance probes take up to 100 independent samples per brand and entity pair. We report the share of answers in which a brand appears, because a single answer can change on the next run.

STOP! Read this, don't skip:

This is not what happens in the real world, the above are not the actual volumes of searches, it's also not how people search or what chat sessions look like or how models respond. There is no pretence here. This is a measurement framework that tests the impact of our SEO experiments on model behaviour in terms of

selection rate

Selection Rate is the rate at which an AI system chooses a given source from the grounding candidates available to it when it composes an answer. It is the AI-search equivalent of click-through rate: where CTR captured a person picking a link from a list, Selection Rate captures a model picking a source to ground its reply on.

Raising it is the aim of Selection Rate Optimization. Related: AI Visibility.

and

grounding

Web search grounding is the process where an AI model answers a fact-seeking question not from memory but by running its own web search, reading what comes back, and weaving some of those pages into its reply. It's the umbrella term for how Google, OpenAI and Anthropic pull live sources into generated answers.

Under the hood every platform runs the same funnel: a search returns pages received, a subset have readable content, and a smaller subset get cited in the answer. The gap between received and cited is where each platform's personality shows. In our head-to-head test on a single query, Google received 7 pages and cited all 7, OpenAI received 39 and cited just 2, and Anthropic received 14 and cited 9.

For AI visibility this is the whole game: your page has to survive the funnel — get retrieved, be readable, and earn the citation. It works through grounding chunks and the grounding snippet, and it is distinct from document grounding, where you hand the model a specific URL.

Related concepts

• Grounding
• Grounding Chunk
• Document Grounding
• Number of Citations

presence. Neat, structured ordinal values are replacing LTR (left to right) order of recommended brands you see in naturally flowing text by the model.

Are synthetic prompts useful?

Of course they are. Prompts are super-useful for qualitative research, citation source discovery, content ideas and fanout-query capture. We use them extensively as part of our Citation Mining workflow.

Here's an example: https://app.dejan.ai/property/43?tab=citations

What I'm saying is that freestyle synthetic prompts are the wrong tool to use for visibility tracking. You would need to work very hard to change my view on this.

Confused?

Here's a clear-headed way to think about this. There's an infinity of prompt variations with the same canonical intent. So instead of all that busywork we opt to quantize them into their most nuclear canonical form and measure that.

Here's a list of 10 synthetic prompts used for citation mining:

https://app.dejan.ai/property/43?tab=citations&run_id=12496&src=&view=&ent_id=&query_q=&loc=&tag_id=

And their fanouts:

https://app.dejan.ai/property/43?tab=citations&run_id=12496&sub=fanouts&view=google

For Google, each one of the fanout queries returns a set of results, best of which are selected for inclusion as grounding candidates.

Each grounding candidate is then represented to the model as a grounding snippet generating using extractive summarization where only the most relevant parts of the page, relative to the fanout query, are extracted from the page and stitched together with ellipses "..." to form one grounding snippet.

Here's what that looks like for fanout query "SERP flux measurement tool" from one of the input prompts for Algoroo project:

The above is from our grounding inspector tool:
https://app.dejan.ai/property/43?tab=grounding_inspector

And this tool will both generate the fanouts and do the fanout heatmap on the page highlighting the most important parts of the page with respect to the fanouts for a given prompt: https://app.dejan.ai/property/43?tab=optimizer&sub=page_grounding