Who fetches it
Page hits counts requests to real pages from the same IP addresses that fetched this file. A zero there means the agent took bundle files but never touched the HTML site. IP addresses are shared within a company’s crawl fleet, so treat the figure as the fleet’s activity rather than that one named agent’s. Cadence is the mean gap between consecutive fetches.
| Agent | Fetches | Share | IPs | Days | Page hits | Cadence | Last seen |
|---|---|---|---|---|---|---|---|
| Browser | 2205 | 77.7% | 333 | 19 | 594885 | 0.2h | 14 Aug 23:35 |
| Unrecognised | 541 | 19.1% | 39 | 20 | 578487 | 0.8h | 15 Aug 03:59 |
| curl | 32 | 1.1% | 10 | 8 | 11016 | 13.5h | 13 Aug 17:17 |
| Googlebot | 10 | 0.4% | 9 | 7 | 50361 | 44.5h | 14 Aug 10:00 |
| AhrefsBot | 6 | 0.2% | 6 | 6 | 1698 | 73.4h | 13 Aug 12:50 |
| ChatGPT-User | 6 | 0.2% | 6 | 6 | 371 | 81.4h | 14 Aug 21:28 |
| Slackbot | 5 | 0.2% | 5 | 3 | 44 | 47.7h | 12 Aug 12:51 |
| Amazonbot | 4 | 0.1% | 4 | 2 | 967 | 30.5h | 05 Aug 20:06 |
| Baiduspider | 4 | 0.1% | 4 | 2 | 601 | 31.6h | 04 Aug 17:03 |
| YandexBot | 3 | 0.1% | 3 | 3 | 293 | 157.0h | 11 Aug 13:00 |
| GPTBot | 3 | 0.1% | 3 | 3 | 363 | 180.3h | 15 Aug 01:46 |
| (no user agent) | 3 | 0.1% | 2 | 2 | 66 | 13.4h | 14 Aug 06:23 |
| PerplexityBot | 3 | 0.1% | 3 | 3 | 809 | 168.5h | 14 Aug 04:16 |
| facebookexternalhit | 2 | 0.1% | 1 | 1 | 17 | 0.0h | 04 Aug 07:59 |
| MJ12bot | 2 | 0.1% | 2 | 2 | 132 | 16.9h | 10 Aug 02:03 |
| OAI-SearchBot | 2 | 0.1% | 2 | 2 | 88 | 22.9h | 05 Aug 13:46 |
| SemrushBot | 2 | 0.1% | 2 | 2 | 4078 | 166.9h | 08 Aug 19:31 |
| Sogou | 1 | 0.0% | 1 | 1 | 65 | — | 06 Aug 19:38 |
| Google-CloudVertexBot | 1 | 0.0% | 1 | 1 | 292 | — | 05 Aug 14:50 |
| aiohttp | 1 | 0.0% | 1 | 1 | 97 | — | 07 Aug 00:05 |
| Claude-User | 1 | 0.0% | 1 | 1 | 724 | — | 13 Aug 12:52 |
| CCBot | 1 | 0.0% | 1 | 1 | 188 | — | 09 Aug 07:11 |
Unrecognised user agents
Raw strings that matched no known crawler pattern. A new AI crawler shows up here first; add its name to the pattern list to give it a proper row above.
| User agent | Fetches | IPs | Last seen |
|---|---|---|---|
| Mediatoolkitbot (complaints@mediatoolkit.com) | 451 | 1 | 15 Aug 03:59 |
| 1'"2000 | 9 | 3 | 13 Aug 16:59 |
| Mozilla/5.0 | 7 | 2 | 04 Aug 12:21 |
| Mozilla/5.0 (compatible; DeepSeekBot/1.0; +https://www.deepseek.com/bot) | 4 | 4 | 12 Aug 07:36 |
| Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0 | 4 | 1 | 13 Aug 09:07 |
| -1' OR 5*5=25 -- | 3 | 2 | 11 Aug 10:48 |
| ClueWeb-Crawler/1.0 (+https://boston.lti.cs.cmu.edu/CMU-ClueWeb-Crawler/; mailto:cmu-clueweb-crawler@andrew.cmu.edu) | 3 | 3 | 05 Aug 10:48 |
| -1 OR 5*5=25 | 3 | 2 | 11 Aug 10:48 |
| -1 OR 5*5=25 -- | 3 | 2 | 11 Aug 10:48 |
| NavCrawl/0.4 (+https://example.com/bot) | 3 | 1 | 14 Aug 17:47 |
| (none sent) | 3 | 2 | 14 Aug 06:23 |
| (select 198766*667891 from DUAL) | 3 | 2 | 11 Aug 10:48 |
| (select 198766*667891) | 3 | 2 | 11 Aug 10:48 |
| Mozilla/5.0 (compatible; IbouBot/1.0; +bot@ibou.io; +https://ibou.io/iboubot.html) | 3 | 3 | 02 Aug 05:32 |
| (select(0)from(select(sleep(15)))v)/*'+(select(0)from(select(sleep(15)))v)+'"+(select(0)from(select(sleep(15)))v)+"*/ | 3 | 2 | 11 Aug 10:48 |
| if(now()=sysdate(),sleep(15),0) | 3 | 2 | 11 Aug 10:48 |
| -1" OR 5*5=25 -- | 3 | 2 | 11 Aug 10:48 |
| Mozilla/5.0 (compatible; metadata inventory builder) | 2 | 1 | 03 Aug 03:28 |
| mwmbl/0.2.0 (https://github.com/mwmbl/mwmbl/ contact daoud@mwmbl.org (Mwmbl Crawler Operator)) | 2 | 1 | 14 Aug 08:31 |
| LinkupBot/1.0 (LinkupBot for web indexing; https://linkup.so/bot; bot@linkup.so) | 2 | 1 | 12 Aug 13:30 |
| Mozilla/5.0 (compatible; xAI-SearchBot/1.0; +https://x.ai) | 2 | 2 | 06 Aug 14:31 |
| Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 [LinkedInApp]/9.32.2485 | 1 | 1 | 04 Aug 20:20 |
| SEBot-WA | 1 | 1 | 01 Aug 20:16 |
| SearchEngineBot/0.1 (+https://example.com/bot) | 1 | 1 | 09 Aug 22:43 |
| search.marginalia.nu | 1 | 1 | 31 Jul 13:45 |
| secretscan/1.0 (+oneshot) | 1 | 1 | 13 Aug 06:01 |
| -1" OR 5*5=25 or "NslJZhfp"=" | 1 | 1 | 31 Jul 08:22 |
| -1" OR 5*5=25 or "RkCyApET"=" | 1 | 1 | 11 Aug 10:48 |
| -1" OR 5*5=25 or "u87oD9wz"=" | 1 | 1 | 31 Jul 08:19 |
| -1' OR 5*5=25 or 'AtiBEEtD'=' | 1 | 1 | 31 Jul 08:19 |
| -1' OR 5*5=25 or 'jDKozIGa'=' | 1 | 1 | 11 Aug 10:48 |
| -1' OR 5*5=25 or 'svwKzAkO'=' | 1 | 1 | 31 Jul 08:22 |
| @@JEEwp | 1 | 1 | 11 Aug 10:48 |
| @@L9O2K | 1 | 1 | 31 Jul 08:20 |
| @@tLEru | 1 | 1 | 31 Jul 08:23 |
| Arquivo-web-crawler (compatible; heritrix/3.4.0-20200304 +https://arquivo.pt/faq-crawling) | 1 | 1 | 04 Aug 03:32 |
| IndusFirehose/1.0 (+https://indus.local/bot) | 1 | 1 | 11 Aug 17:24 |
| LeakScanner/1.0 (self-scan) | 1 | 1 | 10 Aug 19:35 |
| Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/11.0.3 Safari | 1 | 1 | 27 Jul 20:19 |
| Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 | 1 | 1 | 14 Aug 07:10 |
| Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko | 1 | 1 | 05 Aug 07:24 |
| Mozilla/5.0 (compatible; DEJAN capability inventory) | 1 | 1 | 03 Aug 04:41 |
| Mozilla/5.0 (compatible; SeznamBot/4.0; +https://o-seznam.cz/napoveda/vyhledavani/en/seznambot-crawler/) | 1 | 1 | 28 Jul 10:29 |
| Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/d3712ae warc/v0.8.99 | 1 | 1 | 14 Aug 15:00 |
| Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 [LinkedInApp]/9.32.2372 | 1 | 1 | 31 Jul 19:12 |
| Mozilla/5.0 (iPhone; CPU iPhone OS 18_7 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 [LinkedInApp]/9.32.2469 | 1 | 1 | 03 Aug 13:25 |