← back
How I recorded user behaviour on my competitor's websites

How I recorded user behaviour on my competitor's websites

A test that hijacked the Chrome back button to send visitors to a fake Google results page, then to mirrored competitor pages where heatmaps, sessions and typing were recorded. Google de-indexed the whole domain.

Update

Google's team tracked down my test site, most likely using the source code I shared, and de-indexed the whole domain.

Last time I publicly exposed a flaw, Google issued a manual penalty and devalued a single offending page. This time there is no notice in Search Console. The site is removed from the index entirely, without any notification.

I received a lot of criticism for the way I handled this. Many people suggest the right way is to approach Google directly with security flaws instead of writing about them publicly. Others suggest I acted unethically, or even illegally, by running the test. It should be obvious that if I intended to exploit this method I would not write about it. With so much risk and so little gain, is it even worth doing in practice? Of course not. I would be more concerned about the people who do unethical things and do not write about them.

My Wish List

  1. Manipulating the back button in Chrome should not be possible in 2018.
  2. Websites that employ this tactic should be detected and penalised by Google's algorithms.
  3. If such pages are still found in Google's results, they should be labelled with a "this page may be harmful" notice.

Here Is What I Did

  1. A user lands on my page, with Google as the referrer.
  2. When they hit the "back" button in Chrome, JavaScript sends them to my copy of the SERP.
  3. A click on any competitor takes them to my mirror of that competitor's site, set to noindex.
  4. I then generate heatmaps and scrollmaps, and record screen interactions and typing.
Diagram: a Google search results page on google.com with an arrow pointing to a mock landing page on mysite.tld
jQuery snippet that calls window.history.pushState on load, then listens for popstate and, when document.referrer matches google, redirects the browser to a spoofed Google address
Diagram: hitting Back in Chrome on mysite.tld sends the user to google.mysite.tld, a fake results page listing competitors
Diagram: clicking a competitor on the fake results page at google.mysite.tld opens a copy of the competitor page at competitor.mysite.tld

Only about 50% of users found anything suspicious, partly because I used HTTPS on all my pages, which is one of the main trust factors on the web. Many users are simply happy to see the padlock in their browser.

At this point I was able to:

  • Generate heatmaps for clicks, moves and scroll depth.
  • Record actual sessions, including mouse movement, clicks and typing.

I gasped when I realised I could capture every form submission and send it to my own email. I never actually tried that.

Would a Website Doing This Be Penalised?

You would think so.

I had this running for a very brief period, and for ethical reasons I took it down almost immediately once I realised it might cause trouble. After that I changed the topic of the page completely and moved the test to one of my disposable domains, where it stayed for five years and ranked well, though for completely different search terms with low volumes. Its new purpose was to mess with conspiracy theory people.

Alternative Technique

You do not have to spoof Google SERPs to generate a competitor's heatmaps. You can simply A/B test your landing page against your clone of theirs through paid traffic, social media for example. Is the A/B testing version ethically fine? I do not know, but it may get you in legal trouble depending on where you live.

What Did I Learn?

Users seldom read home page fluff. They look for testimonials, case studies, pricing levels, staff profiles and company information, searching for credibility and trust. One of my upcoming tests will combine the home page with "about us", "testimonials", "case studies" and "packages", so users get everything they want on a single page.

Reader Suggestions

"I would have thrown in an exit pop-up to let users know what they had just been subjected to." Marc Nashaat

From Hacker News

Howdy, former Matasano pentester here. FWIW, I would probably have done something similar to them before I worked in the security industry. It is an easy mistake to make, because it is one you make by default: intellectual curiosity does not absolve you from legal judgement, and people on the internet tend to flip out if you do something illegal and say anything but "You're right, I was mistaken. I've learned my lesson."

To the author: the reason you pattern-matched into the blackhat category instead of whitehat or grayhat is that in the security industry, whenever we discover a vuln, we PoC it and then write it up in the report and tell them immediately. The report typically includes background info, reproduction steps and recommended actions. The whole thing is typically clinical and detached.

Most notably, the PoC is usually as simple as possible. alert(1) suffices to demonstrate XSS, for example, rather than implementing a fully working cookie swipe. The latter is more fun, but the former is more impactful.

One interesting idea would have been to create a fake competitor, for example "VirtualBagel: Just download your bagels and enjoy." Once it is ranking on Google, run this same experiment and see if you could rank higher. That experiment would demonstrate two things: (1) the history vulnerability exists, and (2) it is possible for someone to clone a competitor and outrank them with this vulnerability, thereby raising it from sev:low to sev:hi.

So to be clear, the crux of the issue was running the exploit on a live site without their blessing.

But again, don't worry too much. I would have made similar errors without formal training. It is easy for everyone to say "Oh well, it's obvious," but when you feel like you have good intent, it is not obvious at all. I remind everyone that RTM once ran afoul of the law due to similar intellectual curiosity. In fairness, his experiment exploded half the internet, but still.

Source: Hacker News

Related concepts

Dan Petrovic · Aug 21, 10:56