50% click fraud on Yandex.Market or how to detect click fraud using standard tools
You begin to seriously think about the problem of click fraud only when the discrepancy between the data of your own statistics is more than 30% divergent from the data of the advertising system.
In our case, the last straw was the discrepancy between internal counters and Yandex.Market statistics by more than 50%. Are there really more than 50% of the bots that are debited from our balance sheet?
We decided to verify this by analyzing the log data of our server, Yandex.Metrica and Google Analytics.
Let's start in order. So
clickfrod(click fraud) - one of the types of network fraud, which is a fraudulent click on the advertising link by a person who is not interested in the advertisement. It can be carried out using automated scripts or programs that simulate a user's click on Pay per click advertisements. (Tnx 2 Wiki)
Many of us launch advertising campaigns, work with them for years and don’t even think about click fraud. But at some point strange things begin to happen:
In our case, this is exactly what happened!
We knew that there should be no problems with the availability of our server, everything worked like a clock. The Analytics-Metrica combination we used showed approximately the same data on each counter.
We were guarded by the fact that the difference between our statistics and the statistics of each of the advertising systems was no more than 10-15%, and periodically jumped over the Market for 30% (as Metrika reminded) and at one point reached 50% .
During the reporting period, Yandex.Metrica and Google Analytics showed approximately the same number of Market entries, but this number was much (over 40%) different from Market statistics.
A few words about how we tracked ads:
all sponsored links were marked with utm tags, and, we could track both total clicks by category and clicks on specific products;
we looked at the report on traffic sources in GA and the reports “Sources -> advertising systems” and “Sources -> Tags” in the Metric.
By the way, we noticed a discrepancy in the Market itself in the “Orders” report (+1 in favor of the established Metric):
Obviously, the contents of this plate with the reasons for the discrepancy described in it did not satisfy me, and I immediately began to sort it out. This problem did not occur the first time, and as soon as we noticed it, we immediately checked the settings and accessibility of the site.
In order to understand if these clicks really were, we decided to analyze the server logs .
Analysis of the logs showed that indeed the number of server requests with the necessary referrer was almost equal to clicks on Market statistics (less than 1% of the difference).
That is, there are clicks, but no users!
We began to analyze more deeply, and we were immediately struck by a strange fact. There were a lot of identical requests with the same data about systems, the same browsers and screen resolutions . At the same time, a certain logic of clicks was traced, the same product pages were clicked. Moreover, the user browser was suspicious.
I was also surprised by the fact that all IPs belonged to regional mobile operators :
The next step, we looked,Do our counters see transitions from these IPs ?
And they hit the target - counters recorded extremely few transitions from IP mobile operators . Moreover, having excluded live users with the help of WebVisor in the Metric, we just got the missing number of “lost” clicks from the Market.
WebVisor nevertheless recorded some visits to the “bots”:
Obviously, such visitors are not interested in purchases on our site;)
To summarize . We have:
CONCLUSIONS:
Is it really impossible to cut off such a simple way of wrapping up? Or is it not interesting to anyone?
Today, click fraud eats up a huge portion of advertisers' budgets. But it can be extremely difficult to prove this, or even less to get your money back. At the moment, Yandex.Market promised to understand our situation, but we decided to get ahead of it and figured it out ourselves. Let’s see if we can return something.
I hope our experience will help you;)
In our case, the last straw was the discrepancy between internal counters and Yandex.Market statistics by more than 50%. Are there really more than 50% of the bots that are debited from our balance sheet?
We decided to verify this by analyzing the log data of our server, Yandex.Metrica and Google Analytics.
Let's start in order. So
clickfrod(click fraud) - one of the types of network fraud, which is a fraudulent click on the advertising link by a person who is not interested in the advertisement. It can be carried out using automated scripts or programs that simulate a user's click on Pay per click advertisements. (Tnx 2 Wiki)
How to detect click fraud?
Many of us launch advertising campaigns, work with them for years and don’t even think about click fraud. But at some point strange things begin to happen:
- traffic increases, but the conversion does not change
- statistics of the advertising system begins to differ sharply from the readings of their own counters
- some strange patterns are detected in the click statistics of advertising systems
In our case, this is exactly what happened!
We knew that there should be no problems with the availability of our server, everything worked like a clock. The Analytics-Metrica combination we used showed approximately the same data on each counter.
We were guarded by the fact that the difference between our statistics and the statistics of each of the advertising systems was no more than 10-15%, and periodically jumped over the Market for 30% (as Metrika reminded) and at one point reached 50% .
During the reporting period, Yandex.Metrica and Google Analytics showed approximately the same number of Market entries, but this number was much (over 40%) different from Market statistics.
A few words about how we tracked ads:
all sponsored links were marked with utm tags, and, we could track both total clicks by category and clicks on specific products;
we looked at the report on traffic sources in GA and the reports “Sources -> advertising systems” and “Sources -> Tags” in the Metric.
By the way, we noticed a discrepancy in the Market itself in the “Orders” report (+1 in favor of the established Metric):
Obviously, the contents of this plate with the reasons for the discrepancy described in it did not satisfy me, and I immediately began to sort it out. This problem did not occur the first time, and as soon as we noticed it, we immediately checked the settings and accessibility of the site.
In order to understand if these clicks really were, we decided to analyze the server logs .
Analysis of the logs showed that indeed the number of server requests with the necessary referrer was almost equal to clicks on Market statistics (less than 1% of the difference).
That is, there are clicks, but no users!
We began to analyze more deeply, and we were immediately struck by a strange fact. There were a lot of identical requests with the same data about systems, the same browsers and screen resolutions . At the same time, a certain logic of clicks was traced, the same product pages were clicked. Moreover, the user browser was suspicious.
I was also surprised by the fact that all IPs belonged to regional mobile operators :
The next step, we looked,Do our counters see transitions from these IPs ?
And they hit the target - counters recorded extremely few transitions from IP mobile operators . Moreover, having excluded live users with the help of WebVisor in the Metric, we just got the missing number of “lost” clicks from the Market.
WebVisor nevertheless recorded some visits to the “bots”:
Obviously, such visitors are not interested in purchases on our site;)
So what are these clicks?
To summarize . We have:
- clicks from the IP of mobile operators that are suitable for the region (by the way, the mobile Internet is good because it constantly gives different IPs)
- they are all made on devices that give the server the same parameters (browser, permissions, systems)
- these are clicks that are not fixed by counters and do not execute scripts
- similar action scenarios can be traced (pages, delays in requests, number of requests)
CONCLUSIONS:
- There is a typical click fraud option!
- In fact, such a system is very easy to organize with several mobile devices.
There is a question for Yandex.Market?
Is it really impossible to cut off such a simple way of wrapping up? Or is it not interesting to anyone?
Outro
Today, click fraud eats up a huge portion of advertisers' budgets. But it can be extremely difficult to prove this, or even less to get your money back. At the moment, Yandex.Market promised to understand our situation, but we decided to get ahead of it and figured it out ourselves. Let’s see if we can return something.
I hope our experience will help you;)