How to win a click into Ya. Direct and AdWords for 600 thousand rubles a month

Over the past six months, we have managed to win the “clickthrough” of our contextual advertising with a budget of 1 million rubles a month.

The key to defeating the fraud was the per-minute monitoring of traffic with notifications of abnormal changes and disabling problematic API announcements, and a series of reports that reflect the situation in real time.


Figure 1. Visitor number chart by keywords for

How to find out that you are being attacked?

One of the first signs of “skipping” advertising will be an increase in the percentage of funds returned for fraud in Direct and AdWords.

“In Yandex Direct, the fraud expenses are automatically returned to the balance of the advertising campaign. The number of clicks eliminated by the fraud protection system is displayed in the “daily statistics” reports, “general statistics” in the “invalid clicks for the entire selected period.”

- Help Ya. Direct "invalid clicks . "

In AdWords display "invalid clicks" level can be included in the "Columns" tab:


Figure 2. Set up columns to the level of "invalid clicks" in AdWords

In our case, with the average level of " invalid clicks " in Direkte ≈ 10%, Yandex suddenly began to return 40% of the advertising budget, and after a month, altogether 54%.

The next sign of fraud is unreasonable growth in the number of transitions and strong changes in behavioral indicators for a number of ad groups.

We noticed that in a number of keywords for which there were never more than 200 visitors per day, suddenly there were bursts of up to 3,000 visitors. In fact, the budget, in the days of such activities, could go on one advertising campaign, if it was not stopped in time.


Figure 3. Unjustified traffic growth for a specific ad group in Direct

Yandex and Google do not protect against fraud

It is enough to read the discussion of the fraud level in contextual advertising in the official Direct Club to make sure that many advertisers lose money due to fraud.

Google officially recognizes the mistakes of its security system and provides advertisers with “Refund Claims” (refund of the spent budget). According to statistics from the service ClickSease, Google Ads on average returns 12% of the advertising budget.


Yandex, on the other hand, does not recognize the vulnerability of its own protection filters and complaints of obvious fraud cases. It sends a sample response that the problem lies with the advertiser and his website.

In our case, the percentage of “invalid clicks” in the report by Ya. Direct has never risen above the cherished 50% in any of the advertising campaigns, even on the days of the most violent bursts, when 80% of the budget “merged” into the usually not very popular ad group without calls and applications.


Figure 4. The “Invalid Clicks” Level in One of the Campaigns in Direct

What level of attack are we facing

Fraudulent traffic went to four business lines in two cities. When new advertising campaigns were launched or when they were launched for a long time, they were redistributed to them within a few hours.

Click Fraud was not time bound, and the redistribution of the budget, for example, for the night, did not give any effect: we still received our own volume of traffic. "Clicking" occurred equally actively both in YAN, and on Yandex search, and on Google’s GMS.

On the site, bots simulated user behavior, navigated through sections of the site, selected text, naturally scrolled, and moved the cursor.

Are there any ready-made remedies?

All click-protection services have only one direct fight tool - blocking suspicious IP addresses or placement sites in Ya.Direct and AdWords campaigns.

If dynamic IPs are used against you, any anti-click fraud service will always be one step behind the fraudsters in blocking them by IP: the bot has already made a few clicks on your ad by the time the service blacklists the IP and stop advertising on it. In addition, after the fraudulent software will not be able to see your advertisement, it will simply change the IP address, Hardware-ID and continue its actions.

When attacking advertising in the CCM or YAN, various placements are usually used, and automatic protection systems cannot detect suspicious sites for blocking.

Let's go back to blocking by IP - here we come to the most interesting part - if AdWords allows blocking up to 500 IP addresses, then Yandex Direct can block only 25 unique IP addresses for one advertising campaign! Such a small black list of IP addresses is no longer relevant, since now you can safely purchase 500 IPv4 addresses for 10 thousand rubles and bypass this restriction.

You can protect yourself from “clipping” performed at a high level in just two ways:

1. learn not to show advertisements to fraudulent users or bots, for which you need to find certain "patterns" in their behavior and characteristics;
   
2. temporarily stop specific ad groups and keywords that are being attacked.
   

Cut off part of the audience to keep most of it.

If you learn not to show ads click on bots or fraudulent users, then they will not be able to harm.

You can always track similar behaviors and patterns, for example, that fraud usually runs on Windows 7 from 5:00 to 9:00 in Moscow, and set a rate adjustment of -100% for a similar audience in all advertising campaigns attacked. The functionality of bid adjustments in AdWords is quite extensive, which cannot be said about Yandex Direct adjustments .

We are looking for patterns in frode through protection services

In order to have an idea of ​​how exactly we are attacked, and to manually trace patterns in the fraudulent traffic, we have connected the Russian service from fraud ClickFrog . The product is a well-known, popular among CPA and so on.

ClickFrog quickly proved complete incapacity:

1. allocated no more than 40 suspicious IP addresses per day, with traffic from Yandex.Direct of 3,000 thousand hops per day, and, even recognized by Yandex, 1,300 left-click clicks per day;
   
2. The main tool for protecting the service is blocking by IP address, the command about which is sent to Ya.Direct by API. However, as soon as the black list of 25 IP addresses is filled, you must manually delete the last few IP addresses in each advertising campaign and wait for the next list to be filled, and so in a circle.
   

Then we installed the code of the American service ClickSease , aimed at AdWords, and not yet working with Direct. The service, by the way, has, in contrast to ClickFrog, a free test period of 2 weeks.

ClickSease was more useful: he began to catch 300-400 unique fraudulent IP per day. For each blocked IP service gives statistics:

• internet provider;
• platform from which the transition was made;
• operating system;
• unique device ID;
• the time of the first and last transition;
• region.

From the ClickSease report, we were able to identify patterns in frade:

• in 81% of cases, the device simulates a mobile OS: Android or iOS;
• in 59% of cases, geolocation of an IP address does not apply to Moscow, with a ford aimed at Moscow.

We are looking for patterns in Frode manually

However, even such obvious patterns were not enough to reduce the harm from fraud, and I didn’t want to disable advertising on mobile. Services are usually able to only give ideas on identifying similar patterns in the frode, and then it is necessary to detect fraud in the Metric (in the case of an attack on Direct) and select it in separate Yandex Audience segment for subsequent analysis and blocking.


Figure 5. An example of analyzing traffic by age group in the Metric to search for fraud patterns.

Traffic cuts that will help determine fraud patterns:

• audience dynamics by age groups;
• the dynamics of the long-term interests of users;
• device dynamics and OS.

In the case of AdWords, the countermeasure is clear:

1. determine the audience segment "infected" fraud;
2. set rate adjustment -100% for the selected segment;
3. we track changes in indicators: conversion, time on the site, depth of viewing, bounce rate.

In Yandex Direct, combat mechanics are more difficult and are divided into two options:

a) you managed to find an obvious fraud pattern relating to gender, age or mobility:

1. set the rate adjustment -50% or -100% for the selected segment;
2. we trace change of key indicators.

b) no obvious patterns were found:

1. We allocate frod traffic to a separate segment of Yandex. Audience (for example, you knew for sure that from October 1 to October 20 there could not be 5,000 clicks on an ad group, which always had no more than 30 visits per day)
2. we create a segment of users similar to our fraud through the Yandex look-alike ;
3. set up a rate adjustment of -100% for a manually created audience segment;
4. We carefully test the reduction in advertising rates for the segments created by Yandex.

Build charts that show fraud

Frode always gives rise to obvious pockets and spikes , be it abstruse software with imitation of the behavior of a real user or a group of freelancers performing a technical task.


Figure 6. The number of visitors to the chart for keywords by fraud Frock

is uneven for several reasons:

• in order to make the attack “smoothed out” you need to possess confidential information and know who, when and how many transitions you make in your advertisement;
• the software acts in spurts, and on the minute, 10 minute, and sometimes on the hourly chart, its actions will be evident;
• even if “schoolchildren” are working against you from the bulletin boards, then they also act on a specific task with an algorithm, and the anomalies generated by them will be easily traced.

If you learn to quickly find and eliminate foci, you can significantly reduce the harm from fraud. In our case, the obvious sign was the anomalous increase in the number of clicks on contextual advertising in specific 10 minutes or one minute for some keywords. Google Data Studio is

best suited for visualization , as it is only Analytics that is able to correctly collect data divided into 1 and 10 minutes, and Metric gives incorrect results when building reports on the panels.

Set up notifications for fraud sites

In order not to keep track of all fraud cases manually, I made a report in Google Spreadsheets, which updates the data every minute and notifies you that fraud starts.

Google Spreadsheets support the Core Reporting API , which can be accessed through the Script Editor in Spreadsheets.

Step 1. Go to the script editor to access Analytics


Figure 7. Script editor to access the Analytics Core Reporting API via Google Tables

Step 2. Write the API request to Analytics to get data on the required indicators (for example, on the number of users who have paid advertising, in every minute of the day, as in our case).



Step 3. We set a trigger for updating data every minute:


Figure 8. We request fresh data every minute for a quick response to fraud.

Step 4. Create a pivot table from a sheet updated with the necessary data once a minute and analyze these indicators to set up triggers for notifications E-mail or disable ad groups by API Ya. Direct or AdWords.

Figure 9. An example of setting up formulas for anomaly notifications

Results: how to win the click

Counter-click fraud can be divided into three groups:

a) Proactive actions:

• shutdown of "polluted sites";
• Disabling ad serving for an audience with signs that are fraudy for you, for example, for people on tablets from St. Petersburg (more advanced parameters for blocking can be used through AdWords lists and Metrics segments);
• adjusting bids for audience segments that are similar to fraud segments (“look-alike” segments are created in I. Audiences and Google Lists);
• fraud blocking by IP network masks (available only in AdWords).

b) Preventive actions:

• Send complaints about the return of the budget in AdWords and Direct;
• an investigation of "who ordered the attack on you";
• grouping suspicious and frequently attacked ad groups into a single advertising campaign;
• "Traps" for the simplest bots, namely the hidden buttons on the site, which are visible only to the bot and when clicked, it falls into the list.

c) Post factum actions:

• blocking by IP addresses;
• quick disconnection of click points: keywords, ad groups, advertising campaigns, audience segments.

Figure 10. Protection methods for click fraud

Useful links:

• Core Reporting API library ;
• Query Explorer for easy testing API commands;
• We write scripts to automate the work with Google applications ;
• Automate Google Sheets ;
• Analytics Intelligence , to ask the Analytics bot: - “But have there been any abnormalities in paid traffic over the last year?” And get a clear answer.

Those who are also faced with the click-through of contextual advertising - write in the comments, we will try to help each other!