Experience in organizing rotation of Destination URLs for PPC objects

Introduction

As you know, objects of PPC search engines (usually keywords and ads) have
such a parameter as Destination URL (destination URL), which contains a link
to a page advertised through paid advertising.
It was necessary to organize the rotation of the Destination URL so that they successively
changed one after another for a large number of PPC objects (millions of keywords
) on various search engines (Google, Yahoo, MSN, Baidu, ASK, etc.).

At first glance, this problem could be solved by creating several Ad'ov
with the necessary target links, but in fact, this approach had the following
problems:

• It was not possible to control the run time of the URL - i.e. you couldn’t make it possible
  for any URL to work only on weekends, or only two hours a day in the morning
  , etc.
• It was not possible to control the display frequency of a destination URL.
• It was difficult to compile statistics on the use of destination URLs from several
  heterogeneous search engines.

It was possible to simply update destination URLs on a specific schedule using the
search engine APIs , but this also could not solve the problem for several reasons:

• Many search engines (for example, Baidu or Yahoo) after changing URLs put the
  changes in the Pending state and spend considerable time (up to several
  days) to check them.
• On Google, this would result in additional costs for the Google Developer Token.
• Baidu does not allow URLs to be used if their domains are not in the list of pre-
  authorized domains. Therefore, changing URL urls is difficult.
• Google resets Ad statistics if they change the URL, because internally
  for Google it means creating a new Ad.
• Even in the absence of these problems, the problem of quickly updating a
  large number of objects, i.e. if we have millions of keywords, we won’t
  be able to instantly submit changes to any search engine.
• In addition, communication over the Internet is not always reliable - there are disconnections
  between providers, the APIs themselves periodically fail with search engines, etc.

Therefore, to organize such an interaction, it was decided to organize their own
server through which visitors will be redirected to the destination URLs.

Decision.

Here is the way of the search engine visitor to your landing page:

Ie the visitor is taken to an intermediate server and then redirected to the destination
URL. For the visitor, this happens almost imperceptibly.
Destination URLs were combined into groups within which rotation was carried out.
In turn, the group identifier was specified in the destination URL for the PPC objects.
Those. The URLs of keywords and Ad's on the search engine looked like this:
my_redirect_server.ru/?rotation_group_id=123456
When a visitor got to the intermediate server, destination URLs were rotated
and the visitor was redirected to the selected URL. The rotation took place taking into account temporary
destination URLs' settings, as well as other settings (display frequency, etc.).
Thanks to this solution, the problem was solved.

Underwater rocks.

It was very important to ensure the uninterrupted operation of the intermediate server, because
if it crashed, a situation could arise in which visitors
would come from a search engine (that is, they would spend money on clicks), but would not
fall on the destination URL. Those. money would be wasted.

In addition, it was necessary to provide load balancing in the event of a sharp increase in
traffic from a major advertiser.

Therefore, instead of one intermediate server, several physical
servers were used that balanced the load among themselves.

However, in addition to load balancing, these servers would also have to exchange
information with each other in order to ensure uniform rotation. Besides
Moreover, when changing the settings of the groups, each server would have to be notified, which
would not be convenient.
Therefore, to provide a rotation mechanism, a separate service was allocated on a separate
server.

This service was responsible for deciding which destination URL should
be displayed in each case. In the event of a fall in the Rotation service, the
redirect server continued to work on cached results until it was raised.

In the general case, it was generally impossible to allow such a situation when the system could not
send the visitor to any URL.

Since the rotation needed to be performed as quickly as possible, the Rotation service worked
exclusively on cached data groups and the current state of rotation. When changing the
settings, the Rotation service was notified of the need to update the cache.
Thanks to this approach, the high speed of selecting the destination URL was ensured.
Another problem with ensuring high speed of work is the saving of reporting
data, which could not be performed in synchronous mode.
Therefore, a mechanism was implemented in which the servers accumulated reporting data
in memory and dumped them in the background to the
statistics collection server in the background with a frequency of several minutes .

Another problem was the provision of high speed with geographical
remoteness of visitors and redirect servers.
When working with Baidu, most of the visitors were in China and they had
difficulty redirecting through servers located in Europe. This problem
was resolved by installing an additional server in China.

Conclusion

The solution described above was put into operation several years ago and since then
it has been processing up to 100 million requests every month, although the potential of the
solution is much higher.