# How "a simple experiment" brought Obama \$ 60 million [translation]

On October 21, 2010, President Barack Obama visited Palo Alto. To raise money for his campaign, a dinner party was organized, the cost of participation in which was \$ 30,400. This is an extremely effective method of raising funds, although it can only work if you have the opportunity to invite the president. How to raise funds if you only have a website that no one has seen but your grandmother? It was in this situation that we found ourselves in the distant 2007, when Obama was nominated for the presidency, and the lag behind the closest competitor was expressed by a double-digit percentage.

In 2008, when I worked as director of the analytical department of the Obama administration, my responsibilities included making decisions based on data analysis. We started with one simple experiment in December 2007. Based on this experiment, we concluded that each visitor to our site was a unique “opportunity to raise funds” and, more importantly, using this “opportunity” through optimization and A / B testing, we could attract tens of millions of dollars .

This experiment tested two blocks of our site: a block with a picture and video (Media section) and a block with a button (Call-to-action).

For the experiment, we used Google Website Optimizer and conducted a full factorial multivariate test. In simple words, we tested all combinations of buttons and media content. We had 4 button options, 3 pictures and 3 videos, which meant 24 (4 * 6) combinations for the test. One of the combinations was randomly shown to each visitor of the landing page, and we, in turn, tracked the number of visitors who left their e-mail.

Before you see the results, try to guess which button and media content in your opinion were the most successful?

#### results

The metric we used to measure the success of the experiment was the rate at which subscriptions were received. To be more precise, this is the number of people subscribing, divided by the number of people who saw a particular version of the page. Since a total of 310,382 visitors visited the landing page during the experiment, this means that approximately 13,000 people saw each option.

Here are the values ​​we observed for each of the landing page options:

And we observed such values ​​when evaluating combinations of different options:

#### Winner

The best combination of a button and media content was “Combination 11”, which consisted of a “Learn more” button and a family photo:

Before we started the experiment, the people working on the campaign preferred “Sam's Video” (the last in the slide list show above). If we had not started the experiment, we would most likely have put this particular video on our landing page. As a result, such a trifle could be a fatal mistake, because as it turned out, any of the videos affects the conversion weaker than any image.

11.6% - this is the conversion in the subscription that the winning option had, in contrast to 8.26% of the conversion of the original page. Thus, we managed to get 40.6% more signatures. Let's try to understand where this figure came from. If we consider that this improvement will retain its effect throughout the campaign, then later, closer to the end, we will be able to evaluate the final numbers and determine the changes that this experiment led to. Roughly speaking, 10 million people signed up for a landing page during the campaign. If we had not started this experiment and simply left the original page, then this number would have been about 7.1 million people. And this is less by as much as 2.9 million email addresses.

Sending emails to people who left their mail with an offer to become volunteers allowed to attract 10% of them. As a result, it turns out that we managed to attract 288,000 volunteers from 2,880,000 email addresses, and this is much more than originally planned. On average, each email address that was received using our landing page donated \$ 21 for the entire campaign. Altogether, an additional 2,880,000 email addresses from our list resulted in an additional \$ 60 million in donations.