Site testing: measure 7 times, cut once

Posted by Ivan Krasyuk - Kokoc.com Client Network Development Director (Kokoc Group).

Visitors to any site are ordinary people who buy where the most convenient and attractive interface is. Some executives or marketers believe that they know the user's preferences best. In fact, it is impossible to guess whether they will like the new site design or the color change of the basket. See for yourself.

Suppose you need to choose how the elements will be located on one of the pages of the site. Which of the two options would you choose?


or


Which of the proposed options do you prefer here?



What kind of basket do you think is better?



or



In fact, no matter which option you choose, you made a mistake at least once with a probability of more than 65%. A similar error for the online store can lead to a decrease in the number of orders. Therefore, in order for the site to work efficiently, and its innovations do not accidentally scare away users, it is necessary to conduct testing.

What can be tested on the site? Everything:

• buttons (size, text, color);
• headers (size, text, location);
• calls to action;
• input forms;
• default page options;
• personal recommendations;
• arrangement of blocks;
• the mechanics of the action of elements on the site;
• photographs, drawings, and other graphic elements;
• the presence and composition of individual blocks and elements;
• sorting of goods and services;
• order of menu items;
• filters, search.

Almost anything can affect the conversion: adding goods, changing the basket or order form. You cannot embed a single element on a site without first having tested it.

The principle of testing is as follows. For example, you have two options for a picture on a page. In this case, it is necessary to divide the audience of the site into two parts, each of which sees a page with one of the pictures. From which page the conversion will be higher, that option should be accepted. The ratio of parts when dividing the audience depends on the timing and willingness to take risks. If the results need to be obtained quickly, visitors are divided exactly in half. In this case, the conversion may fall. The risk is minimized if you show a new version of the page at first 10% of users and gradually increase their share, watching the conversion. However, such a cautious approach will take longer.

The main types of tests:

A / A.This test is always carried out first in order to understand that the audience is divided evenly. You must create an identical copy of the page and separate 50% of the audience. Thus, when entering the site, half of the users see the old page, and the other half - the new one, which completely duplicates it. The conversion of both pages should be the same.

A / B. It is used when only one factor is considered that affects conversion, and there can be many options. For example, you decide which color to buy button: red, green and blue. For testing, it is necessary to divide the audience of the site into 3 equal groups and show them pages with buttons of different colors, each group by color. Next, after studying user behavior, choose the most conversion option.

MVT.A multivariate test is performed if it is necessary to test different versions of several factors. Let's say you need to understand in what order it is better to place the buy button and a video clip. You have chosen 3 options for the color of the button (red, green and blue) and 2 options for the position (video above the button, video below the button), that is, 6 different page options are obtained. Next, you need to divide the site visitors into 6 equal parts and see which of the combinations gives the greatest conversion. For example, the video above and the red button.

There are five steps to effective testing. Consider them on the example of an online store of mobile and digital technology.

Hypothesis identification and definition of macro and micro conversions (business goals).
At this stage, analysts, along with business owners and marketers, find weak points on the site, that is, places where the audience is leaving. Experts identify hypotheses (reasons for low conversion), from which they choose the highest priority, that is, the problem whose solution will bring more profit.

Pre-test analysis with the economic potential of the test. With the help of Google Analytics data, assumptions are made about how much the conversion can increase if you change the selected item. The forecast is expressed as a percentage of conversion, cash profit and turnover.

The formation of variations in the representation of the influencing factor. Together with designers and programmers, analysts think through and create several options for the element being tested.

Testing process with regular assessment of the significance of the results.Testing can be carried out using various software, for example, Google Content Experiments, Visual website optimizer or Optimizely. GA tester differs from other programs in that it is built into GA and is free. Visual website optimizer or Optimizely cost from $ 50 to $ 500 per month, depending on the amount of traffic and the number of tests per month.

Post-test analysis with a business case and confidence assessment. Based on the results of testing, analysts identify the most conversion option and explain what results the introduction of this element will bring to the business.

Consider the stages of testing on the example of an online store of mobile and digital technology. The business goal of the test in this case is to increase the number of orders. According to Google Analytics, the conversion rate for site users who viewed reviews exceeds the site average by 1.5 times. At the same time, the opinions of other buyers are very difficult to find on the product page and there is no clear indication that the diamonds from the rating are the result of user ratings.



The priority hypothesis was the statement that the new page of the product card contributes to an increase in the number of orders. In order to draw attention to the reviews, it is necessary to modify the unit with the rating, as well as its location. Using Google Analytics data, Kokoc.com analysts suggested that this would increase conversions by 0.06%. It was proposed to add a heading to the block indicating that it is formed on the basis of the opinions of users, and place a link to them nearby. For testing, 3 options were developed.


Option number 1.


Option number 2.


Option number 3.

A multivariate test was carried out for a week using the program Visual website optimizer (VWO). For completeness of the analysis of results in VWO, the following goals were additionally configured:

• Goal number 1 - any interaction with the page;
• Goal number 2 - click on the buy button;
• Goal №3 - click on the button "Order in 1 click";
• Goal 4 - opening a loan calculator;
• Goal №5 - click on the button "Buy on credit";
• Goal №6 - go to the product reviews page;
• Goal 7 - order on the site;
• Goal number 8 - click on the image of the product.

As a result of testing, option No. 1 won, the indicators of which were maximum in terms of the number of orders, clicks on the "buy" button and clicks on the feedback page. The VWO report showed that visitors who saw an alternative page opened product reviews 4.1% more often than those who saw a regular page (Fig. 12). Also, indicators of option No. 1 by clicks on the buy button above the control option by 0.23% (Fig. 13), and by orders - by 0.16%.


Screen of the visual website optimizer report on review views.


Screen report Visual website optimizer on clicks on the buy button.


Screen report Visual website optimizer on orders.

According to VWO for the reporting week, the product page was visited by 249,439 unique users. Thus, when approving an alternative page option and an average order of 12,000 rubles, analysts predicted the following increase in turnover:

• minimum: +742 928 rubles per month (~ 8 915 143 rubles per year);
• expected: +19 322 687 rubles per month (~ 231 872 254 rubles per year);
• maximum: +37 902.447 rubles per month (~ 454 829 366 rubles per year).

What increase will ultimately depend on many factors: from the weather to the economic situation in the country.

In addition to the economic justification of the results, analysts assess their reliability and significance. For example, if you test two page options on two users, the results will not be true. The more views, the more reliable the testing.

When analyzing the results, it is also necessary to take into account that the conversion is different: macro and micro. Macro conversion is a purchase or a call. Micro conversion can mean viewing a product’s card, using a search, adding it to the basket, clicking the “buy on credit” button. It’s also important to consider indicators such as average bill and total revenue. For example, the conversion might not have grown, but the average bill increased, and as a result, the total income increased. There may be a reverse situation, an increase in conversion is accompanied by a decrease in the average check, as a result of which the total income indicator can also decrease. A truly objective picture is shown by the aggregate analysis of all parameters, macro and micro targets using various segmentation.

Testing is an indispensable tool when introducing any innovations or changes to the site. Most often, out of 10 tests, only 4 give a positive result, and 6 do not give any result or reduce the conversion. In this regard, testing must be carried out continuously. It’s best to create a calendar of tests, for example, for a quarter. Moreover, if some test did not give results today, it can be put at the end of the calendar and re-run. During this time, the site has changed, so the results may be different, the test may work. The more tests, the more effective your site.