June 21, 2015 at 19:44Guidelines for optimizing sites for beginners. Part 1
- Transfer
Website optimization, known as A / B testing, has gained popularity as a working technique for many teams creating websites. But so far there are too few comprehensive books, articles or training materials aimed at single developers seeking to introduce this technique in their organization.
In the first part of the manual, I will present detailed instructions on how to build, configure and improve this technique for my use.
This is a method of experimenting to check which design is best for your site. The idea is simple:
to create several versions of the page design or the page of your website
to share your site's traffic so that every visitor has seen or the current version (control group) or one of the new options
to keep track of which version performs better with the help of specific indicators
The indicators are chosen so as to directly reflect your business goals. These may include the number of items purchased on the site; the number of people who subscribe to the newsletter; the number of people watching the instruction video. Sometimes indicators are called conversion - this is the percentage of visitors who performed the desired action in relation to the total number of visitors.
Site development
There are two main types of optimization tests: A / B tests (also known as A / B / N) and multivariate tests.
In this case, you compare the work of several fixed design options with each other. They can differ only in one element (the color of the button or replacing the image on the video), or many elements at once (another page design).
Three different buttons for testing
Two different design options for
A / B pages are easier to develop and analyze, and they return results faster because they usually contain fewer options than multivariate tests. Most people work with just such tests.
Multivariate tests change two or more page properties and check which combinations work best. Their main difference from A / B tests is that they check how several different design dimensions work with each other and which leads to better results. In the following example, we are trying to understand which combinations of text and button colors will get the most clicks.
The simplest kind of multivariate test is a complete factorial experiment. This is a test of all combinations of factors. The downside is that such tests take the most time, since you share traffic between a much larger number of options than A / B tests.
Partially factorial methods use statistics and interpolation of results for certain combinations, reducing the amount of traffic needed for tests. But these methods are prettycomplicated mathematically .
Why are we doing tests? Goals, advantages and justifications
Optimization allows you to introduce a system based on indicators that determine the success or failure of a design. Thus, your team learns with each test. People will not argue unreasonably over design details. The effect "decides the one with the highest salary" will no longer work. By setting clear goals and suitable indicators, you get accurate data.
Personally, I’m not sure that you need to check absolutely all the smallest changes on the site. But in every organization, your web development strategy must be tied to measurable goals that are related to your business goals.
If you are told that the site should “provide the best customer support,” you need to determine which metrics best reflect this. Maybe this is the total number of tickets and emails that were answered from the site along with the user satisfaction rating, or the average user rating for answers in the FAQ section. As Galileo said: "measure what can be measured, and make measurable what is impossible."
In addition, you should try to measure the actual conversion, and not some simple indicators that are indirectly associated with it. For example, in the online store you should observe the number of paid orders, and not just placed ones. Your team should observe such real conversions as well as secondary steps and intermediate goals.
Even experts do not predict with 100% probability how optimization will affect the process. Therefore, it is necessary to engage in testing. Do not let team members push their design options simply based on their confidence. Test it out.
Make small changes often, and thanks to testing, you will be able to understand what exactly affects the conversion. Consider the previous example of A / B testing:
Imagine that the new director decided to completely redo the page. In a few months, you are launching a new three-column design. And you decide to conduct A / B testing, showing 10% of visitors an old design, and 90% - a new one.
It turns out that the old design works better. What to do? You can’t just throw out a new operating time. Surely some elements work better in the new design than in the old one. But since you made such a drastic change, it's hard to separate the good from the bad.
A better strategy is to constantly optimize different details on the page and conduct tests that will gradually evolve into a new version. As Jared Spool wrote in his article “ Silent Death of Big Restarts ,” “the best sites have replaced revolutionary processes with evolutionary ones. A complete redesign is a thing of the past, and gradual improvements are appearing in its place. ”
Part 2
In the first part of the manual, I will present detailed instructions on how to build, configure and improve this technique for my use.
Basics: what is website optimization?
This is a method of experimenting to check which design is best for your site. The idea is simple:
to create several versions of the page design or the page of your website
to share your site's traffic so that every visitor has seen or the current version (control group) or one of the new options
to keep track of which version performs better with the help of specific indicators
The indicators are chosen so as to directly reflect your business goals. These may include the number of items purchased on the site; the number of people who subscribe to the newsletter; the number of people watching the instruction video. Sometimes indicators are called conversion - this is the percentage of visitors who performed the desired action in relation to the total number of visitors.
Site development
A / B testing and multivariate method
There are two main types of optimization tests: A / B tests (also known as A / B / N) and multivariate tests.
A / B tests
In this case, you compare the work of several fixed design options with each other. They can differ only in one element (the color of the button or replacing the image on the video), or many elements at once (another page design).
Three different buttons for testing
Two different design options for
A / B pages are easier to develop and analyze, and they return results faster because they usually contain fewer options than multivariate tests. Most people work with just such tests.
Multivariate tests
Multivariate tests change two or more page properties and check which combinations work best. Their main difference from A / B tests is that they check how several different design dimensions work with each other and which leads to better results. In the following example, we are trying to understand which combinations of text and button colors will get the most clicks.
The simplest kind of multivariate test is a complete factorial experiment. This is a test of all combinations of factors. The downside is that such tests take the most time, since you share traffic between a much larger number of options than A / B tests.
Partially factorial methods use statistics and interpolation of results for certain combinations, reducing the amount of traffic needed for tests. But these methods are prettycomplicated mathematically .
Why are we doing tests? Goals, advantages and justifications
Optimization allows you to introduce a system based on indicators that determine the success or failure of a design. Thus, your team learns with each test. People will not argue unreasonably over design details. The effect "decides the one with the highest salary" will no longer work. By setting clear goals and suitable indicators, you get accurate data.
Three popular testing philosophies
1. We strictly follow the indicators
Personally, I’m not sure that you need to check absolutely all the smallest changes on the site. But in every organization, your web development strategy must be tied to measurable goals that are related to your business goals.
If you are told that the site should “provide the best customer support,” you need to determine which metrics best reflect this. Maybe this is the total number of tickets and emails that were answered from the site along with the user satisfaction rating, or the average user rating for answers in the FAQ section. As Galileo said: "measure what can be measured, and make measurable what is impossible."
In addition, you should try to measure the actual conversion, and not some simple indicators that are indirectly associated with it. For example, in the online store you should observe the number of paid orders, and not just placed ones. Your team should observe such real conversions as well as secondary steps and intermediate goals.
2. Nobody knows which option will be a winner
Even experts do not predict with 100% probability how optimization will affect the process. Therefore, it is necessary to engage in testing. Do not let team members push their design options simply based on their confidence. Test it out.
3. The release strategy "a little bit, but often"
Make small changes often, and thanks to testing, you will be able to understand what exactly affects the conversion. Consider the previous example of A / B testing:
Imagine that the new director decided to completely redo the page. In a few months, you are launching a new three-column design. And you decide to conduct A / B testing, showing 10% of visitors an old design, and 90% - a new one.
It turns out that the old design works better. What to do? You can’t just throw out a new operating time. Surely some elements work better in the new design than in the old one. But since you made such a drastic change, it's hard to separate the good from the bad.
A better strategy is to constantly optimize different details on the page and conduct tests that will gradually evolve into a new version. As Jared Spool wrote in his article “ Silent Death of Big Restarts ,” “the best sites have replaced revolutionary processes with evolutionary ones. A complete redesign is a thing of the past, and gradual improvements are appearing in its place. ”
Part 2