April 25, 2013 at 14:41How I implemented A / B testing in PHP
There was a problem of conducting split testing of our online store. Googled on this topic. I found a couple of services, but for one reason or another they did not suit us.
For example, using testing through google analytics did not work, because we use Yandex metrics for statistics and we don’t want to embed another analytical system in the site. And besides, as I understand it, there is a problem with Google when testing dynamic pages or some of their elements.
Other services, such as abtest.ru (which, by the way, for some reason does not work) or similar, are also not suitable, because there you can only change the appearance of something. And we, for example, need to test the site for the conclusion / non-withdrawal of one or another information from the database (well, as an example, whether to display a brief description of the goods in the list of goods or else display specifications, etc.).
I decided to try to implement A / B testing on my own, adding a little php code to the site (a self-written online store, in php).
It is necessary to check how the change in one or another element of the site will affect the conversion. Those. All visitors to the site are conditionally divided into “visitors-A” and “visitors-B”. We divide naturally equally, i.e. every first is “A”, every second is “B”. Depending on whether this is “A” or “B”, we show him the corresponding version of the site element. At the end, we fix whether the visitor placed an order or
not I will not describe the code, but only briefly explain what and how he did:
1. At the entrance to the site, each user in the session (say $ _SESSION ['split']) writes the value "A" or "B". Every first one is “A”, every second one is “B”. To do this, the database has a special “last visitor counter” that contains data about the last visitor to the site. That is, the visitor visited the site, they wrote the value “A” in $ _SESSION ['split'] and the counter in the database also changed to “A”. When the next visitor visits the site, the script takes the counter value from the database, and if it = "A", then the visitor writes "B" to the session and vice versa. Those. thus, user rotation is ensured. By the way, the question is: can someone tell me how to implement the alternation of visitors without accessing the database, how to find out what value to assign to the visitor “A” or “B”? After all, for this you need to know what significance the previous one had. Or can you not take a steam bath and just assign randomly the first or second option with this random? Will the visitors split evenly in this case?
2. After the value “A” or “B” was recorded in the user’s session, now depending on this value, the user is shown either the first or second version of the element being tested. Those. a person went to the page where the element under test is located, immediately a check is made of what is written in $ _SESSION ['split'] and depending on the result, the first or second option is shown.
3. Saving the results. For this, there are counters in the database for counting results (for counting the number of orders “visitors-A” and “visitors-B.” That is, each of the two options “A” or “B” has its own counter. If the visitor has in $ _SESSION ['split'] sits the value “A” placed an order, then counter “A” increases by 1. If visitor “B” placed an order, then counter “B” increases by 1.
In fact, it turns out pretty simple code. But this is scary. It seems that I did not take into account some subtleties and nuances of the A / B testing process and everything is much more complicated and deeper. After all, various controversial situations are possible. For example, how to take into account visitors who visited the site, then left the site and returned again? Whether to show them during the second visit to the site that version of the test element that they showed during the first visit, or to re-take this visitor into the rotation queue. And here the question already arises of using not the session mechanism, but the cookie mechanism.
I wrote this post to share my implementation of the simplest A / B testing and at the same time listen to the opinions of experts on this subject. Maybe there will be some tips, comments, etc.
For example, using testing through google analytics did not work, because we use Yandex metrics for statistics and we don’t want to embed another analytical system in the site. And besides, as I understand it, there is a problem with Google when testing dynamic pages or some of their elements.
Other services, such as abtest.ru (which, by the way, for some reason does not work) or similar, are also not suitable, because there you can only change the appearance of something. And we, for example, need to test the site for the conclusion / non-withdrawal of one or another information from the database (well, as an example, whether to display a brief description of the goods in the list of goods or else display specifications, etc.).
I decided to try to implement A / B testing on my own, adding a little php code to the site (a self-written online store, in php).
It is necessary to check how the change in one or another element of the site will affect the conversion. Those. All visitors to the site are conditionally divided into “visitors-A” and “visitors-B”. We divide naturally equally, i.e. every first is “A”, every second is “B”. Depending on whether this is “A” or “B”, we show him the corresponding version of the site element. At the end, we fix whether the visitor placed an order or
not I will not describe the code, but only briefly explain what and how he did:
1. At the entrance to the site, each user in the session (say $ _SESSION ['split']) writes the value "A" or "B". Every first one is “A”, every second one is “B”. To do this, the database has a special “last visitor counter” that contains data about the last visitor to the site. That is, the visitor visited the site, they wrote the value “A” in $ _SESSION ['split'] and the counter in the database also changed to “A”. When the next visitor visits the site, the script takes the counter value from the database, and if it = "A", then the visitor writes "B" to the session and vice versa. Those. thus, user rotation is ensured. By the way, the question is: can someone tell me how to implement the alternation of visitors without accessing the database, how to find out what value to assign to the visitor “A” or “B”? After all, for this you need to know what significance the previous one had. Or can you not take a steam bath and just assign randomly the first or second option with this random? Will the visitors split evenly in this case?
2. After the value “A” or “B” was recorded in the user’s session, now depending on this value, the user is shown either the first or second version of the element being tested. Those. a person went to the page where the element under test is located, immediately a check is made of what is written in $ _SESSION ['split'] and depending on the result, the first or second option is shown.
3. Saving the results. For this, there are counters in the database for counting results (for counting the number of orders “visitors-A” and “visitors-B.” That is, each of the two options “A” or “B” has its own counter. If the visitor has in $ _SESSION ['split'] sits the value “A” placed an order, then counter “A” increases by 1. If visitor “B” placed an order, then counter “B” increases by 1.
In fact, it turns out pretty simple code. But this is scary. It seems that I did not take into account some subtleties and nuances of the A / B testing process and everything is much more complicated and deeper. After all, various controversial situations are possible. For example, how to take into account visitors who visited the site, then left the site and returned again? Whether to show them during the second visit to the site that version of the test element that they showed during the first visit, or to re-take this visitor into the rotation queue. And here the question already arises of using not the session mechanism, but the cookie mechanism.
I wrote this post to share my implementation of the simplest A / B testing and at the same time listen to the opinions of experts on this subject. Maybe there will be some tips, comments, etc.