An example of convenient organization of split (A / B) testing on the server side

Hello, dear habravchane!
I would like to share with you some of the experience of organizing split testing on a project I had a chance to work with, also please do not judge strictly, as this is my first article and really should not be expected to make a huge profit.
As we all know, split (A / B) testing on a site is an experimental approach with the goal of increasing the conversion of a process on a resource by displaying different content to different groups of users.

The goal of the task was to organize a fairly complex testing of different content options (both static and dynamic), with a possible unlimited number of experiments and options on each page of the resource. After some analysis of the available ready-made solutions to the task, we managed to find several options - these were GWO (no longer exists as a separate service, integrated into GA), a solution based on nginx and Visual Website Optimizer. But not one of them was suitable for a number of reasons: external services transfer rendering to the client side, which can be very uncomfortable with complex tests, and nginx is far from everywhere, plus developers do not always have access to the web server configuration and there is also a problem smearing logic. Therefore, I decided to implement my option bike , based on the example of the implementation of the nginx module.
I also did not like the standard implementation tips (the remainder of making the last digit of the client’s IP address, etc.), therefore I decided to use checksums from the test key to determine the options — the profit of the option is to use absolutely any alphanumeric key, the possibility of creating a huge number of options testing with any percentage. The choice of the hash function fell on MurmurHash2 , since it is characterized by a very good distribution and, what can I hide, is used for hashing in the nginx module. The project I'm working on is implemented in php, I recommend using this extension to implement the hash function .
Now let's touch on the implementation. To determine the value of a test case for a particular experiment, I implemented the SplitTestModel class, using it as a singleton:
class SplitTestModel
public static function getInstance($key = ''){...}
private function __construct( $key ){...}
public function getSplitVariantByExperiment( $experimentName ){...}

I put information about each experiment into the config, which, in my implementation, is an array of the form:
    'Experiment1' => array(
        'Variant1' => 50,
        'Variant2'   => 50
    'Experiment2' => array(
        'Variant1' => 15,
        'Variant2' => 35,
        'Variant3' => 12,
        'Variant4' => *

Where the top-level key is the names of the experiment, the nested keys are the names of the options, and the corresponding values ​​are the percentage of impressions, '*' means all remaining percentages.
As already possible, it became clear that the key is hashed using MurmurHash2. In the example given in the first experiment, for hash values ​​from 0 to 50% of the maximum hash value, the user will receive a value variant equal to 'Variant1', from 50% of the maximum possible value to 100% - respectively, the value of 'Variant2'.
The convenience of the method lies in the fact that now, in any place of the code, for a specific user, we can get its variant values ​​for any experiment and, depending on the variant, give the client this or that content of any part of the page. Usage example:
$variant1 = SplitTestModel::getInstance($user_id)->getSplitVariantByExperiment('experiment1');
$variant2 = SplitTestModel::getInstance($remote_addr)->getSplitVariantByExperiment('experiment2');

Everything is extremely simple!
Thank you very much for your attention, and once again I ask you not to judge strictly. If anyone is interested, I will gladly send the source code of the class.

reference to class implementation on github

Also popular now: