Growth Hacking at Retail Rocket: From Hypothesis Search to Testing Techniques

    Growth Hacking is full of myths. Some consider him a panacea for all ills, others - almost a quackery. Cases with incredible growth figures of tens and hundreds of times that try to copy thoughtlessly and without receiving the same growth are declared mistrust by declaring the approach inoperative.

    But in order to “hack the growth” of a company, it’s not enough just to single out a team and set them the task of looking for growth points. Growth Hacking is a very complex process that requires high expertise and a clear methodology.

    Since the founding of Retail Rocket, hacking growth has become an integral part of the company's work. For more than six years, we have developed a unique system for testing and selecting algorithms. Thanks in part to this methodology, we can provide the highest ROI in the market. And today we want to share the experience of using Growth Hacking in ecommerce.

    What is Growth Hacking and why any business needs it

    Growth Hacking is a continuous work of a separate team on the formulation, organization and analysis of experiments to ensure a high growth rate of business performance. This means that the business creates a separate department, which is engaged in the generation and testing of hypotheses that should affect the conversion, revenue, profit and other metrics.

    The emergence and active use of Growth Hacking is associated with a large-scale rise in the culture of startups, when there was a need for an understandable and affordable tool for testing hypotheses that provide companies with multiple growth.

    Growth Hacking is based on three principles:

    • Rapid Metrics Upgrade
    • Continuous optimization of results
    • Open process

    If you depict the process schematically, then everything looks quite simple: you need to generate a hypothesis that supposedly increases one or more metrics, check it, for example, using A / B testing, and analyze the result. Successful hypotheses can be scaled and optimized, and if unsuccessful, the cycle repeats.

    In practice, everything is much more complicated. From the choice of a hypothesis to the testing process and evaluation of results - at each stage a huge expertise and experience is needed. Partially, we talked about this in the article “Pitfalls of A / B testing or why 99% of your split tests are carried out incorrectly?” .

    At Retail Rocket, we have two Growth Hacker teams. One is engaged in the growth of metrics on the site, and the other is responsible for increasing the effectiveness of trigger emails. The approaches of the teams to the processes of breaking growth are similar at some points, but differ somewhere, which we will discuss in more detail during the course of the article.

    Let's start with the process of selecting hypotheses.

    How to find and select a hypothesis to test

    Globally, there are 2 approaches to Growth Hacking:

    • Random generation of ideas and their verification
    • Systematic testing of hypotheses collected in advance

    The first method is more expensive and resource-consuming, there are more chances of failure, but sometimes strong and breakthrough ideas are born.

    The second method works great if there is repeatability of experiments and you can collect a whole pool of hypotheses. It gives more predictable results, since there are already collected statistics from the results of previous tests.

    For Retail Rocket customers, we use a combination of these methods. First, we test the solutions that are most likely to work on this store.

    We conducted more than 2,000 tests on websites and more than 5,000 tests in trigger letters, stepped on all possible rakes during these tests, took into account the experience of not only Russian but also foreign stores, so we can recommend this option to every online store with great confidence which will be effective for him. Using a systematic hypothesis test, we increase mathematical expectations and reduce the likelihood of error.

    Many of our hypotheses, we not only describe in cases and show the results of their implementation, but also tell why we are testing this particular solution. Some hypotheses are based on psychological postulates and research, others destroy common stereotypes.

    After a systematic check of the hypotheses recommended for the store, the stage of more experimental and risky work begins. Growth Hacking specialists are looking for new solutions that will bring the store an increase in conversion and an average check. For trigger letters, new hypotheses, as a rule, are created during the brainstorm and each hypothesis must be substantiated from the point of view of psychology. The site team prefers a more practical approach based on research studies.

    Some hypotheses are offered by our customers. We evaluate the appropriateness of testing them, build approximate calculations of how this can work, and if we understand that the result is worth the effort, we conduct a test. Of course, customers know their audience better, therefore, in combination with our experience, some hypotheses give excellent results. The most successful ideas even become part of our pool of hypotheses. For example, the trigger scenario “Notification of a reduction in the price of goods in the basket” after a series of successful tests took a worthy place in our trigger map.

    Thus, by combining the expertise of store marketers with the experience of Retail Rocket specialists, we get a WIN-WIN system in which both parties benefit.

    How to test a hypothesis, or why we chose A / B tests

    For testing hypotheses in the online space, the AB testing process is excellent. He has a clear methodology and gives transparent results, of course, if you conduct a split test correctly.

    We have a clear test methodology, which includes several stages of checks and eliminates possible errors. There is a tool for sharing traffic. There is expertise and experience, thanks to which, when questions arise, we know where to look and where to look for problems.

    A / B test can be divided into several stages:

    • Running test. There could be many potential errors. The typical list contains incorrect layout and segmentation errors. Our methodology is based on cross-checking of tests and careful coordination of all details with a retailer, which helps to minimize possible errors.
    • Stop the test. How long does the test have to go so that the results can be interpreted unambiguously? How to understand where the effect of novelty goes into a working method of increasing conversion?
    • Evaluation of the results. Starting from the methodology for collecting and cleaning data to the conclusions that any analyst can verify, in the test reports we give the most complete and transparent information about what effect a particular solution gives.

    Thus, according to the results of testing, the store clearly understands which hypothesis will bring store growth in conversion and sales, and which one makes no sense to introduce.

    What it looks like in practice

    Here's an example of a couple of cases of how Growth Hacking works in Retail Rocket.

    Case 1. Using the principle of social evidence in trigger letters

    As we mentioned above, in trigger scenarios, hypotheses are often based on various postulates of psychology. For example, a good result in most cases shows the application of the principle of social proof. Its essence is that people trust the opinions of other people when choosing a product. Social proof can be used in various ways, for example, product reviews, ratings, information about how many users are viewing the product at the same time, etc.

    We decided to test the hypothesis of increasing the demand for the product through the implementation of the “Purchased today” block, which demonstrated the quantity of goods bought by other users.

    The work was carried out through A / B testing, in which all recipients of letters are randomly divided into two segments. The initial version of the letter is sent to segment A, and a letter with a change-hypothesis is sent to segment B, which should increase the efficiency of mailings.

    The hypothesis was tested in the trigger scenario “Letter with related products after the order” of the Mamsy sales club:

    According to the test results, the conversion to letters of the letter with the implemented hypothesis was 60.5% higher than the standard version (the statistical reliability of the result was 96.8%) .

    Case 2. Choosing the most effective option for displaying recommendations on the search page

    Now let's move to the site and look at the hypothesis test on the example of the ZdravCity online pharmacy.

    The main emphasis in testing on the websites of online stores is to test the effectiveness of various algorithms in order to understand what kind of recommendation mechanics will bring the retailer the maximum result. But we also check how various design elements will affect conversion rates, average receipt and revenue. This can be the introduction of a slider in recommendations, the addition of discount labels or other design decisions.

    In this case, the hypothesis was that if you add a CTA element to the goods in the recommendation blocks, it will be easier for the user to add the goods to the basket and this will increase the conversion and the average check.

    A performance study was conducted using A / B testing mechanics. All visitors to the site were randomly divided into 3 segments:

    • The first segment showed a block of recommendations without a CTA element (basic appearance)
    • The second segment was shown a block of recommendations with the addition of the CTA-element "Add to Cart" (adds the product to the cart)
    • No recommendations were shown to the third segment (control group)

    According to the test results, the implementation of the hypothesis “Block of recommendations with the addition of a CTA-button“ Add to Cart ”on the search page of the online store increases the conversion by 1.05% (statistical significance 99.5%). In combination with an increase in the average check by 7.3%, which gives a projected increase in revenue by 8.4%.

    These are just a couple of examples of how you can generate and test various hypotheses for metric growth. We regularly talk about this in cases, so if you want to find out more interesting hypotheses and the results of their verification, go to our blog .

    And remember, the more experiments you do, the higher your result will be over a long distance. Therefore, conduct A / B tests, test hypotheses and find those solutions that will bring growth to you.

    Also popular now: