Vanilla metrics: how we fool ourselves, management and customers

The subscriber base of 50,000 people, 10,000 website visitors, 700 application downloads, 500 Facebook likes, 300 followers on Twitter - it sounds beautiful, right? We insert these figures in the report to the management, show them in presentations to clients and always get enthusiastic responses. For example, “I have 14,000 article views on Megamind.” The crowd amicably says: "Wow, you're cool." But what do these numbers ultimately give? What is the end result? Let's talk about it?

When analyzing our activities, we often use the following indicators:
• number of registered users;
• group size and the number of followers on social networks;
• the number of downloads of your application in the Google Play or Apple Store;
• size of email database;
• number of site views;
• time spent by the user on the site.

These are the so-called "vanilla metrics." This concept includes any non-objective indicators, as a rule these are big beautiful numbers, but you cannot judge the quality of the result and generally the existence of the result as such. In presentations, such graphs of these indicators look like a hockey stick =)


I encounter vanilla metrics every day, many do not understand that work cannot be evaluated in this way.
Despite the fact that these metrics are, frankly, stupid, you should not refuse them. There are areas where these metrics are used successfully, but usually not from a good life:

1) Sales.

We need to convince the buyer that we have a super reliable company with high performance. Here various vanilla metrics and ratings and recommendations of various slaughter companies, associations and associations come into play.
When I worked in one network company to create a successful image, we added to these metrics the ratings of the high ratings of consulting companies that few people had ever heard of, but these are numbers that you can grasp at.
"Our company, according to the consulting company Horns and Hooves, has an AAA + rating, only Deutsche Bank and a couple of companies have such a rating, even Microsoft has it lower."

2) Reports.

When there is no result, but it is necessary to show the result - here, too, big beautiful numbers help us out. In business this does not always go away (except for situation 3 in the figure), but in state institutions this happens very often. The management and founders like to see the dynamics, if you show that last year there were 5,000 visitors on the corporate website, and this year 7,000 visitors, then you are great, you work well.
The main thing is successful and confident growth, it does not matter that it is the growth of vanilla metrics. If you ask how many of these visitors became customers and how much they brought money, then the answer will be depressing, but it does not occur to anyone. Although, maybe it's just a giveaway game. Management does not invest in this area and therefore looks through these fingers through these reports.



As can be seen from the figure, such tactics are particularly effective in the third case, when a stupid boss and smart employee.

A separate example is the site performance reports. Google Analytics and Yandex Metric are configured by default for vanilla metrics. Few people customize them to their own needs, setting goals and monitoring the conversion. Rejoicing at the incredibly beautiful indicators, managers choose some of the most beautiful indicators to demonstrate to management. Bosses are happy with the schedules, marketers justified the costs. The sheer gain!

The correct metrics.

Now I will say a few words about the correct metrics.
The main criteria for the correct metrics:
1) If the metric changes, but you do nothing, then this is a bad metric. Changing metrics should change our behavior. The correct metric is its indicator of whether everything is going smoothly or problems are observed somewhere. If you don’t react to metrics in any way, why measure them at all?
2) Metrics should be Comparative (compare X1 with X2) and relative (in%). For example, sales growth in April compared to March was 10%. Here sophisticated civil servants go to a new level by making comparative vanilla metrics, as was the case with the reports above.

Dave McClure, creator of Pirate Metrics (AARRR), identifies 5 key metrics:



Attraction - attracting users to the site from different channels. Vanilla metrics show thousands of downloads, but most of them are dead souls. Pay attention to the number of transitions and the share of failures.
Activation - visitors are satisfied with their first visit and received their first “positive experience” with the service / product. I looked over 2 pages, subscribed to the newsletter, downloaded the application. The first stage of the sales funnel is passed, what is the conversion?
Retention - visitors regularly return to the site, often visit it, begin to use the service or application.
Income - users begin to pay for the proposed services and services thereby confirming the hypothesis of the monetization model
Recommendations- users like the product so much that they start recommending it to friends.

Steve Blank, founder of Customer Development, describes this process of attracting and developing customers as follows:



Stage 1 - attracting and activating consumers;
Stage 2 - retaining consumers (loyalty program, contests, updates);
Stage 3 - growing consumers (additional sales, next sales , cross-selling, recommendations).

Each stage in both funnels can be measured with separate metrics.
Read more about Steve Blanca and his concept of “Customer Development” in one of the following articles.

What metrics is still important to track?

• CAC - customer acquisition cost
• LTV - the customer’s lifetime value (the full profit from the client received for the entire time of cooperation with him).
• The ratio of CAC and LTV. Ideally, LTV should be three times the CAC.
• ARPU - user revenue.

There are many other metrics that are beyond the scope of this article. Write in the comments if you are interested to know more about this.

An example of using “pirated metrics” (Dave McClur)

Summary.

Vanilla metrics are good for dusting someone’s brains, but if you are guided by them yourself when analyzing the effectiveness of a business or startup, it will lead to a disastrous result.