Cohort analysis shows a picture completely different from our usual perception.
Let me take you back a little while. Imagine that you are standing instead of me at one of the boards and trying to explain to your colleagues your new concept of metrics. If to say about my feelings at that moment - it was despair. I clearly understood that, unfortunately, my words could not reach the interlocutors. None of the participants in the meeting at all took any of my thoughts. They did not believe me.
They didn’t believe it because I didn’t logically explain the essence or said something stupid. Not. From this point of view, everything was fine. “What you offer is really interesting and innovative, but ... let's do it all the same.” What a shame it was to hear.
I was developing a system of metrics that several thousand people should focus on. Imagine that you are very clearly aware that the people around you are stubbornly and persistently, with all their forces, strive to fall into the deepest hole, the price of falling into which is incredibly high. You just do not have the right to allow this to happen, understand? Just do not have the right!
But at that moment I, like a dog, looked with devoted eyes at my colleagues and could not say anything. You know, there are several outstanding people in the world who I really like. And one of them is Elon Musk. After another unsuccessful launch of the Falcon rocket, his company was completely despondent. Despite the fact that the day was very difficult, despite 20 hours spent on his feet and the blow that befell him, Musk spoke to the company, supported the employees and concluded his speech with the words: “I myself will never give up. Never! ”
The words of Mask then surfaced in my head:“ I will not give up! ”
Feverish thoughts streamlined into the understanding that my interlocutors simply did not see the differences, did not see confirmation of my theoretical layouts. Maybe they even believed them, but didn’t want to spend the company’s money on developing what “most likely will not work”. Those. I needed to show the real differences between the cohort representation of metrics and the usual one — to show what was the ambush of the usual “sugary” metrics and what the cohorts beat them with.
But how to do this, if there is no data, the necessary metrics are still not collected by anyone, and there is no mention of any tools for them either? How to be The only way with which it was possible to show at least something was to simulate everything in regular Excel.
Eyes are afraid, but hands are doing. Exactly 4 hours later, a demo dataset and visualization for it were ready, showing the fundamental difference between the usual and cohort presentation of metrics. Honestly, the results shocked even me.
What do we use metrics for? To shorten the time-to-market, the time taken by teams to complete the entire software product development chain.
As an example, the demo data set analyzed the time the team successfully passed the DevOps deployment booth (checking distributions for correctness).
The task is to understand how much time the teams spend on passing the stand and whether their performance improves. Here is what the data were (created using the www.generatedata.com service ):
The first thing that comes to mind is to see how the teams go through the booth.
The X axis is the month starting January 1, 2017. On the Y axis - the number of teams that have passed the stand.
Even a cursory analysis shows that this kind of schedule is completely useless, because it does not reflect the objective picture - the same team makes many assemblies and passes the stand several times a year, and this is not visible on the schedule.
Anyway, it is not clear how to analyze such a schedule and what all these rises and falls mean. In a word, the usual useless metric is like site traffic.
OK. Let's try to calculate the average duration of the stand by the commands:
“What a good schedule!” Exclaimed my interlocutor when he saw this picture on the screen.
- Everything is clear: the Finance team is the best, and Tech Support is the worst. The average time the stand travels is 3 months. Beauty! We will arrange a competition and will give a bonus for shortening the period for passing the stand
- Eeee .... Wait, wait. There is clearly something wrong here ...
- But what could be wrong? Everything is super! What you need
And it was here that the cohort analysis showed itself in all its glory. A typical view of a table that provides the time for passing the stand by cohorts:
I am sure that you would not have believed such a table either. But despite the apparent incomprehensibility, the table opens a real new universe of metrics and shows the picture in a completely different light. It turns out that the “worst” team, Tech Support, is in fact almost the coolest editor who has reduced stand time by more than 100%:
Can anyone be punished for this? Yes, it is necessary to reward!
The solid middle Accounting really works very badly, constantly worsening its results:
And the very “best” Finance team even increased the time so much that the value of the first column is not visible behind the greatly increased second:
But this completely changes the whole picture! Do you understand ?! Totally! Everything becomes exactly the opposite, scruff of the neck. It turns out that the ambush is hiding there.
As you can see, the “average in the hospital” reflects a completely incorrect distorted picture of the world, which in principle cannot be used to make managerial decisions. Fundamentally can not. But the cohorts, on the contrary, give the right answer - the performance of specific teams improves or worsens and by how much.
Yes, maybe they don’t answer the question of who is the best, but is it necessary to know this at all? ..
That is why cohort analysis is the future that has come now. This is exactly what you need to implement everywhere wherever there are measurements, metrics and graphs.
Join me with the cohort analysis team! And success in work.
PS: It was nice to see the sagging jaws of the interlocutors at the second show.
They didn’t believe it because I didn’t logically explain the essence or said something stupid. Not. From this point of view, everything was fine. “What you offer is really interesting and innovative, but ... let's do it all the same.” What a shame it was to hear.
I was developing a system of metrics that several thousand people should focus on. Imagine that you are very clearly aware that the people around you are stubbornly and persistently, with all their forces, strive to fall into the deepest hole, the price of falling into which is incredibly high. You just do not have the right to allow this to happen, understand? Just do not have the right!
But at that moment I, like a dog, looked with devoted eyes at my colleagues and could not say anything. You know, there are several outstanding people in the world who I really like. And one of them is Elon Musk. After another unsuccessful launch of the Falcon rocket, his company was completely despondent. Despite the fact that the day was very difficult, despite 20 hours spent on his feet and the blow that befell him, Musk spoke to the company, supported the employees and concluded his speech with the words: “I myself will never give up. Never! ”
The words of Mask then surfaced in my head:“ I will not give up! ”
Feverish thoughts streamlined into the understanding that my interlocutors simply did not see the differences, did not see confirmation of my theoretical layouts. Maybe they even believed them, but didn’t want to spend the company’s money on developing what “most likely will not work”. Those. I needed to show the real differences between the cohort representation of metrics and the usual one — to show what was the ambush of the usual “sugary” metrics and what the cohorts beat them with.
But how to do this, if there is no data, the necessary metrics are still not collected by anyone, and there is no mention of any tools for them either? How to be The only way with which it was possible to show at least something was to simulate everything in regular Excel.
Eyes are afraid, but hands are doing. Exactly 4 hours later, a demo dataset and visualization for it were ready, showing the fundamental difference between the usual and cohort presentation of metrics. Honestly, the results shocked even me.
What do we use metrics for? To shorten the time-to-market, the time taken by teams to complete the entire software product development chain.
As an example, the demo data set analyzed the time the team successfully passed the DevOps deployment booth (checking distributions for correctness).
The task is to understand how much time the teams spend on passing the stand and whether their performance improves. Here is what the data were (created using the www.generatedata.com service ):
The first thing that comes to mind is to see how the teams go through the booth.
The X axis is the month starting January 1, 2017. On the Y axis - the number of teams that have passed the stand.
Even a cursory analysis shows that this kind of schedule is completely useless, because it does not reflect the objective picture - the same team makes many assemblies and passes the stand several times a year, and this is not visible on the schedule.
Anyway, it is not clear how to analyze such a schedule and what all these rises and falls mean. In a word, the usual useless metric is like site traffic.
OK. Let's try to calculate the average duration of the stand by the commands:
“What a good schedule!” Exclaimed my interlocutor when he saw this picture on the screen.
- Everything is clear: the Finance team is the best, and Tech Support is the worst. The average time the stand travels is 3 months. Beauty! We will arrange a competition and will give a bonus for shortening the period for passing the stand
- Eeee .... Wait, wait. There is clearly something wrong here ...
- But what could be wrong? Everything is super! What you need
And it was here that the cohort analysis showed itself in all its glory. A typical view of a table that provides the time for passing the stand by cohorts:
I am sure that you would not have believed such a table either. But despite the apparent incomprehensibility, the table opens a real new universe of metrics and shows the picture in a completely different light. It turns out that the “worst” team, Tech Support, is in fact almost the coolest editor who has reduced stand time by more than 100%:
Can anyone be punished for this? Yes, it is necessary to reward!
The solid middle Accounting really works very badly, constantly worsening its results:
And the very “best” Finance team even increased the time so much that the value of the first column is not visible behind the greatly increased second:
But this completely changes the whole picture! Do you understand ?! Totally! Everything becomes exactly the opposite, scruff of the neck. It turns out that the ambush is hiding there.
As you can see, the “average in the hospital” reflects a completely incorrect distorted picture of the world, which in principle cannot be used to make managerial decisions. Fundamentally can not. But the cohorts, on the contrary, give the right answer - the performance of specific teams improves or worsens and by how much.
Yes, maybe they don’t answer the question of who is the best, but is it necessary to know this at all? ..
That is why cohort analysis is the future that has come now. This is exactly what you need to implement everywhere wherever there are measurements, metrics and graphs.
Join me with the cohort analysis team! And success in work.
PS: It was nice to see the sagging jaws of the interlocutors at the second show.