June 23, 2014 at 12:01The line between cargo cult and evolution in learning to succeed
Entrepreneurs with great enthusiasm are adopting tools for verifying future profits. The idea of A / B testing fell into fertile soil. There is no more need to puzzle over the color of the button, you can immediately test and choose the best.
Let me remind you how this is done: you create two pages whose difference will only lie in the color of the “checkout” button. Then, at random, half of the visitors to the site show page number 1, and the other half - page number 2. As a result, on one of the pages, users will click the checkout button more than the same number of other users on another page.
Does it follow that the color of the button affects the number of placed orders? Let's do a thought experiment. We will have two teams, five people each. One team will be in red t-shirts, the other in blue. Each member of the team will toss a coin and write down what happened: heads or tails. Let everyone toss a coin, say, three times. After, we calculate the number of “eagles” received for each of the teams.
We will see that one of the teams scored more “eagles” than the other. Can we conclude that the color of the T-shirt determined the winner? Should we wear a red (blue) t-shirt when we are going to earn more “eagles”?
We all know perfectly well that if our teams would toss a coin an infinite number of times, then the results for the teams would be the same. We know this, only thanks to the understanding of the model, since there was no immortal and at the same time a great lover to test everything in practice.
The conclusion suggests itself, but let's not rush into conclusions. Imagine that you received a message saying that a group of scientists has developed an excellent technology for predicting the results of football matches. And to prove its revolution, the message tells you the outcome of the match, which will take place tomorrow. Of course, the result is correct. A week later, you again receive a message from the same “group of scientists” who, to finally convince you to purchase their product, again send you the correct outcome of the next match.
If you are surprised by their insight, then it has a simple foundation. The first message with the text “Brazil will defeat Mexico” is sent to one group of people, and with the text “Mexico will defeat Brazil” to the second group of people. After the results of the Mexico – Brazil match become known, the second message with the next “prediction” is sent only to the group to which the correct prediction was previously randomly given.
This may be a good way of persuading, but obviously a bad way of predicting. Is there something similar in the way we are now trying to verify the success of one strategy over another? Is it really worth looking for the relationship of the color of the button with the number of placed orders?
We can make an assumption: the color of the button may affect the purchase decision. If this assumption is correct, then the results of A / B testing are fair, and if not, then the test results are useless. Asymmetry is in the face: we gain significantly if we are right and lose nothing if not.
Under conditions of uncertainty, when we cannot be sure whether event A is related to outcome B, and our testing costs are insignificant compared to possible lost profits, we will, of course, be right in putting our system to the test.
Can the same principle be applied to learning success? What if we scrupulously record all the decisions and events that entrepreneurs make and experience, then we will identify characteristic patterns, and then apply them in similar conditions?
To understand whether this idea will work, one must first decide: can the actions that ever preceded success be repeated in a similar situation and also lead to a successful outcome? Isn't this the same mechanics that allowed the “group of scientists” to accurately predict the outcome of football matches?
The actions of entrepreneurs who brought them success have changed the market. In the literal sense of the word, if someone began to regularly receive money that they had not received before, then someone began to receive it. The market favors one business model at the expense of others (not necessarily direct competitors). Yes, sometimes such a load on the market is distributed among many enterprises and it is difficult to feel the burden of someone else's success, but it is.
Therefore, choosing the key of profitable solutions to the castle, the distribution of goods in the market, if successful, we change the castle for our followers. But how then to explain the phenomenon of success of copycat, does this mean that every successful local analogue of a foreign service survived and brought profit to its founder only due to a random combination of circumstances?
Let us mentally isolate the minimum decision-making patterns from all the stories of the foundation and development of the company, imagine them in the form of a chain of genes that form the body of companies in a changing market environment. Having identified the same “genes” of successful companies and the same “genes” of failed companies, we will be tempted to draw a simple conclusion about the recipe for success and failure. But, if we search for “successful genes” among loser companies, we will definitely find them there. Yes, some loser companies will carry the “winners” genes. Therefore, simply re-creating “successful patterns” is not enough for success.
If the computability of a successful sequence of actions is impossible and it can only be solved by enumerating the available combinations, then how can one explain the success of copied businesses? Include failed copycat projects in the calculations and recognize the results as random?
Copycat-projects, this is the use of the already found whole and integral key of solutions, joint exploitation of the winning combination with the discoverer. Similarly in nature we see groups, populations, species. The evolution of living systems did not come up with a “successful gene” that could be pasted into any arbitrary genotype and, thereby, increase the success of the future organism.
So can it be worth applying the same principle that I proposed in the case of the color of the buttons? Can a pattern found in successful startups unambiguously have a loss-of-profit asymmetry? Is it possible to say that since in a similar situation, someone successful once made a decision, then having done the same thing, I will either gain or not suffer loss?
There is no doubt the assumption that decision-making can be fraught with both positive outcomes and negative ones. The decision affecting the future well-being of the project cannot be subjected to A / B testing, since we have only one project and cannot be divided into two, checked in the market and returned to its previous state. Does this mean that the patterns of other people's experience are useless for us?
By no means, we just need to look for not a philosopher's stone, but its opposite: something that turns gold into dust. Those. negative decisions immediately followed by failure. So, at the decision-making level, based on the common patterns of failed startups, we will make a statistically sound choice between death and survival. And this is an adaptive, profitable strategy.
Let me remind you how this is done: you create two pages whose difference will only lie in the color of the “checkout” button. Then, at random, half of the visitors to the site show page number 1, and the other half - page number 2. As a result, on one of the pages, users will click the checkout button more than the same number of other users on another page.
Does it follow that the color of the button affects the number of placed orders? Let's do a thought experiment. We will have two teams, five people each. One team will be in red t-shirts, the other in blue. Each member of the team will toss a coin and write down what happened: heads or tails. Let everyone toss a coin, say, three times. After, we calculate the number of “eagles” received for each of the teams.
We will see that one of the teams scored more “eagles” than the other. Can we conclude that the color of the T-shirt determined the winner? Should we wear a red (blue) t-shirt when we are going to earn more “eagles”?
We all know perfectly well that if our teams would toss a coin an infinite number of times, then the results for the teams would be the same. We know this, only thanks to the understanding of the model, since there was no immortal and at the same time a great lover to test everything in practice.
The conclusion suggests itself, but let's not rush into conclusions. Imagine that you received a message saying that a group of scientists has developed an excellent technology for predicting the results of football matches. And to prove its revolution, the message tells you the outcome of the match, which will take place tomorrow. Of course, the result is correct. A week later, you again receive a message from the same “group of scientists” who, to finally convince you to purchase their product, again send you the correct outcome of the next match.
If you are surprised by their insight, then it has a simple foundation. The first message with the text “Brazil will defeat Mexico” is sent to one group of people, and with the text “Mexico will defeat Brazil” to the second group of people. After the results of the Mexico – Brazil match become known, the second message with the next “prediction” is sent only to the group to which the correct prediction was previously randomly given.
This may be a good way of persuading, but obviously a bad way of predicting. Is there something similar in the way we are now trying to verify the success of one strategy over another? Is it really worth looking for the relationship of the color of the button with the number of placed orders?
We can make an assumption: the color of the button may affect the purchase decision. If this assumption is correct, then the results of A / B testing are fair, and if not, then the test results are useless. Asymmetry is in the face: we gain significantly if we are right and lose nothing if not.
Under conditions of uncertainty, when we cannot be sure whether event A is related to outcome B, and our testing costs are insignificant compared to possible lost profits, we will, of course, be right in putting our system to the test.
Can the same principle be applied to learning success? What if we scrupulously record all the decisions and events that entrepreneurs make and experience, then we will identify characteristic patterns, and then apply them in similar conditions?
To understand whether this idea will work, one must first decide: can the actions that ever preceded success be repeated in a similar situation and also lead to a successful outcome? Isn't this the same mechanics that allowed the “group of scientists” to accurately predict the outcome of football matches?
The actions of entrepreneurs who brought them success have changed the market. In the literal sense of the word, if someone began to regularly receive money that they had not received before, then someone began to receive it. The market favors one business model at the expense of others (not necessarily direct competitors). Yes, sometimes such a load on the market is distributed among many enterprises and it is difficult to feel the burden of someone else's success, but it is.
Therefore, choosing the key of profitable solutions to the castle, the distribution of goods in the market, if successful, we change the castle for our followers. But how then to explain the phenomenon of success of copycat, does this mean that every successful local analogue of a foreign service survived and brought profit to its founder only due to a random combination of circumstances?
Let us mentally isolate the minimum decision-making patterns from all the stories of the foundation and development of the company, imagine them in the form of a chain of genes that form the body of companies in a changing market environment. Having identified the same “genes” of successful companies and the same “genes” of failed companies, we will be tempted to draw a simple conclusion about the recipe for success and failure. But, if we search for “successful genes” among loser companies, we will definitely find them there. Yes, some loser companies will carry the “winners” genes. Therefore, simply re-creating “successful patterns” is not enough for success.
If the computability of a successful sequence of actions is impossible and it can only be solved by enumerating the available combinations, then how can one explain the success of copied businesses? Include failed copycat projects in the calculations and recognize the results as random?
Copycat-projects, this is the use of the already found whole and integral key of solutions, joint exploitation of the winning combination with the discoverer. Similarly in nature we see groups, populations, species. The evolution of living systems did not come up with a “successful gene” that could be pasted into any arbitrary genotype and, thereby, increase the success of the future organism.
So can it be worth applying the same principle that I proposed in the case of the color of the buttons? Can a pattern found in successful startups unambiguously have a loss-of-profit asymmetry? Is it possible to say that since in a similar situation, someone successful once made a decision, then having done the same thing, I will either gain or not suffer loss?
There is no doubt the assumption that decision-making can be fraught with both positive outcomes and negative ones. The decision affecting the future well-being of the project cannot be subjected to A / B testing, since we have only one project and cannot be divided into two, checked in the market and returned to its previous state. Does this mean that the patterns of other people's experience are useless for us?
By no means, we just need to look for not a philosopher's stone, but its opposite: something that turns gold into dust. Those. negative decisions immediately followed by failure. So, at the decision-making level, based on the common patterns of failed startups, we will make a statistically sound choice between death and survival. And this is an adaptive, profitable strategy.