Do not try to predict tomorrow's problems.
- Transfer
Well, or start doing it right.
If I were asked to point out one specific problem that ruined the most software products, then I would definitely call the developers craving for the foresight of the distant future. This can be expressed in many ways, but the general scheme is approximately as follows:
“We need to implement the solution {X}, even though there is a much simpler and more suitable solution now {Y}, because when {Z} occurs in the future, {X} will work much better than {Y}. "
At the same time, there is no exact information about the probability of occurrence of the {Z} event and cannot be.
Here are a couple of examples:
Recently, I wrote an article about imaginary problems - those whose solutions people entertain themselves, because they are more interesting to solve than real ones. This also includes these attempts to foresee the future. Even more, this is the favorite imaginary problem of most small start-up companies.
But let's not throw everything in one pile: trying to be prepared for the future can be useful if you approach it wisely. But very few people are smart enough - people are overwhelmed by fantasies, fears, emotions and other human feelings.
Every person sometimes fantasized about how it would be if he were someone else. Someone rich, famous, strong, endowed with power. To think about it is interesting enough and it happens somehow by itself, involuntarily. So you saw the photo on the cover of the magazine - and thought, what would I do on the site of this celebrity? Oh, here I would spend money on it, and if then I would do this. And this. And if I could still fly and have super powers! Yes, it would be great!
Software developers are also people, and they too are amenable to fantasy. This means that Facebook built its platform on such and such technologies and it is scaled to a billion users ... Well, we are not worse, and the technologies are available, let's do everything well, with a reserve for a billion users (although there are a hundred of them). But Facebook's magic was not in the technology of scaling to a billion users. She was able to give people the right product at the right time and place. Software capable of marshalling a billion users was a secondary and less important part of the company. It was created only when it became necessary and only the way it was needed.
The medal has two sides:
a) Achieving growth is a more difficult task than maintaining the achieved scale
b)Most exceptionally skilled and talented programmers are working on products that need good scaling.
Item “a” is easy to understand. Think about it - out of all software companies that have ever been created, only, probably, about 0.05% reached the level of millions of users and billions in profits. The rest had collapsed before or were rewarded with less.
So, most fantasies about the necessary software features in the future are usually reduced to trying to solve the problems of these 0.05% of companies. “We will have a team of 1000 developers, 10 million users and a dozen large corporate clients with their own complex requirements, and then we will need ...” No, you do not need. With a probability of 99.95%.
But to say NO to such attractive ideas is hard, because it destroys faith in those very fractions of a percent of the probability of success. We have to stop representing ourselves as the owner of the new Amazon and return to today's problems. And today you have 50 users, of which 30 are family and friends. Yes, awareness of the current state of affairs can demotivate.
Item balso does not help cope with obsession. It is clear that the best programmers work in top companies. Either because they created them because of their talent, or because top companies are able to hire the best programmers. The Pareto principle works here against us: better programmers write books, make reports, design the best systems. Each of us heard from them these fascinating stories about fault-tolerant clusters distributed over thousands of nodes, processing petabytes of data using software optimized to some incredible performance figures. But most of us do not need to think about how to build such a cluster in our company here and now. He just does not need.
So, to close your eyes and present your company in 5 years big won't help? Do you just need to stop thinking about the future?
Of course not. Thinking about the future is helpful. And designing software with the future is also useful, but it should be done correctly.
Better to do less, but fine. Very few products actually meet the needs of their customers. Such that you make A and 90% of your users need exactly A - never will be. 90% of your potential users need some kind of B , and your A is just the closest alternative to B , and no one does or sells B himself , so some of the buyers will be satisfied with a tit in their hand.
What is good in this scenario? Once you get customers, you can still try to understand what they really needed and in time to do just that. Well, or a little better alternative to this. The user base helps you explore the market, find niches on it and fill them. As soon as you grope for this niche, you start working in it - here your growth begins.
And this approach is really productive. You are implementing something small, but working well, giving it to users - and then listen to their thoughts about your product. You no longer guess, do not solve imaginary problems, do not add unnecessary complexity. You adapt, add something, remove something. And it creates your unique product.
And on this path - the less your codebase was originally, the easier it is to adapt to something new.
Poor software is more viable. It is possible to create it in a reasonable time and for reasonable money. It often does everything or almost everything that is needed. It, being imperfect, by definition leaves room for its development and freedom of maneuver.
It is important to remember that the world around you is not static. The problems that will arise before you a few years later may well be easily solved with the help of technologies available after several years. Many people design something, not only disregarding future opportunities, but generally relying only on tools that have been around for decades. They limit themselves not even today, but yesterday.
Let me tell you about one specific example: the design of distributed systems in terms of being ready for any growth. One of the common fears leading to the creation of such systems is the fear that at some point one of your servers will not be able to serve all of your users. And it really happens. Sometimes. But not in small companies, not in startups. In addition, most people writing software in 2018, for some reason, are confident that it will work on servers created in 2005. Computers are getting better every year and good servers can be rented not so expensive.
Let me describe such an initial start-up "real" server:
Yes, I’m willing to bet that half of the existing distributed systems in the world will run on this server completely, with all its components and dependencies, servicing all their existing user base in a regular mode. And this is not the coolest server today. This can be taken for a price of $ 800 to $ 1,300 per month (depending on where to get it). You can rent a dozen of these for the salary of one qualified engineer in London.
What else is good about this server is that the price of its rent in 2 years will fall by 2 times.
Computers are developing, developing very linearly and predictably, and will continue to evolve so, according to forecasts, sometime before the end of the 2020s. It is hard to think further, but it is unlikely that mankind will come up with something new. And people still remember the iron of the beginning of the century and are afraid that they will not have enough of it to service a couple of thousand requests per day.
But we are talking about iron. And think about all the software that appears and develops around. Few people 20 years ago seriously thought about voice control with something. And look at today's world - “OK, Google,” “Hello, Alex,” “What is the weather now, Siri?” The one who started writing voice frontend in the 2016th year - as soon as he managed by 2018th.
What to start writing in the 2018th? Oh, if I knew :) This is something that has already appeared on the horizon, but has not yet become so big as to overshadow the sun. Look around, maybe notice something like that?
Progress in software is incredible. Absolutely unnoticed with the advent of WASM browsers have become universal virtual machines. After 2 years, you can build a universal, complex and high-performance application by compiling it under exactly one platform: web assembly. And it will run everywhere.
But people still live around 2012. They use Babel, although 99% of users have at least one browser with ES6 support.
New programming languages appear constantly. And some of them are quite good. Only in the last 8 years, we got Go, Rust, Scale and D - everyone found their niche. In the next 2 years, I expect to see how Julia will contribute to scientific programming. And this is just what I personally care about and what I follow. The total amount of technology and knowledge strikes incredibly.
Being inspired by the future is relatively easy. But, frankly, in addition to the linear progress of productivity growth - it is hard to imagine what will happen in 2 or 5 years. Some ideas are floating in the air, teams are working on different software and hardware projects, but what will “shoot” out of this?
Still, if you want to prepare your software for the future, you need to begin to realize the present. What is good in the present is already there, it is observable, measurable. And it will still hold on for some time. Making your software relevant at least today is a good idea. You will not be ready for the realities of 2020, using the approaches of 2000. But software written with approaches that are relevant for 2018, can work quite well in 2020.
So do not deny yourself the pleasure to develop software with a reserve for the future. Just do it more correctly. Estimate not only the development of your future product, but also the development of its surrounding ecosystem. Everything that can be designed flexibly should be designed flexibly. This will give you the opportunity to maneuver at the moment when you know which way this maneuver should be completed. And it will save you from the time spent on preparing for what will never happen.
If I were asked to point out one specific problem that ruined the most software products, then I would definitely call the developers craving for the foresight of the distant future. This can be expressed in many ways, but the general scheme is approximately as follows:
“We need to implement the solution {X}, even though there is a much simpler and more suitable solution now {Y}, because when {Z} occurs in the future, {X} will work much better than {Y}. "
At the same time, there is no exact information about the probability of occurrence of the {Z} event and cannot be.
Here are a couple of examples:
- We need to use kubernetes and docker! Yes, one server works fine with the current load and is easy to configure and maintain, but when we need a dozen servers, it will be easier to deploy them with kubernetes and docker.
- We need a distributed data processing architecture! Yes, as long as one mediocre PC handles everything, but when we have an industrial-level solution and customers will require up to five nines in SLA - we will be ready for this.
- We need to hire a development team and create a website from scratch, despite the fact that it would be much faster to deploy something based on wordpress, because when we have 100 times more clients than now, wordpress will not be so convenient.
- We need to use inheritance instead of composition, because after 5 years the code base will grow so that without it there will be no way.
- We need to write this code in C ++, despite the fact that it will be much faster in Python, because after years it will process terabytes of data and Python may not be able to cope here.
Recently, I wrote an article about imaginary problems - those whose solutions people entertain themselves, because they are more interesting to solve than real ones. This also includes these attempts to foresee the future. Even more, this is the favorite imaginary problem of most small start-up companies.
But let's not throw everything in one pile: trying to be prepared for the future can be useful if you approach it wisely. But very few people are smart enough - people are overwhelmed by fantasies, fears, emotions and other human feelings.
Achieving success is more difficult than living with success already achieved.
Every person sometimes fantasized about how it would be if he were someone else. Someone rich, famous, strong, endowed with power. To think about it is interesting enough and it happens somehow by itself, involuntarily. So you saw the photo on the cover of the magazine - and thought, what would I do on the site of this celebrity? Oh, here I would spend money on it, and if then I would do this. And this. And if I could still fly and have super powers! Yes, it would be great!
Software developers are also people, and they too are amenable to fantasy. This means that Facebook built its platform on such and such technologies and it is scaled to a billion users ... Well, we are not worse, and the technologies are available, let's do everything well, with a reserve for a billion users (although there are a hundred of them). But Facebook's magic was not in the technology of scaling to a billion users. She was able to give people the right product at the right time and place. Software capable of marshalling a billion users was a secondary and less important part of the company. It was created only when it became necessary and only the way it was needed.
The medal has two sides:
a) Achieving growth is a more difficult task than maintaining the achieved scale
b)Most exceptionally skilled and talented programmers are working on products that need good scaling.
Item “a” is easy to understand. Think about it - out of all software companies that have ever been created, only, probably, about 0.05% reached the level of millions of users and billions in profits. The rest had collapsed before or were rewarded with less.
So, most fantasies about the necessary software features in the future are usually reduced to trying to solve the problems of these 0.05% of companies. “We will have a team of 1000 developers, 10 million users and a dozen large corporate clients with their own complex requirements, and then we will need ...” No, you do not need. With a probability of 99.95%.
But to say NO to such attractive ideas is hard, because it destroys faith in those very fractions of a percent of the probability of success. We have to stop representing ourselves as the owner of the new Amazon and return to today's problems. And today you have 50 users, of which 30 are family and friends. Yes, awareness of the current state of affairs can demotivate.
Item balso does not help cope with obsession. It is clear that the best programmers work in top companies. Either because they created them because of their talent, or because top companies are able to hire the best programmers. The Pareto principle works here against us: better programmers write books, make reports, design the best systems. Each of us heard from them these fascinating stories about fault-tolerant clusters distributed over thousands of nodes, processing petabytes of data using software optimized to some incredible performance figures. But most of us do not need to think about how to build such a cluster in our company here and now. He just does not need.
So, to close your eyes and present your company in 5 years big won't help? Do you just need to stop thinking about the future?
Of course not. Thinking about the future is helpful. And designing software with the future is also useful, but it should be done correctly.
Design flexibly, implement imperfectly
Better to do less, but fine. Very few products actually meet the needs of their customers. Such that you make A and 90% of your users need exactly A - never will be. 90% of your potential users need some kind of B , and your A is just the closest alternative to B , and no one does or sells B himself , so some of the buyers will be satisfied with a tit in their hand.
What is good in this scenario? Once you get customers, you can still try to understand what they really needed and in time to do just that. Well, or a little better alternative to this. The user base helps you explore the market, find niches on it and fill them. As soon as you grope for this niche, you start working in it - here your growth begins.
And this approach is really productive. You are implementing something small, but working well, giving it to users - and then listen to their thoughts about your product. You no longer guess, do not solve imaginary problems, do not add unnecessary complexity. You adapt, add something, remove something. And it creates your unique product.
And on this path - the less your codebase was originally, the easier it is to adapt to something new.
“I hate the code and would like to see as little as possible in our product” - Jack DiederichIf you did something perfectly working - you did it wrong. You had to make too big sacrifices along the way. Perhaps it was a waste of time or money, perhaps you abandoned flexibility, maybe something else. The ideal is not achieved for free.
Poor software is more viable. It is possible to create it in a reasonable time and for reasonable money. It often does everything or almost everything that is needed. It, being imperfect, by definition leaves room for its development and freedom of maneuver.
Optimistic software architecture design - the future can be pleasantly surprising
It is important to remember that the world around you is not static. The problems that will arise before you a few years later may well be easily solved with the help of technologies available after several years. Many people design something, not only disregarding future opportunities, but generally relying only on tools that have been around for decades. They limit themselves not even today, but yesterday.
Let me tell you about one specific example: the design of distributed systems in terms of being ready for any growth. One of the common fears leading to the creation of such systems is the fear that at some point one of your servers will not be able to serve all of your users. And it really happens. Sometimes. But not in small companies, not in startups. In addition, most people writing software in 2018, for some reason, are confident that it will work on servers created in 2005. Computers are getting better every year and good servers can be rented not so expensive.
Let me describe such an initial start-up "real" server:
- Two Xeons E5–2680v4 (28cores & 56 threads, cloking at 2.4GHz to 3.3GHz)
- 512 Gigabytes of DDR4–2400 RAM
- 2 NVMe SSDs of 1.2TB each (~ 3GB / s read and ~ 1.5GB / s write each).
Yes, I’m willing to bet that half of the existing distributed systems in the world will run on this server completely, with all its components and dependencies, servicing all their existing user base in a regular mode. And this is not the coolest server today. This can be taken for a price of $ 800 to $ 1,300 per month (depending on where to get it). You can rent a dozen of these for the salary of one qualified engineer in London.
What else is good about this server is that the price of its rent in 2 years will fall by 2 times.
Computers are developing, developing very linearly and predictably, and will continue to evolve so, according to forecasts, sometime before the end of the 2020s. It is hard to think further, but it is unlikely that mankind will come up with something new. And people still remember the iron of the beginning of the century and are afraid that they will not have enough of it to service a couple of thousand requests per day.
But we are talking about iron. And think about all the software that appears and develops around. Few people 20 years ago seriously thought about voice control with something. And look at today's world - “OK, Google,” “Hello, Alex,” “What is the weather now, Siri?” The one who started writing voice frontend in the 2016th year - as soon as he managed by 2018th.
What to start writing in the 2018th? Oh, if I knew :) This is something that has already appeared on the horizon, but has not yet become so big as to overshadow the sun. Look around, maybe notice something like that?
Progress in software is incredible. Absolutely unnoticed with the advent of WASM browsers have become universal virtual machines. After 2 years, you can build a universal, complex and high-performance application by compiling it under exactly one platform: web assembly. And it will run everywhere.
But people still live around 2012. They use Babel, although 99% of users have at least one browser with ES6 support.
New programming languages appear constantly. And some of them are quite good. Only in the last 8 years, we got Go, Rust, Scale and D - everyone found their niche. In the next 2 years, I expect to see how Julia will contribute to scientific programming. And this is just what I personally care about and what I follow. The total amount of technology and knowledge strikes incredibly.
But I got distracted ...
Being inspired by the future is relatively easy. But, frankly, in addition to the linear progress of productivity growth - it is hard to imagine what will happen in 2 or 5 years. Some ideas are floating in the air, teams are working on different software and hardware projects, but what will “shoot” out of this?
Still, if you want to prepare your software for the future, you need to begin to realize the present. What is good in the present is already there, it is observable, measurable. And it will still hold on for some time. Making your software relevant at least today is a good idea. You will not be ready for the realities of 2020, using the approaches of 2000. But software written with approaches that are relevant for 2018, can work quite well in 2020.
So do not deny yourself the pleasure to develop software with a reserve for the future. Just do it more correctly. Estimate not only the development of your future product, but also the development of its surrounding ecosystem. Everything that can be designed flexibly should be designed flexibly. This will give you the opportunity to maneuver at the moment when you know which way this maneuver should be completed. And it will save you from the time spent on preparing for what will never happen.