"Mamkin is an architect." Part 1: The ups and downs of software engineering
Good day, dear readers of Habr. I present to your attention a series of articles “Mamkin Architect”. This series of articles is inspired by Eric Evans' book, Object Oriented Design (DDD). Structuring complex software systems. " In it, I will try to reflect my personal opinion regarding the construction of flexible architectures. Along the way, I will explain how this will be useful to managers and help developers to strengthen the basic concepts.
This article will talk about the "creative rise", "creative decline" and the poker concept of "tilt." The latter perfectly reflects the state of the developer in various states of the subject model in the context of software architecture. Opus come in handy:
Creative growth is a psychological concept, primarily associated with a qualitative leap in the program. In simple terms, this is a wave-like feeling of satisfaction from the work done. A quantum leap is achieved through refactoring. I will explain to non-technical specialists what exactly the developers mean by the definition of "need to refactor."
First, we quote from Martin Fowler’s book “Refactoring. Improving Existing Code ”:
Difficult? Then we break refactoring into three main subspecies, namely:
As you might guess, one type of refactoring comes from another according to the principle of communicating vessels. But what happens when the last vessel is filled? One can imagine some fundamental shifts of the system as a whole, for example, when a liquid spills. In fact, the overflow of the last vessel is an impetus for in-depth refactoring. The conceptual model is changing, the architecture of the project is not. At this point, the software product begins to have serious problems. The system model no longer corresponds to the programmatic expression of the architecture. Many companies try to postpone this moment, periodically applying refactoring of a higher level. The reason for the tactics is simple: in-depth refactoring will simply stop the plant.
Using refactoring of a higher level, we can substitute a bucket under imaginary vessels or wrap them in a bag. If you take all these “gags” and multiply by the time and classification of specialists, you can get an uncontrolled system. The latter is confined to several members of the development team. Knowing these subtleties, you can even arrange sabotage without fear of dismissal. Several initiates understand how the system works with all of these packages, chewing gum, launchers, and more. Undoubtedly, a boiler in hell is ready for such people, but we will return to this.
It is regrettable to admit that sooner or later the critical mass in the last vessel will reach the ultimate strength of the vessel itself, causing it to burst. At such times, refactoring usually doesn’t end. It’s easier and cheaper to rewrite everything from scratch. Therefore, we will return to a more optimistic model.
Creative growth is preceded by a qualitative leap in the software product. Before a qualitative leap in a software product, in-depth refactoring, etc., up to nano refactoring, takes place.
Participating in one of the projects, we spent about two months on the refactoring of the system for creating meetings between people. It was superficial and only eliminated the dire consequences of outsourcing. Having made a decision on the readiness of the system for further development, we encountered a number of problems. I will exaggerate the complicated algorithm of the recommendation system a little, but the meaning will remain the same.
In fact, the recommendation system was supposed to offer users for hiking in the woods, walks in drinking establishments and more. It is worth noting that I am not an expert in this field. There was a task to improve the system by adding additional criteria. Since the recommendation system was part of the domain model, it turned out to quickly find the right specialist. For a couple of meetings, he helped to find out the intricacies of the system and there were no unresolved issues.
At first, we were optimistic. For more than three months, we managed to work out the architecture of the software product. Then it was “synchronized” with the subject model.
Clarifying the rules for the selection, we made changes that solve problems in the current mode. But with the addition of each new selection criterion, the complexity of the architecture grew. Quickly reach stable and reliable functionality did not work.
Even more we began to worry about errors in rounding off calculations. This could identify users in groups with which they are in no way associated. Of course, these errors were insignificant from one point of view. But from the perspective of the accuracy of the “recommendation system” they raised certain questions. At this point, we began to suspect that these inaccuracies are not just cosmetic problems, but symptoms of fundamental architecture problems.
Suddenly, an understanding of the problem and its solution descended on us. In our model, users and groups were connected in a way that did not correspond to the real state of affairs. Such a discovery had a wide resonance. We made an appointment with domain experts, with whose support we drew a new model on the board. It is worth noting that the details of this have not yet crystallized, but we already knew which new feature is critical for the model. Defining a user group could occur regardless of their preferences. It's funny, but in the framework of this software product - rightly. Having made a similar conclusion, we proceeded to go through various scenarios of the behavior of this model and its vulnerability.
Now the groups were formed in shares, we introduced the concept of "shares" in our common language. The need for the concept of “group” has disappeared, since the formation has already occurred according to the shares of the user's interests. We began to cultivate a common language of terms, which made it easier for us to communicate in meetings with domain specialists. Everyone clearly understood the essence of the conversation, because we operated on the terms of the subject model. And she perfectly laid down on the architecture of the application.
Do you think that at this moment we felt great? Nearly. Yes, the hypothetical implementation of the domain model fits perfectly into the new architecture. But for the project, tight time frames were set, which we could hardly fit into. The main feeling that I personally experienced at that time was fear .
The refactoring paradigm is the movement in small steps and the need to constantly maintain the software product in working condition. But refactoring our code will require deep refactoring. The software product will have to not only be temporarily incapacitated, but also postpone its release. We will also lose working intermediate states. The history of the product will be divided into “before” and “after”. Our team saw small improvements that could lead to certain positive results. But this did not bring us closer to the conceptual integrity of the system. After a couple of technical meetings, a sequence of steps was identified that could prepare the modules for the transition to a new conceptual model. But they could hardly keep their performance. Is it worth saying that in severe heredity there were no automatic tests? We could not guarantee backward compatibility of the modules corresponding to the new and old object models. Failures and errors would be unambiguous. Along the way, new efforts were required to introduce this model, and we were already exhausted with mouse fuss.
Slowly, I began to realize that my attitude towards the project was becoming mediocre. It has become difficult for me to reason in the framework of the old model. It later turned out that I was in a state of tilt. This term was introduced into my everyday life by a friend who actively plays poker and takes prizes in tournaments. He explained to me the following: “During tilt, you are no longer guided by reason, but by emotions. This is simply unacceptable for a poker player. " Drawing an analogy, I realized how much I show extra emotions about global alterations.
It was at this moment that the development team met with the leader, which I will never forget. Our leader was a smart and confident person. He asked us some questions. I give an example dialogue in which P is the leader, K is the team, and T is the team leader.
R: How much will the new model be introduced to return to the current functionality?
K .: About six weeks.
R: Is it possible to solve the problem without such global changes?
K .: It is possible, but there are no guarantees.
R: Is it possible to release the next version of the program if you do not implement these changes now?
K .: Without the introduction of changes, the forward movement will slow down significantly and make new adjustments will be increasingly difficult. The project will stop moving after installing the entire code base.
R: Are you personally sure that redoing everything is right?
T .: We know that now the program is loose and its performance is hanging by a thread. If necessary, we can handle the old fashioned way, because we ourselves are tired. But it’s better to say yes, insisting on a simpler solution that is better suited to the project’s goal. If we talk about the prospects for the development of the project, then introducing a new model is less risky.
Then the leader approved our actions and noticed that he would block us from all attacks from the customer. We rolled up our sleeves and finished everything in five weeks. There was a disproportionate amount of work, but we became a real family. The implementation went quite smoothly, perhaps our modular strategy affected.
At this moment, we felt a real lift. My nervous breakdown was postponed indefinitely. The implementation became more meaningful and complete, there were no problems with the recommendation system. With everything else, too. All modules worked in the same architecture, reflecting the subject model. We released the first version and were very warmly received by users. We stood firmly on our feet, because subsequent versions were no longer a problem. The development of the model was accompanied by the development of a single language. By the third version of the program, everyone actively applied the terms “our” language. Marketers have used it extensively in communication with potential investors, explaining the capabilities of the program.
On my way, there were projects whose leaders lacked the courage to "step into the unknown." The motives were different, starting with distrust and ending with “put off for later.” But, no matter how much the rope curls, the critical mass will increase. Tilt comes with it, followed by an unsupported program. These are signs of global shifts that are not always pleasant and painless.
Advising something is very difficult, because it all depends on the project. Do not try to artificially cause a quantum leap, this will only freeze the work. The jump potential usually arises after the refactoring cascade. Then comes the insight, a new vision of the model and its display in software architecture. Most of the time is occupied by small changes, preparation for a breakthrough, and a set of critical mass.
Preparing the way for the leap, focus on acquiring knowledge about the subject model and cultivating a single language. “Feel for” the concepts of the subject area and express them in the model in the most explicit form. Improve the architecture of the program, making it more flexible. Distill the knowledge of the model, emphasize the most important and dispute. Improve your understanding and visibility. These factors contribute to a quantum leap. A quantum leap has a beneficial effect on all branches of software development. Developers become more confident in their program, managers sensibly assess the tasks. Peace, friendship chewing gum!
This article will talk about the "creative rise", "creative decline" and the poker concept of "tilt." The latter perfectly reflects the state of the developer in various states of the subject model in the context of software architecture. Opus come in handy:
- Managers solving issues from developers like “let's bury and redo everything”.
- Applied developers who will be interested in looking at the encapsulation armor of a carefully constructed (or house) object model in its conceptual form.
- Architects or system designers will be interested in my experience of implementing, supporting and retaining a subject model in conceptual contours.
So what is creative upsurge?
Creative growth is a psychological concept, primarily associated with a qualitative leap in the program. In simple terms, this is a wave-like feeling of satisfaction from the work done. A quantum leap is achieved through refactoring. I will explain to non-technical specialists what exactly the developers mean by the definition of "need to refactor."
First, we quote from Martin Fowler’s book “Refactoring. Improving Existing Code ”:
“Refactoring is a process of changing a software system in which the external behavior of the code does not change, but its internal structure improves. This is a way to systematically bring the code in order, in which the chances of new errors are minimal. In fact, when refactoring code, you improve its design after it is written. ”
Difficult? Then we break refactoring into three main subspecies, namely:
- Nano refactoring is ongoing. The most common form that is not used by all developers. But there is a fairly large percentage of people who are able to change the program at a very high level. Most often, nano refactoring is aimed at changing methods / functions that are not of great value to the subject model. Example: dividing a large method into smaller functions without side effects.
- Micro refactoring aimed at changing logical structures within the system object. It is more often used with the advent of new knowledge about the subject model, but the motivation is sometimes deeper than just creating readable code. The general meaning is as follows: to fragment the process of moving to an in-depth, meaning-loaded model at the stage. Such refactoring is convenient because it does not “freeze” the development process. Its main goal is to help the developer understand not only the principle of operation of a particular object, but also why the object does what it does. At the same time, this understanding is also awakening among specialists in the applied field, generating a single language in the team.
- In-depth refactoring is the most complex, aimed at radical changes in the subject model. It is motivated by a deep understanding of the subject model. Its direction is the most dangerous, since it has a radical impact on the entire software product. An example from real life is a regular car, the motor of which was changed to a rocket engine. You can’t call such a technique a simple machine, and it will not fulfill its basic functions.
As you might guess, one type of refactoring comes from another according to the principle of communicating vessels. But what happens when the last vessel is filled? One can imagine some fundamental shifts of the system as a whole, for example, when a liquid spills. In fact, the overflow of the last vessel is an impetus for in-depth refactoring. The conceptual model is changing, the architecture of the project is not. At this point, the software product begins to have serious problems. The system model no longer corresponds to the programmatic expression of the architecture. Many companies try to postpone this moment, periodically applying refactoring of a higher level. The reason for the tactics is simple: in-depth refactoring will simply stop the plant.
Using refactoring of a higher level, we can substitute a bucket under imaginary vessels or wrap them in a bag. If you take all these “gags” and multiply by the time and classification of specialists, you can get an uncontrolled system. The latter is confined to several members of the development team. Knowing these subtleties, you can even arrange sabotage without fear of dismissal. Several initiates understand how the system works with all of these packages, chewing gum, launchers, and more. Undoubtedly, a boiler in hell is ready for such people, but we will return to this.
It is regrettable to admit that sooner or later the critical mass in the last vessel will reach the ultimate strength of the vessel itself, causing it to burst. At such times, refactoring usually doesn’t end. It’s easier and cheaper to rewrite everything from scratch. Therefore, we will return to a more optimistic model.
Creative growth is preceded by a qualitative leap in the software product. Before a qualitative leap in a software product, in-depth refactoring, etc., up to nano refactoring, takes place.
An example from a real project
Participating in one of the projects, we spent about two months on the refactoring of the system for creating meetings between people. It was superficial and only eliminated the dire consequences of outsourcing. Having made a decision on the readiness of the system for further development, we encountered a number of problems. I will exaggerate the complicated algorithm of the recommendation system a little, but the meaning will remain the same.
In fact, the recommendation system was supposed to offer users for hiking in the woods, walks in drinking establishments and more. It is worth noting that I am not an expert in this field. There was a task to improve the system by adding additional criteria. Since the recommendation system was part of the domain model, it turned out to quickly find the right specialist. For a couple of meetings, he helped to find out the intricacies of the system and there were no unresolved issues.
The model we developed was good, but ...
At first, we were optimistic. For more than three months, we managed to work out the architecture of the software product. Then it was “synchronized” with the subject model.
Clarifying the rules for the selection, we made changes that solve problems in the current mode. But with the addition of each new selection criterion, the complexity of the architecture grew. Quickly reach stable and reliable functionality did not work.
Even more we began to worry about errors in rounding off calculations. This could identify users in groups with which they are in no way associated. Of course, these errors were insignificant from one point of view. But from the perspective of the accuracy of the “recommendation system” they raised certain questions. At this point, we began to suspect that these inaccuracies are not just cosmetic problems, but symptoms of fundamental architecture problems.
Jump
Suddenly, an understanding of the problem and its solution descended on us. In our model, users and groups were connected in a way that did not correspond to the real state of affairs. Such a discovery had a wide resonance. We made an appointment with domain experts, with whose support we drew a new model on the board. It is worth noting that the details of this have not yet crystallized, but we already knew which new feature is critical for the model. Defining a user group could occur regardless of their preferences. It's funny, but in the framework of this software product - rightly. Having made a similar conclusion, we proceeded to go through various scenarios of the behavior of this model and its vulnerability.
Now the groups were formed in shares, we introduced the concept of "shares" in our common language. The need for the concept of “group” has disappeared, since the formation has already occurred according to the shares of the user's interests. We began to cultivate a common language of terms, which made it easier for us to communicate in meetings with domain specialists. Everyone clearly understood the essence of the conversation, because we operated on the terms of the subject model. And she perfectly laid down on the architecture of the application.
A sober look at things
Do you think that at this moment we felt great? Nearly. Yes, the hypothetical implementation of the domain model fits perfectly into the new architecture. But for the project, tight time frames were set, which we could hardly fit into. The main feeling that I personally experienced at that time was fear .
The refactoring paradigm is the movement in small steps and the need to constantly maintain the software product in working condition. But refactoring our code will require deep refactoring. The software product will have to not only be temporarily incapacitated, but also postpone its release. We will also lose working intermediate states. The history of the product will be divided into “before” and “after”. Our team saw small improvements that could lead to certain positive results. But this did not bring us closer to the conceptual integrity of the system. After a couple of technical meetings, a sequence of steps was identified that could prepare the modules for the transition to a new conceptual model. But they could hardly keep their performance. Is it worth saying that in severe heredity there were no automatic tests? We could not guarantee backward compatibility of the modules corresponding to the new and old object models. Failures and errors would be unambiguous. Along the way, new efforts were required to introduce this model, and we were already exhausted with mouse fuss.
Slowly, I began to realize that my attitude towards the project was becoming mediocre. It has become difficult for me to reason in the framework of the old model. It later turned out that I was in a state of tilt. This term was introduced into my everyday life by a friend who actively plays poker and takes prizes in tournaments. He explained to me the following: “During tilt, you are no longer guided by reason, but by emotions. This is simply unacceptable for a poker player. " Drawing an analogy, I realized how much I show extra emotions about global alterations.
It was at this moment that the development team met with the leader, which I will never forget. Our leader was a smart and confident person. He asked us some questions. I give an example dialogue in which P is the leader, K is the team, and T is the team leader.
R: How much will the new model be introduced to return to the current functionality?
K .: About six weeks.
R: Is it possible to solve the problem without such global changes?
K .: It is possible, but there are no guarantees.
R: Is it possible to release the next version of the program if you do not implement these changes now?
K .: Without the introduction of changes, the forward movement will slow down significantly and make new adjustments will be increasingly difficult. The project will stop moving after installing the entire code base.
R: Are you personally sure that redoing everything is right?
T .: We know that now the program is loose and its performance is hanging by a thread. If necessary, we can handle the old fashioned way, because we ourselves are tired. But it’s better to say yes, insisting on a simpler solution that is better suited to the project’s goal. If we talk about the prospects for the development of the project, then introducing a new model is less risky.
Then the leader approved our actions and noticed that he would block us from all attacks from the customer. We rolled up our sleeves and finished everything in five weeks. There was a disproportionate amount of work, but we became a real family. The implementation went quite smoothly, perhaps our modular strategy affected.
Retribution
At this moment, we felt a real lift. My nervous breakdown was postponed indefinitely. The implementation became more meaningful and complete, there were no problems with the recommendation system. With everything else, too. All modules worked in the same architecture, reflecting the subject model. We released the first version and were very warmly received by users. We stood firmly on our feet, because subsequent versions were no longer a problem. The development of the model was accompanied by the development of a single language. By the third version of the program, everyone actively applied the terms “our” language. Marketers have used it extensively in communication with potential investors, explaining the capabilities of the program.
but on the other hand
On my way, there were projects whose leaders lacked the courage to "step into the unknown." The motives were different, starting with distrust and ending with “put off for later.” But, no matter how much the rope curls, the critical mass will increase. Tilt comes with it, followed by an unsupported program. These are signs of global shifts that are not always pleasant and painless.
Finally
Advising something is very difficult, because it all depends on the project. Do not try to artificially cause a quantum leap, this will only freeze the work. The jump potential usually arises after the refactoring cascade. Then comes the insight, a new vision of the model and its display in software architecture. Most of the time is occupied by small changes, preparation for a breakthrough, and a set of critical mass.
Preparing the way for the leap, focus on acquiring knowledge about the subject model and cultivating a single language. “Feel for” the concepts of the subject area and express them in the model in the most explicit form. Improve the architecture of the program, making it more flexible. Distill the knowledge of the model, emphasize the most important and dispute. Improve your understanding and visibility. These factors contribute to a quantum leap. A quantum leap has a beneficial effect on all branches of software development. Developers become more confident in their program, managers sensibly assess the tasks. Peace, friendship chewing gum!