July 24, 2015 at 21:05Continuous Success and why you should not forget about it when developing a project (using Drupal as an example)
Your goal is a reliable and capable product on Drupal (yes, by the way, on anything, but Drupal is closer to me in spirit, therefore I will concentrate examples on it)?
If so, then the long and thorny path of Continuous Integration, continuous inspection and continuous feedback is your way. As you might have guessed, the path is also continuous.
Guys, the last time I looked at the calendar, it was 2015, the summer was hot, but the coffee was already cold. And in this era of technological innovation, accessible information, tons of materials on the Internet about how to do it, and even more so on how to do it, it’s NOT NECESSARY, people still wildly dump their projects. And at the same time, they drive companies into an immeasurable hole in technical debt.
Nevertheless, I constantly get into the hands of projects in such a state that it is truly impossible to not believe in God or the devil, in this particular case. Such a number of defects somewhere there, "deep down", is simply unimaginable. It’s not that I was against it, thanks to such a campaign I am always equipped with work and good material compensation. I write all this a) simply out of interest: I really want to know your opinion, and b) you look, I will help someone. And I don’t worry much about reducing the amount of work, I don’t think that in India (namely, the lion's share of everything that needs to be fixed and redone comes from there) they read Habr. Sometimes it seems to me that the local coders do not read anything at all, but simply randomly drum on the keys until at least something somehow works. Well, okay, the conversation is not about them.
Of course, I do not claim that continuous integration or its elements is the quintessence of a perfect deployment, but practice is good. Damn good.
According to SEI, cold fixes cost $ 2.656 for each line of code (I don’t give a link to the proof, everything is googled easily). And there are usually a great many such lines, since defects sit deep, look far away and usually directly affect both the immediate functionality and technical / business requirements. Whether it’s the case when the defect is noticed and exterminated immediately upon appearance, when a few simple manipulations, sleight of hand and (sometimes) a little fraud are enough. This means less code required, significantly reduced time and functionality that meets or even exceeds the requirements both in stability and in further operation and maintenance.
As everyone is well aware, managers and programmers are different from each other. The latter write code with their strained pens and know it by heart. So the manager can assume the stability of a product to launch based on the information received (true or distorted), while the programmer determines the absence of defects in the code, thereby eliminating doubts about the readiness of a particular build.
The NI (continuous integration) eliminates doubts and gives access to a certain level of proactivity. Problems that may occur in any version (git branches) are monitored on an ongoing basis, thereby adding the best QA elements to the development process.
Everything that I write here will have absolutely no meaning without concrete examples. Yes, and we are not bargaining for the lamb leg in the market, so that you take my word for freshness.
Key points of continuous integration:
Continuous inspection is an important component of NI and its calling in the following: this process is obliged to show and visualize the entire development process, so that it is clear to those big uncles at the helm who have the key to the apartment where the money is. That is, the inspection is an important and unshakable part, and it can be divided into the following elements:
1. Code analysis and report
2. Trends
3. Collected information that tells about all the defects that have occurred in recent changes
4. Direct focus on the main ones (for business) elements
5. Absolute control over all possible problems and defects
Nokia - connecting people! This slogan is famous and applicable in a much larger number of cases than the marketers of the manufacturer of the unshakable hammers and fly swatter originally suggested. In our particular case, continuous feedback takes on the role of the Finnish tool and builds bridges between everyone involved in the creation of the product.
Why is it needed? Well, well, cultural communication is the basis of a healthy society, and a customer who is constantly up to date with everything is a satisfied customer (who does not get into the process with unnecessary, stupid questions). The main thing, as elsewhere, is to do everything right! Follow the simple rule: “The right people. At the right time. In the right form. ” How exactly to convey information - this is not even a question for a century in which there is:
Well, now let's get to the point. Here is a small case for you to consider.
Project: an online news portal that contains blog functionality, shows video clips (no, not what you thought, fii on you!) And represents a whole community. Technologies that feed our portal:
For the sake of conspiracy and because of the pile of signed papers, all the names have been changed, and the similarities with reality are purely random (well, or not quite random, heh heh). Let's go back to our sheep. Here is the picture that initially appeared before my clear eyes:
Don’t worry, I know this is difficult, but try to overcome the desire to sprinkle the monitor with holy water, everything is ready.
So, the conclusion I came to:
Architecture is clearly crooked, which led to the creation of a series of botneks, did not allow efficient distribution within the system and became a huge headache in the content phase. In general, everything is not very repairable. The code does not meet the standards, and those debt is as much as one and a half (one and a half) years!
What was undertaken:
What happened?
Judge for yourself, as for me - sweetie. Compared to what it was, at least. Anyway, work at 5 with a plus, because:
You yourself saw the result of good work with the help of NI. Why is the methodology still so poorly distributed among those who are developing so many external projects? It just destroys outsourcing as an industry and undermines trust in normal specialists. And yes, I know that you can eat with a spoon or a fork, and there are also other solutions. But why reinvent the wheel? So tell me, do you use NI, if so, how, if not, why?
If so, then the long and thorny path of Continuous Integration, continuous inspection and continuous feedback is your way. As you might have guessed, the path is also continuous.
Why am I doing this? (Disclaimer)
Guys, the last time I looked at the calendar, it was 2015, the summer was hot, but the coffee was already cold. And in this era of technological innovation, accessible information, tons of materials on the Internet about how to do it, and even more so on how to do it, it’s NOT NECESSARY, people still wildly dump their projects. And at the same time, they drive companies into an immeasurable hole in technical debt.
Nevertheless, I constantly get into the hands of projects in such a state that it is truly impossible to not believe in God or the devil, in this particular case. Such a number of defects somewhere there, "deep down", is simply unimaginable. It’s not that I was against it, thanks to such a campaign I am always equipped with work and good material compensation. I write all this a) simply out of interest: I really want to know your opinion, and b) you look, I will help someone. And I don’t worry much about reducing the amount of work, I don’t think that in India (namely, the lion's share of everything that needs to be fixed and redone comes from there) they read Habr. Sometimes it seems to me that the local coders do not read anything at all, but simply randomly drum on the keys until at least something somehow works. Well, okay, the conversation is not about them.
Of course, I do not claim that continuous integration or its elements is the quintessence of a perfect deployment, but practice is good. Damn good.
Simple math
According to SEI, cold fixes cost $ 2.656 for each line of code (I don’t give a link to the proof, everything is googled easily). And there are usually a great many such lines, since defects sit deep, look far away and usually directly affect both the immediate functionality and technical / business requirements. Whether it’s the case when the defect is noticed and exterminated immediately upon appearance, when a few simple manipulations, sleight of hand and (sometimes) a little fraud are enough. This means less code required, significantly reduced time and functionality that meets or even exceeds the requirements both in stability and in further operation and maintenance.
So why is continuous (integration + inspection + feedback) = continued success?
As everyone is well aware, managers and programmers are different from each other. The latter write code with their strained pens and know it by heart. So the manager can assume the stability of a product to launch based on the information received (true or distorted), while the programmer determines the absence of defects in the code, thereby eliminating doubts about the readiness of a particular build.
The NI (continuous integration) eliminates doubts and gives access to a certain level of proactivity. Problems that may occur in any version (git branches) are monitored on an ongoing basis, thereby adding the best QA elements to the development process.
How to integrate correctly continuously?
Everything that I write here will have absolutely no meaning without concrete examples. Yes, and we are not bargaining for the lamb leg in the market, so that you take my word for freshness.
Key points of continuous integration:
- Often (at least 1 time per day) the code is entered into the repository
- Automated tests are written
- Private builds are run (the process consists of assembling the source code, which is stored in the developer repository, Developer repository)
- Broken code is not entered into storage
- Broken builds are repaired
- All checks and tests are verified, as well as the fact that the code from the repository of broken code is not used is verified
Continuous inspection
Continuous inspection is an important component of NI and its calling in the following: this process is obliged to show and visualize the entire development process, so that it is clear to those big uncles at the helm who have the key to the apartment where the money is. That is, the inspection is an important and unshakable part, and it can be divided into the following elements:
1. Code analysis and report
2. Trends
3. Collected information that tells about all the defects that have occurred in recent changes
4. Direct focus on the main ones (for business) elements
5. Absolute control over all possible problems and defects
Continuous feedback
Nokia - connecting people! This slogan is famous and applicable in a much larger number of cases than the marketers of the manufacturer of the unshakable hammers and fly swatter originally suggested. In our particular case, continuous feedback takes on the role of the Finnish tool and builds bridges between everyone involved in the creation of the product.
Why is it needed? Well, well, cultural communication is the basis of a healthy society, and a customer who is constantly up to date with everything is a satisfied customer (who does not get into the process with unnecessary, stupid questions). The main thing, as elsewhere, is to do everything right! Follow the simple rule: “The right people. At the right time. In the right form. ” How exactly to convey information - this is not even a question for a century in which there is:
- SMS
- Special browser plugins
- Yes, even sirens or others, specially designed sound notifications for this matter
Well, now let's get to the point. Here is a small case for you to consider.
Case study
Project: an online news portal that contains blog functionality, shows video clips (no, not what you thought, fii on you!) And represents a whole community. Technologies that feed our portal:
- Drupal
- MySQL (cluster)
- Mongodb
- Javascript
For the sake of conspiracy and because of the pile of signed papers, all the names have been changed, and the similarities with reality are purely random (well, or not quite random, heh heh). Let's go back to our sheep. Here is the picture that initially appeared before my clear eyes:
Don’t worry, I know this is difficult, but try to overcome the desire to sprinkle the monitor with holy water, everything is ready.
So, the conclusion I came to:
Architecture is clearly crooked, which led to the creation of a series of botneks, did not allow efficient distribution within the system and became a huge headache in the content phase. In general, everything is not very repairable. The code does not meet the standards, and those debt is as much as one and a half (one and a half) years!
What was undertaken:
- Restructuring of architecture
- Code Restructuring
- Platform hosting
- Powerful (very) external caching
What happened?
Judge for yourself, as for me - sweetie. Compared to what it was, at least. Anyway, work at 5 with a plus, because:
- Database with 10,000,000 records
- 10,000 messages per second
- The site has repeatedly shone in the main page of the issuance on Yahoo
- 10 + K users per hour
Conclusion
You yourself saw the result of good work with the help of NI. Why is the methodology still so poorly distributed among those who are developing so many external projects? It just destroys outsourcing as an industry and undermines trust in normal specialists. And yes, I know that you can eat with a spoon or a fork, and there are also other solutions. But why reinvent the wheel? So tell me, do you use NI, if so, how, if not, why?