Tom DeMarco: software engineering - an idea whose time has passed?
- Transfer
I often communicate with people on the topic of flexible software development methods, sometimes I write articles about this (for example, a recent article on the Haban about Kanban in IT ).
And I can say that the main argument that people give against these methods, which stops many even from thinking about Kanban, Scrum or XP, is the supposedly low level of control over the development of these methodologies.
At the same time, some perceive, as unprofessionalism, the argument that the level of control does not depend very much on the methodology, and indeed control in the field of software development is, by and large, a fiction.
For such people, I translated a new article by Tom Demarco, one of the founders of software engineering, a developer of software metrics, and co-author of a famous book“The human factor: successful projects and teams.”
This article is highly provocative and is now widely discussed in English blogs and it is strange that I have not yet seen its translations into Russian. But, despite the provocation, it has some very correct ideas that can change someone's idea of the importance and possibility of control over the development.
In general, read the translation of the article under the cut.
Tom DeMarco: Software Engineering - an idea whose time has passed?
More recently, the 40th anniversary of the NATO conference on Software Engineering, which took place in Garmisch, Germany, was held. It was there that the engineering discipline for software was first proposed.
Since my early books became part of this new discipline, now it seems like a great moment to revise, revise the position. My early book on metrics, Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall / Yourdon Press, 1982) , played a big role in how many prominent developers shared their work and planned their projects.
And now I’m wondering, were these my tips correct in the past, are they still correct, and do I still believe that metrics are necessary for any successful development project?
My answers: NO, NO and NO.
The book for me is a curious combination of generally true things written on each page, but together they create a common message that is incorrect.
It is as if the young author of the book never met metrics that he would not like. The book’s deeper meaning is that metrics are good and the more metrics the better.
Now we all understand that metrics cost money and time and should be used with cautious moderation.
In addition, software development is essentially different from science, such as physics, and metrics are much less accurate because of this. And they should be considered very skeptical, and not unconditionally.
Subordinate to control
The most quoted line of my book is the very first sentence:
“You cannot control what you cannot measure”
This sentence is true, but I find more and more that my use of this sentence is wrong. The proposal implies that control is an important aspect, perhaps the most important for any software development project.
But this is not so!
Many projects developed without any control, but still the result was excellent products - such as GoogleEarth or Wikipedia.
To understand the real role of control, you must understand the difference between two completely different types of projects :
Project A will ultimately cost $ 1 million and make a value of approximately $ 1.1 million.
“Project B” will ultimately cost about a million dollars and bring a value of more than 50 million. It
immediately becomes clear that control is really very important for project A, but not at all important for project B. This leads us to completely the strange conclusion that tight control is something that is very important for relatively useless projects and much less important for useful projects.
This suggests that the more you focus on control, the more likely you are working on a project that will ultimately bring very little profit.
I think that a much more important question is “What kind of devil are we doing so many projects that bring so little profit?”
Can I really say that it is normal to launch projects without control or with relatively little control? In practice!
I assume that first we need to choose projects where precise control will not be needed so much. Next, we need to reduce our expectations of how much we will control it, no matter how hard we try to do it.
Disturbing analogy
Imagine that you are trying to control the upbringing of a teenager. The very idea of controlling your child may seem a little disturbing, unpleasant to you. In addition, the stakes that you will succeed may not be high.
If you fail this task, completely fail, it can destroy someone’s life. So it is obvious that you will not loosen your grip completely and do not let the teenager go free-swimming.
So you, as a fencer, learning to hold your sword, as if it were a bird: too much and the bird would be damaged; too weak and she will fly away ...
Now apply the rule “You cannot control what you cannot measure” to the teenager. Most of the things that are really important - honor, dignity, discipline, personality, politeness under pressure, values, ethics, ingenuity, loyalty, humor, goodwill - they are not measurable.
You should just guide your child as well as you can without any help from any metrics. And this is difficult, but raising children is difficult.
You will have a few “measurements” in the form of school grades, and you are grateful for that. But you also know that a math grade for your child is a better indicator of achievement than a Spanish grade, as understanding of math is easier to measure.
And his assessment of behavior is likely to tell you a lot more about the teacher than about the child.
So how do you manage a project without control?
Well, you control people and control time and money. You tell the team leader, for example:
“I know the final date, and I'm not even going to tell you. One day I will come to you and say that the project will end in a week - you should be ready to finish all the work and do what you have with a finished product. Your task is to work incrementally, adding pieces to the project in order of importance, and doing integration, documentation and testing incrementally and continuously. ”
This may sound like a recommendation for flexible methods, but I'm too far from actually creating software to recommend it at the method level. Rather, I recommend a managerial approach - one that can lead the team towards agile methods, at least toward incremental ideas from the agile school.
So far, I have mainly discussed metrics from software engineering. What about the rest?
Little by little, I come to the conclusion that Software Engineering is an idea whose time has come and gone!
I still believe that it makes a lot of sense in software engineering. But that's not exactly what software engineering has come to mean. This term surrounds a specific set of disciplines, including a specific process, expertise and research, requirements development, metrics, engineering, quality control, strict planning and tracking, coding and documentation standards.
All this is necessary for the consistency of practices and for predictability. Consistency and predictability are still desirable, but they have never been the most important things.
Over the past 40 years, for example, we have tortured ourselves for not being able to complete projects on time and without exceeding the budget. But, as I noted earlier, a much more important goal is to create software that changes the world, or that transforms the company or how it conducts its business. We were rather much successful in the changes that often take place out of control.
Software development is and always will be something experimental. Yes, the creation of a program is not necessarily experimental, but the concept of a program is always.
And this is exactly the place where we should focus. And this is the place where we always had to focus before.
And I can say that the main argument that people give against these methods, which stops many even from thinking about Kanban, Scrum or XP, is the supposedly low level of control over the development of these methodologies.
At the same time, some perceive, as unprofessionalism, the argument that the level of control does not depend very much on the methodology, and indeed control in the field of software development is, by and large, a fiction.
For such people, I translated a new article by Tom Demarco, one of the founders of software engineering, a developer of software metrics, and co-author of a famous book“The human factor: successful projects and teams.”
This article is highly provocative and is now widely discussed in English blogs and it is strange that I have not yet seen its translations into Russian. But, despite the provocation, it has some very correct ideas that can change someone's idea of the importance and possibility of control over the development.
In general, read the translation of the article under the cut.
Tom DeMarco: Software Engineering - an idea whose time has passed?
More recently, the 40th anniversary of the NATO conference on Software Engineering, which took place in Garmisch, Germany, was held. It was there that the engineering discipline for software was first proposed.
Since my early books became part of this new discipline, now it seems like a great moment to revise, revise the position. My early book on metrics, Controlling Software Projects: Management, Measurement, and Estimation (Prentice Hall / Yourdon Press, 1982) , played a big role in how many prominent developers shared their work and planned their projects.
And now I’m wondering, were these my tips correct in the past, are they still correct, and do I still believe that metrics are necessary for any successful development project?
My answers: NO, NO and NO.
The book for me is a curious combination of generally true things written on each page, but together they create a common message that is incorrect.
It is as if the young author of the book never met metrics that he would not like. The book’s deeper meaning is that metrics are good and the more metrics the better.
Now we all understand that metrics cost money and time and should be used with cautious moderation.
In addition, software development is essentially different from science, such as physics, and metrics are much less accurate because of this. And they should be considered very skeptical, and not unconditionally.
Subordinate to control
The most quoted line of my book is the very first sentence:
“You cannot control what you cannot measure”
This sentence is true, but I find more and more that my use of this sentence is wrong. The proposal implies that control is an important aspect, perhaps the most important for any software development project.
But this is not so!
Many projects developed without any control, but still the result was excellent products - such as GoogleEarth or Wikipedia.
To understand the real role of control, you must understand the difference between two completely different types of projects :
Project A will ultimately cost $ 1 million and make a value of approximately $ 1.1 million.
“Project B” will ultimately cost about a million dollars and bring a value of more than 50 million. It
immediately becomes clear that control is really very important for project A, but not at all important for project B. This leads us to completely the strange conclusion that tight control is something that is very important for relatively useless projects and much less important for useful projects.
This suggests that the more you focus on control, the more likely you are working on a project that will ultimately bring very little profit.
I think that a much more important question is “What kind of devil are we doing so many projects that bring so little profit?”
Can I really say that it is normal to launch projects without control or with relatively little control? In practice!
I assume that first we need to choose projects where precise control will not be needed so much. Next, we need to reduce our expectations of how much we will control it, no matter how hard we try to do it.
Disturbing analogy
Imagine that you are trying to control the upbringing of a teenager. The very idea of controlling your child may seem a little disturbing, unpleasant to you. In addition, the stakes that you will succeed may not be high.
If you fail this task, completely fail, it can destroy someone’s life. So it is obvious that you will not loosen your grip completely and do not let the teenager go free-swimming.
So you, as a fencer, learning to hold your sword, as if it were a bird: too much and the bird would be damaged; too weak and she will fly away ...
Now apply the rule “You cannot control what you cannot measure” to the teenager. Most of the things that are really important - honor, dignity, discipline, personality, politeness under pressure, values, ethics, ingenuity, loyalty, humor, goodwill - they are not measurable.
You should just guide your child as well as you can without any help from any metrics. And this is difficult, but raising children is difficult.
You will have a few “measurements” in the form of school grades, and you are grateful for that. But you also know that a math grade for your child is a better indicator of achievement than a Spanish grade, as understanding of math is easier to measure.
And his assessment of behavior is likely to tell you a lot more about the teacher than about the child.
So how do you manage a project without control?
Well, you control people and control time and money. You tell the team leader, for example:
“I know the final date, and I'm not even going to tell you. One day I will come to you and say that the project will end in a week - you should be ready to finish all the work and do what you have with a finished product. Your task is to work incrementally, adding pieces to the project in order of importance, and doing integration, documentation and testing incrementally and continuously. ”
This may sound like a recommendation for flexible methods, but I'm too far from actually creating software to recommend it at the method level. Rather, I recommend a managerial approach - one that can lead the team towards agile methods, at least toward incremental ideas from the agile school.
So far, I have mainly discussed metrics from software engineering. What about the rest?
Little by little, I come to the conclusion that Software Engineering is an idea whose time has come and gone!
I still believe that it makes a lot of sense in software engineering. But that's not exactly what software engineering has come to mean. This term surrounds a specific set of disciplines, including a specific process, expertise and research, requirements development, metrics, engineering, quality control, strict planning and tracking, coding and documentation standards.
All this is necessary for the consistency of practices and for predictability. Consistency and predictability are still desirable, but they have never been the most important things.
Over the past 40 years, for example, we have tortured ourselves for not being able to complete projects on time and without exceeding the budget. But, as I noted earlier, a much more important goal is to create software that changes the world, or that transforms the company or how it conducts its business. We were rather much successful in the changes that often take place out of control.
Software development is and always will be something experimental. Yes, the creation of a program is not necessarily experimental, but the concept of a program is always.
And this is exactly the place where we should focus. And this is the place where we always had to focus before.