Evaluation of software projects or is the whole sum equal to the sum of the terms?

    The next cross-post (oh, me, beech!) From my main blog.

    In this article, I am going to talk about things that you all have long known. Now I suggest thinking about them.

    If you are not new to the software industry, then you are aware that from time to time any manager is in a phase when he is suddenly interested in estimates of time, work ... And usually not just anyhow, but by some fashionable technique. You are prompted to inform in a week how long it will take for bugs that have not yet been found and that feature whose design has not yet been completed, not to mention the fact that the planned work should certainly be broken into pieces up to an hour, if it is not possible every minute. ..

    How is project time typically estimated? Everyone knows that it’s practically impossible to evaluate a large piece, so it is divided into smaller pieces, and then these small pieces are put together, for example, in some Gantt chart or just on a calendar or Excel.

    Today I would like to talk about two completely incorrect assumptions that underlie this technique.

    For starters, what is an estimate of the project time or is there a project step?

    No, no, I understand that, in anticipation of management, you, as Kassandra, must absolutely accurately predict the future, and unlike Kassandra, it’s not more optimistic to do this, as business tenets require (I don’t know how this thing translates, well, something like a moral code of the young builder of communism - according to it we have compulsory training every year). In general, everyone understands that this is difficult, but your prophecies should be “good enough!” from all points of view.

    Well, when managers say that, that’s understandable. But we are not managers, we are engineers, techies. We know that “good enough” is something very slurred. And to be part of engineering discipline, a concept must be measurable in numbers. That is, I want to somehow measure this matter, to say clearly “good enough” or still not “good enough”. And when an engineer wants to measure something, he uses such a harmful science - mathematics. She is harmful because she constantly talks about managers what she thinks about them, and even turns out to be right ...

    So, and what is an estimate in mathematics, more precisely in mathematical statistics and probability theory. And this is the interval into which the expected random variable falls with a given probability. A random value is how much time a project will take or is there a separate project step. It seems that it is necessary that the doctor prescribed, but what to do with a given probability?

    Try to approach the manager and say, “With a probability of 80 percent, this project will be completed in a month.” What will he tell you?

    Yeah, have you already felt a slight malaise? Further is better. He wants “for sure,” but what does mathematics say? Imagine a picture of some habitual distribution, well, say, of a Poisson or normal-Gaussian distribution. What is the interval that is 100 percent likely to hit? That's right, alas, if a manager wants a guaranteed assessment, then mathematically it is always infinity. Well, or practically, the term for which the project will simply be beaten. Any project has a chance of failure. Good engineers, who were allowed to make good grades, have less chance of failure, and bad ones have much more, but still under no circumstances is it equal to zero.

    That is, volens-nolens will tear off your grades, which will be considered as your promise, although in reality they will be grades for a probability of eighty, ninety, ninety-five percent ...

    So, the assessment of the project or the project step is the time interval in which it will fit in with a comfortable probability for you. Or you can say with a comfortable risk for you not to meet.

    Suppose you have chosen 80 percent as such (respectively, with a risk of not meeting 20 percent). Yes, I know, it sounds very risky, but watching the programmers on natural conditions, I got the feeling that even 80 percent for the majority is excessively high. Normally, smart managers multiply developer ratings by at least a half to two times.

    So, you have a project, it has two steps. You estimated that each step will take two and a half days with a probability of 80 percent. Does this mean that the entire project will take a week with the same probability of 80 percent? If you remember even a little from the Theorver, then you already answered that no, the probability will be completely different, well, except with the exception of particularly exotic distributions that usually do not happen in nature.

    See for yourself an example of, say, a normal distribution. A large blue vertical line is the same project time achievable with a probability of 80 percent. The picture shows the deviation back and forth by the same value t 0 , and the shaded areas show the probability of such a deviation, respectively. As you can see, the green area is much larger than pink, that is, lagging behind a fixed time is much easier than overtaking a project by the same fixed time. The fact, I think, is not intuitively surprising for anyone, right?



    For the most popular distribution for such quantities - Poisson, the picture will be the same. As in general for most distributions encountered in life. And this means that if you gave an estimate of two project steps with a probability of 80 percent as t 1 and t 2 , then t 1 + t 2 will not be an estimate of the entire project with a probability of 80 percent.

    In order not to fool your head with abstract problems, let's also look at the question a little more practical. But does it happen that a person finishes work ahead of schedule? Well, sometimes it happens, but as they say, a negligible amount. After all, even if you finish earlier, most likely you are delighted to use the free time to clean the code, test, make sure once again, right? Actually, everyday wisdom spread in collections dedicated to Murphy’s law simply says: “Any work takes all the time allotted for it.”

    With this amendment, everything becomes even more ridiculous. Since if we assume that the work does not take less than planned, then for the project to continue to fit into t 1 + t 2, it is necessary that both tasks be completed in the planned time, which, I recall, was estimated with a probability of 80 percent. Now it’s time to pull out the school textbook on mathematics and calculate how likely the entire project is to meet the time counted in Project or else by adding:

    P (T1 & T2) = P (T1) * P (T2) = 0.8 * 0.8 = 0.64

    That is, if you divided the project into only two parts, evaluated each with 80 percent probability, then the amount has a “reliability” of only 64 percent! Try as a homework, what happens if the project is divided into ten steps? Have you tried it? Yeah. A little over ten percent. Even if you shake the engineers with an estimate of each step at 90 percent, the chances of your project out of ten steps will be only 35 percent.

    By the way, that is why Joel Spolsky in his Evidence Based Scheduling ( http://www.joelonsoftware.com/items/2007/10/26.html ) insists that the estimates should not be added, but obtained from the step estimates by the Monte Carlo method .

    Of course, you can see that the quality of individual evaluations greatly affects the quality of the evaluation of the whole. Thus, improving the estimate of steps from 80 to 90 percent improved the assessment of the project from ten steps from 10 to 35 percent. But let's be honest, and 35 percent will suit you? And how often do you do projects in just ten steps? And with a hundred steps, even very high-quality step estimates with 99 percent probability for each step will give you at the end less than 40 percent chance for the entire project. Are you happy with 40 percent? And where do you find engineers mistaken only once out of a hundred and at the same time not taking the whole day to write one line of commentary?
    Of course, in fairness, it should be noted that such a “collapse” of the chances of success occurs only for steps on the critical path. But do not forget that if you use your resources (people) effectively, then even with a sufficiently small delay, any step can be on the critical path. So the situation may not be as bad as in the examples above, but not as good as we would like. However, you too already know this ... :-)

    Of course, now there will be a natural question, but what to do? You know, something is possible. But this is a subject for a separate article, and not one.

    And really dotting the "i", briefly, what I tried to say in this article:

    1. Evaluation is not when the project is done. This is the time at which he meets with a given probability . Even if the manager or engineer does not know this, it is still there and depends on the optimism of the engineer and the manager’s aggressiveness towards the lagging ones.

    2. Management makes an order of magnitude big errors in the project estimates, adding up the estimates of individual steps than engineers, evaluating these individual steps.

    3. Even small errors in the estimates of individual steps can significantly reduce the reliability of the assessment of the entire project.

    4. And as usual, the calculator does not replace the brain, and Microsoft Project - the art of management.

    5. Oh, yes, one more thing: Do not make yourself an idol. Including from fashionable techniques.

     

    Also popular now: