Observer effect

blv May 5, 2011 at 11:50

To find out whether the soup is salted, just drop two electrodes into it, and then put current on them. If the smell of chlorine appears, then the soup is already salted :)

In physics, there is such a thing as an “observer effect”: if an observation is carried out on a system, it makes a change in its behavior. This effect is very interesting manifested in the organization of work of development teams (and indeed in any production processes). As soon as we begin to count any metrics, we make changes in the behavior of the team and its individual members.

"Do no harm"

From the point of view of management, to measure numerically (introduce a metric), the idea, at first glance, is very robust. We receive accurate data and based on them we perform the necessary actions. But, unfortunately, not everything is so simple: as soon as we introduce the metric, the behavior of the team begins to change. The team and each of its members begins to adapt to the metric we have introduced.

It’s not necessary to go far for examples. Everyone knows the concept of “Hindu code”: as soon as developers begin to pay for the number of lines of code written, they begin to write more code, often meaningless, forget about refactoring, it becomes unprofitable to do it, and so on. Thus, our measure of productivity is turning against us.

And yet, paradoxically, this is a very accurate metric. Two programmers in a team correct approximately the same number of lines for the same period of time. Of course, you need a lot of conventions (similar tasks, the presence of coding standards, for example, the maximum length of the method, the absence of duplication, refactoring, and so on), but still the metric is quite accurate ... if you do not use it as a performance rating.

You can try to reduce the number of defects in our product, for which consider how many bugs each developer made. We will reward the best developers with bonuses, and sanctions will be applied to the worst developers. Everything is simple and transparent. But in reality it will turn out differently: the developers will argue with each tester for each bug, there will be a fear and a desire to avoid risks, as a result of a lack of initiative and unwillingness to do the tasks (“Less tasks, fewer bugs”). There is a very good topic on the habr on this subject: "They live like a tester with a programmer" by Yulia Nechaeva.

Let's approach the other side and consider how many bugs testers find. Indirect effects will also appear in this situation, because each tester will regard the slightest deviation as a defect.
The conclusion from the above examples is simple, when introducing metrics, it is necessary to be guided, first of all, by the principle “Do no harm”.

What to do?

Analyze how measurements can affect the behavior of the team and its individual members. Moreover, the analysis should be carried out primarily regarding non-obvious aspects that may not appear immediately. Think about which metrics can serve as a basis for discussions, for example, in retrospectives, if you use Scrum. Maybe nobody really needs your measurements and you measure, simply because you can measure?

Calculate how much time it takes you to collect metric information. It is very likely that this process can be automated, especially if we are talking about code metrics, quality metrics, and the like.

When introducing metrics, like any other innovation, it is very nice to use the Deming cycle: Plan-Do-Check-Act, so that you have the opportunity to receive feedback from the team and make changes if necessary.