December 14, 2007 at

Jacob Nielsen: usability, expensive and cheap

    Summary:
    Calculation of net present value (CTC, net present value, NPV) allows you to evaluate the most profitable level of investment in usability. The use of expensive usability methods can be economically justified in the case of large projects.
    I usually advocate “ usability at a low price ” - fast and cheap methods, the use of which leads to the direct improvement of your user interface. But sometimes, it makes sense to spend more to get more .

    When evaluating the possible uses of usability methods, one often has to choose from two alternatives, the cost of which can vary significantly. For instance:

    To select the appropriate option, we use a combination of 4 parameters :
    1. The expected value of the project;
    2. Costs for each of the usability of the alternatives;
    3. Estimated difference in results for alternatives;
    4. The percentage of the depreciation of future cash flows to account for the uncertainty of the discount rate). (

    Calculation Example

    To illustrate my idea, I will give the following calculation example. First, let's say that the expected value of our project is $ 1 million . For example, it can be an electronic store with an expected sales of $ 5 million per year and a return of 20%, or a redesign of an organization’s intranet with 10 thousand employees with an expected productivity increase of $ 100 per employee per year.

    Note: Now I am only considering the part of the value that will be realized only during the first year after the launch of the product. You can consider this as a first approximation, since the project life cycle is often longer. For instance,the average time between intranet redesigns is 3 years . If you expect your project to be relevant for more than one year, then you certainly must take into account the value added over the coming years (do not forget to take into account the additional uncertainty of future periods of time by reducing the current expected value by the selected percentage of the depreciation of cash flows). For simplicity, in this article I will only consider the first year after launch.

    Now, let's evaluate the difference in costs and the expected gains from using two usability alternatives. Let, for example, a cheap approach cost 10 thousand dollars , and expensive - 40 thousand .

    Let us also take such a very restrained assessment that the results of an expensive approach will only20% better than cheap results. Since the use of usability tends to double the value of design projects , we assume that a cheap approach will provide an increase in expected results by 90%, and expensive - by 110%.

    An increase in results by 20% is quite typical, for example, for a situation of transition from testing an interface with local users to testing with international ones. Even if you learn more by testing in more countries, most of your results will be the same for each country. It turns out that an increase in usability by 20% will cost you an increase in costs by 300%. You will probably decide now that you should definitely choose a cheap approach, but wait - you should read on.

    Finally, set the depreciation percentage to 100% . Thus, everything that we get a year after the launch of the project will cost exactly twice as much in today's money: a win of $ 2 in a year will cost $ 1 in today's money. Such a huge percentage of depreciation can be justified by the fact that design projects have a high degree of uncertainty :
    • Firstly, a project may be canceled or failed for reasons beyond our control. The best design costs nothing if the application is not running;
    • Secondly, all the numbers used by us in the calculations are only estimates . We can’t say how successful the project will actually be, and we don’t know what results the user research will give us until we spend money on it. Some usability consultants offer solid quotes, but if you do the research yourself or you have an hourly pay for the consultant, you can’t even be sure what the costs of implementing each of the alternatives will be.

    Thus, we have 3 choices: do without usability, cheap usability, expensive usability:
    • Option: No usability . It is easiest to calculate, since we do not consider the cost of usability here. Take our expected value of 1 million and multiply by 100% depreciation. Thus, we obtain the current value from the project: 500 thousand dollars, excluding the costs of the project itself. Thus, if the costs of design and development (excluding usability) are, say, 200 thousand, then the net present value (CTC) of the project will be 300 thousand dollars ;
    • Cheap usability . Increases the value of the project by 90%, which ultimately gives $ 1.9 million, or 950 thousand, taking into account depreciation. Now subtract 10 thousand usability costs and get a current value of 940 thousand. Thus, the use of usability added 440 thousand dollars to the expected current value of the project. After deducting 200 thousand development costs, the total project CCT will be 740 thousand dollars ;
    • Expensive usability . Increases value by 100%, which gives 2.1 million or 1 million 50 thousand, taking into account depreciation. Now subtract 40 thousand and get the current value equal to 1 million 10 thousand dollars. This usability alternative added 510 thousand to the current value of the project. With the deduction of development costs, the total CTC is 810 thousand dollars .

    Using an expensive usability approach instead of a cheaper one, taking into account all our assumptions, increased the CTC by $ 70 thousand , so in this case it makes sense to use a more expensive approach.

    (Our calculations assume that the development costs are constant, regardless of whether or not usability methods are applied. Simply put, I assume that we chose the usability methods correctly and set the design direction from the very beginning of the project, so we don’t have to redesign the developed functionality.This assumption is incorrect if usability is only involved closer to the completion of the project, in this case it is necessary to add the cost of fixing the defects discovered by usability.) s, for example, the cost of correcting defects constitute 50 tysyach usability for cheap and expensive for 60 tysyach (Detected 20% more defects). The deduction of these costs does not change our final decision to use a more expensive approach, it only reduces the gain from its use to 10 thousand dollars.)

    Expensive or cheap: what to choose?

    All these rather complicated calculations can be avoided by using five simple rules for choosing between expensive and cheap usability:
    1. The higher the expected value of the project , the more you should invest in usability. Usability pays off by increasing the value of the project through more frequent use (or more efficient use in the case of an intranet). If the value of your project is small, then you will not earn much by doubling it.
      • For example, if the expected value of the project is 5 thousand dollars, then in this case they should not even resort to usability on the cheap, because a gain of 5 thousand will not justify 10 thousand costs. (Although there are ultra-cheap usability methods that can be applied to such tiny projects.)
    2. The higher the degree of uncertainty of the project , the higher the percentage of depreciation. In this case, it makes sense to choose usability on the cheap. Yes, a more expensive option can lead to greater gains if the project succeeds, but the likelihood of a complete failure may be too high to justify such an investment.
    3. The sooner you start to use usability in the project , the more you will have to invest, because you will be able to implement additional research results without significant alterations to the product. If you start research in the later stages of the project, then make them cheap, because you will still be unable to implement any significant changes.
      • Note the contradiction between the phases of the project and the degree of uncertainty : at the very beginning of the project, the degree of uncertainty is higher (which speaks against high costs), but the potential for implementing changes is greater (which speaks in favor of more thorough research). Therefore, you should save more money in your budget for future research, when you will have the opportunity to test a more advanced design.
    4. The higher the expected difference in results , the more you should be inclined to use expensive methods. If the expected gain is small, then save your money by choosing usability on the cheap. A great example of this is quantitative research , they are very expensive and usually justified only for gigantic projects, because otherwise they give very little gain in results.
    5. Finally, the most obvious: the more expensive the methods, the less often you should use them . In our example, the CTC of both alternatives will be equal if the cost of the expensive method is 110 thousand dollars. Thus, if the expensive method gives a gain of 20%, then it can be applied if the costs are no more than 10 times higher than the costs of the cheap method. If the excess cost is 11 or more times, then using the expensive method does not give any gain.

    To select the appropriate usability of the method, it is necessary to take into account many different parameters, and many different scenarios. Both expensive and cheap usability methods make sense in a suitable environment.

    Original article
    Tags:

    Also popular now: