Cost effective code

Once upon a time there were two development teams in the two neighboring villages of Villaribo and Villabaggio. Both of them did code reviews, wrote tests, carried out refactoring, but after a year of development, Villaribo had already released a release and went into production, while Villabadjo still refactored and fixed bugs. What is the matter?

Software development is an area at risk. In our sphere, when one or several risks occur, the delivery time for the working version may shift not by the usual and comfortable 10-20%, but by all 150-300%. And I must admit that this is far from the limit.

We can either cross our fingers and hope that success will accompany the project in everything, or admit that, according to statistics, most software development projects “fail” and take additional efforts to mitigate possible risks.
My practice shows that customers are extremely reluctant to work according to the T&M scheme and more often prefer Fixed Price . In the conditions of fixed cost, the occurrence of a risk event means an automatic decrease in the project's profitability: employees receive a salary on a monthly basis, and not for completed projects.

Prior to Agile and XP, all responsibility for working with risks lay with managers. In agile methodologies, developers are much more involved in the process and share responsibility with managers. However, the principles of XP and Agile are more methodological than technological. I think that it is more efficient to work with risks comprehensively at all levels, including at the lowest level, i.e. in the design and coding .

Why should a developer think about it if there is a manager?

1. It’s no secret that if the fakap happens, management will make the only “super-smart” decision: “let's work overtime on weekends”
2. Employees also receive bonuses usually for delivered on time, and not for failed projects.
3. The feeling of work done, after all. It is much more pleasant to pass the project on time and see the client’s smile than to get rid of the “difficult child” six months later

From my point of view, a quiet working environment instead of rumors and bonuses is a good motivation to start taking care of this.


So, we will consider the main risks in software development.

Schedule Error

When evaluating labor costs were estimated incorrectly.

Methodological Risk Mitigation

The team evaluates the complexity of the task using Planning Poker .

Criticism

In programming, we are faced with a large number of unique tasks. Despite previous experience, it can be missed at times because of unaccounted details.

Additional mitigation techniques

Development of Proof of Concept (prototype)

Do you have a description of the client API, but it is not clear how it will behave in a combat environment? The prototype on the “knee” will help test your most terrible hypotheses. The prototype code will most likely just have to be thrown out. As a bonus, designing a system the second time will probably work better than the first.

If there are concerns about the load, you should conduct load testing at the very beginning on the array of generated data to check the viability of the architecture under load.

Research testing

The method is especially good when working with unknown APIs, legacy code, and new frameworks. Before you start, you only have guesses about someone else's code. Perhaps you have documentation, but it can blatantly lie. Writing research tests helps confirm or refute your guesses.

Clear acceptance criteria for assessing tasks

Quite often, the client and the programmer have a different understanding of the meaning of the word “done”. For the programmer, “ready” - this is the code written, but for the client “ready” - this is all configured, uploaded to the server, the data is uploaded to the system. If the team has formal criteria for the readiness of work, then it will be more difficult to ignore them when evaluating .

The emergence of new requirements or changes to existing

In process of work more and more requirements appear or current requirements change.

Methodological Risk Mitigation

Short iterations with fixation of requirements (scrum) or continuous delivery (kanban).

Criticism

Working in small iterations, we are constantly forced to refactor and remodel existing code. The team has requirements only two weeks in advance and does not see the full picture. The slip price decreases, but the efficiency becomes less, because lacking the completeness of information, it is not known how much the “sophisticated” architecture the project will need.

Additional mitigation techniques

Changing requirements is a risk with an almost 100% probability of occurrence, so it is worth considering that it has already occurred before the start of the project and put it in the schedule and estimate of labor costs. However, it is he who is most painfully perceived by programmers. It is psychologically difficult to throw out code that worked well, because, you see, the business model did not work. Unfortunately, you just have to accept the fact that we will regularly send part of the code to the trash. Moreover, the more old code and crutches we throw out, the better.

Lean and the progressive jpeg method

Focus on the most important functionality first. All "gadgets" that can be dispensed with should be implemented at the end of the project. Need to show and hide the panel? For starters, a simple hide and show will go. Complex animation can be added later.

Weak connectivity and modular design, onion architecture

Fortunately, today there are many ready-made solutions that provide poor connectivity. Sing the SOLID mantra daily at work. Think of interfaces, not implementations. Potentially, any implementation can go to the trash. Make it a rule to use the IOC / DI principle . If you have a lot of JavaScript code, be sure to use RequireJS or one of the frameworks , otherwise you will drown in a swamp of jQuery noodles from callbacks, DOM calls, selectors and logic.

Give up the idea of ​​designing an entire application in a single style. Divide it into subsystems. You will have to pay for this with some code duplication, but the ability to throw and rewrite the whole application subsystem from scratch without breaking other parts is much more important.

Evolutionary refactoring with classic top-down design

There are very, very few people who really can be called System Architect. A real architect should participate in three or four really big projects, fill up a couple of them, write and throw away a carriage and a small code trolley. If you have such a person, honor and praise be to him. The trouble is that he will not have time to write code. His full-time job will be designing at the level of large blocks and system contexts. Designing, sometimes, entire subsystems will be left to Team Leads and senior developers. Some of them will cope with the task, and some will not, so it is important that all modules are independent.
Establish developer-client relationships between related teams, for example, the UI team — the client of the Backend team.

This approach provides the following benefits:

1. You get rid of the sacred idol and are ready to constantly refactor and improve the quality of the code. “Smelling” pieces may appear, but only within a specific module. The overall level of quality of the code will be constantly maintained at a high level.
2. It’s easier for you to interact with the client and management, because to explain that this week we have +3 new features is much easier than ours +100500 new classes
3. You save time, including coordination and integration. I have repeatedly observed the play in three acts:
   Act 1 The UI team does not like the API and everything needs to be done differently and in general the fields are named completely differently.
   Act 2 The backend team believes that the UI team is dumb and do not understand how the core works
   Act 3 The managers of both teams persuade them to do something to work, otherwise they will fire everyone in FIG. A curtain.

If communication between the teams is established, then each of them receives valuable feedback. Surely there is a way to slightly modify the file to change the specification on both sides, which will allow trial and error to get an architecture that works in every sense of the word.

Really reused code should be taken to the kernel. Each module (subsystem) may depend on the kernel functions, but the modules themselves should not know anything about each other. If you want to ensure the interaction of the two modules, use event-oriented programming. Subscribe to events only through the kernel. If you have to duplicate some functionality in two or three modules, this is a signal to include it in the kernel or highlight it as a separate external dependency. Such organization of the code will allow to throw out any module or rewrite it from scratch, possibly even using a different programming language. Details of this approach using JavaScript are described here .

Graceful Degradation for low value code, tricky YAGNI

The business value of the application being developed is not evenly distributed throughout the code base. This statement is true for the amount of development effort. Until recently, IE6 support was an example of a tremendous waste of resources on code with potentially low value.

We cannot eliminate the costs of writing code with a low value indicator, but we can reduce them: do not write unit tests, do not refactor, score on the quality of the code in certain parts of the system. For this we will have to pay support in the future. Unless of course this particular piece of the system really needs to be supported and developed in the future.
And if not?And if not, then we are well done and were able to save. One of the secrets of the Villaribo team is that, ceteris paribus (and we agreed that the qualifications of the programmers and their stacks are similar), they were able to direct their main efforts to developing really valuable code, and Villabaggio used a unified approach to the entire code base.

Until they threw me tomatoes in the comments, I hasten to reveal the idea with the lack of tests and the “crutch” parts of the system.
Not a single person in the world can know for sure which part of the system will remain unchanged, and what will have to be rewritten from scratch or thrown away altogether. However, there is the likelihood of a change in a particular requirement. For example, the processing of financial transactions with a high probability will not change over the long term. In addition, the cost of error in this part of the software is extremely high. In this part of the application there is a place DDD, TDD, BDD and any other cool * DD that help you in solving problems.

But if the company decided to arrange a promotion and you were asked to make a landing page for your online bank - this is a completely different story. Most likely, it’s easier to outsource this functionality and make a page in PHP. Fence off such a code with a choke using the Facade pattern. Keep the system core clean.

Sending too much effort to the frequently-modified part of the code base is unprofitable

This approach is described in detail in the article The Good, the Bad and the Ugly code .

Application update mechanism

If you are developing a desktop application, take care of your users and provide an auto-update mechanism from the Internet. For win exists, for example, ClickOnce .

Change of employees (Bus Factor)

Key employees can leave the company, go on vacation, become pregnant or even die, taking with them important knowledge both from the subject area and about the application code.

Methodological Risk Mitigation

Co-ownership of code, pair programming, code review .

Additional mitigation techniques

Use well-known design patterns, name classes in an obvious way (Repository, Specification , DAL, DTO, ValueObject, Entity). It will not be superfluous to leave links to resources on the network directly in the code, if some concept is not too well known, and the material successfully reveals it.

Use the Conventions Over Configurations principle , functional programming elements (especially for working with collections ) and Side-effect free programming , if your platform allows.

Minimize the number of bicycles. If bikes appear, add a comment explaining the presence of a non-obvious code and why such an implementation was chosen. If you still really need a bicycle, put it in open-source or use a separate product. Let someone support the new technology.

For complex subject areas, it is highly desirable to use DDD concepts , especially Ubiquitous Language . This will help new employees quickly "cut into" the project. It is much easier to understand the code if it looks like a natural language from the specification that you just learned.

Use annotations / attributes to help intellisense and code analyzers.

Use standard application deployment methods as much as possible. Modern frameworks come with a database migration mechanism . If the configuration is very complicated, it makes sense to write a deployment script. For severe cases, it may be advisable to use things like vagrant .

Simplify the dev configuration as much as possible. If you use Memcached to speed up the application, consider implementing a MemcachedDummy implementation that will work without a server installed and will always return a cache miss. The faster a new developer can deploy a working, albeit perhaps not entirely, version, the better.

Try to use generally accepted methods for specifying and describing bugs: user stories, use cases, UML, expected / actual behavior, fix the basic business rules of the system in unit tests. Use style guides for UI and server code.

Create support tests for at least basic business rules Use bdd notation in test names to make their purpose obvious. Detailed recommendations on the organization of unit testing on a project can be found in the article Unit testing for dummies .

Decomposition specification

The specification is incomplete and / or contains conflicting requirements

Methodological Risk Mitigation

Using the SpecByExample methodology , the ability to exclude low business value requirements from the sprint plan, formally checking for consistency before starting work.

Criticism

Typically, risk is paired with a “change in requirements.” If the first has come, then the second comes almost automatically. For large systems, the formal task of checking the consistency of requirements is at least nontrivial.

Additional mitigation techniques

Common sense. If an obvious nonsense is written in TK or another change request is insane, then it is best to raise this issue at an early stage and make corrections. Until a decision is made, you just need to postpone this task and take the next most important one from the backlog.

If you do not have enough materials to get started and you know for sure that they will not appear by the deadline, you do not need to play Counter Strike. Try to start working without the necessary materials. No graphics? Use the squares and circles. No visual design, but prototypes? Take Bootstrap . Even slow motion is better than a complete stop.

The presence of tests for business logic perfectly helps to mitigate this risk: you will learn about the conflict of requirements from the fallen test, and not from a floating bug on the production.

Technological risks

Whether the technology stack will satisfy the task. Do you have to change the programming language, DBMS due to load or lack of interoperability? Will the chosen “architecture” / framework fit, won't they become too expensive to support? How bad is someone else's legacy code?

Attenuation Methods

Mortgage horizontal scaling at an early stage, Provide Persistence Ignorance .

Use the Data Mapper or determine the place where it can appear during the refactoring process - this will help to flexibly change the data source if necessary. Modern mappers are equipped with Assertes that will help you remember to map all the fields correctly, regardless of the data source.

Follow the Low Coupling - High Cohesion principle when designing. Clearly define the responsibilities of each class, do not allow the appearance of God Objects .

Prefer QueryObjectRepository pattern. Repository issues in long-run are well-documented in this article . QueryObject has no drawbacks compared to the Repository, but there is an advantage - the ability to quickly move from a monolithic architecture to a distributed ( CQRS and data bus ) to ensure full horizontal scaling .

Use defensive programming , sorry for the tautology, to protect invariants (an invariant is a consistent internal state of an object). Protective constructions should be placed in the constructor, then the post-conditions will practically not have to be checked, since the result of the function will also be an object protected in its constructor.

Use the Composite and Specification patterns to organize the code and break up complex sections into smaller and simpler ones. Use the Producer-Consumer pattern to organize multi-threaded applications, if possible.

Prefer Rich Domain Model over Anemic . Even when using ORM in most cases, you can transfer Entity to another layer without resorting to creating a DTO. Responsibility for creating the DTO should be assigned to the Data Mapper .

Provide at least minimal application self-healing and self-diagnosis systems: installation, filling out the necessary directories, creating configuration files if they were deleted, restarting the service / daemon in case of an error.

Low productivity

The intensity of work is directly proportional to the proximity of the deadline. While the delivery dates are far, there is a temptation to play the fool and give in every way to prokarstination.

Methodological Risk Mitigation

Short iterations, stand-up rallies , practice of demonstrations: after each iteration, the team presents the work done to the client’s representative or internal Product Owner.

Additional mitigation techniques

The investment of the times at the beginning of the project in infrastructure and meta-programming. If your project has many forms, think about the form generator, instead of riveting the same type of View-files. If you need CRUD-functionality for a large number of entities, one well-designed controller with all the virtual methods will help get rid of writing tons of routine code. Virtual methods will allow you to override the behavior, if necessary.

Performance problems with meta-programming (especially reflection) can be solved by dynamic compilation or code generators.

The less code, especially of the same type of routine, you write, the easier it will be to support. In addition, the routine code is the nastiest. The client, in principle, does not matter, you were interested in the process of solving his problems or bored. You, as a developer, can set yourself the task of optimizing your routine work and solving the problem more elegantly. New tasks are always more interesting than dull copy-paste.
It is important to understand that you should not get involved in abstract factories of abstract factories either. If the game is not worth the candle, then it is better to stop and use the proven method. Generally always, if you dig in for 3-4 days in the task for 4 hours, you should stop and ask yourself "what am I doing wrong?" It is advisable to ask not only yourself, but also a couple of experienced colleagues and a couple of non-programmers (“non-techies” can give very useful feedback, they think differently). Almost certainly, the task can be solved easier .

New untested technologies and approaches can be very useful for the project. But they also bring additional risks. Therefore, experiments should be carried out strictly at the beginning of the project, and not a week before the deadline. The beginning of the project is a great time to configure Continuous Integration , automate routine operations and market analysis for new tools that simplify the life of the developer. If you have long heard for example that R #- cool, but you still have not tried it, because somehow there was no time - try it. Who knows how it can improve your productivity.

In continuation of the topic, I recommend this screencast and Fowler's Microservices .