What to do with other people's debts?

    One aspect of the developer’s profession is the dedication of laypeople to a particular software development process.
    S. McConnell, Perfect Code

    The purpose of this publication is to share experience working on a project with a complex history and heavy legacy. After leaving the next so-called "Startup", I decided that I want to try new sensations: enterprise, legacy, etc. To do this, I took up work on a corporate application for a transnational concern. Development at that time was already in its third year, the application survived several generations of developers, but there was no stable release.

    I believe the publication will be useful:

    • similar decision makers to weigh the pros and cons
    • managers of "difficult" projects in order to better understand the causes and consequences of technical problems
    • and of course just curious

    Questions addressed in the article:

    • Low competence of developers, and what can be done about it?
    • What arguments are convincing in the eyes of the customer for non-functional changes in the project?
    • Why is the work of analysts and QA very important from the point of view of development in particular and for the project as a whole?


    Input


    What's at the entrance? At the first stage interview, I was intrigued by the following details:

    • The used version of the framework over the years has been outdated
    • The release of a stable version of the product in two years did not take place
    • The business complexity of the system sometimes rolls over

    At the second stage interview, with the participation of the project manager and lead developer, I received the following portion of nuances:

    • The current development team has a low level of competency in the used framework, language, testing, IoC, object-oriented design, design patterns and architecture
    • The subject area is really not easy
    • The customer is convinced that there are no problems

    By the time of the positive decision to participate in the project, I had not seen a single good or reliable place in it: absolutely everything that I managed to find out was a sign of problems. This is what prompted me to a positive answer =)

    Summarizing the state of affairs at the start, we can distinguish the following ballast categories that represented a risk for the success of further development:

    • Low developer competency
    • Poor code quality (derived from the previous paragraph), a large technical debt
    • Lack of established infrastructure
    • Lack of automated testing
    • Lack of acceptance testing culture
    • Lack of culture of versioning, modification, deployment
    • Requirements are not standardized.
    • Managerial debt to the customer - concealment of the problems existing on the side of the executing team, exaggeration of success, new promises to old debts

    The general message that I would recommend to developers claiming to be the "rescuer" (as senior, leading developer, technical expert, etc.): do not lie or be afraid . If you find a project in a deplorable state, this does not mean that the people who worked in it before you consider the same. First: the current state of the project is largely the result of their efforts, it is psychologically difficult for people to perceive criticism of the result of their work. Secondly: most likely, the level of the team that allowed the existing problems does not allow them to be discerned - for them, objectively, there are no problems. In this case, the banal thick controllers - containing all the business logic - were considered acceptable. The lack of unit tests is not perceived as something terrible if a person is simply not familiar with unit testing.

    In such a situation, you will inevitably have to plant your own best practices in the team. There are two ways to do this: evolutionary - to train (first of all by our own example) the available personnel. The revolutionary is to look for more competent performers, which means systematically replacing the team.

    Start with the tests


    One of my first events was connecting a test framework, helping colleagues to start and write tests. Evolution, of course, is a more humane option - you don’t have to “merge” anyone, spend time and money on the selection of more experienced (and probably expensive) specialists. And karmically it is much more useful to share knowledge. But this is a long way. If a developer, mature enough to consider himself ready to develop enterprise applications for money, did not master the testing, something in his professional growth curve went wrong. The drama is that most likely, unable to withstand the new requirements, many will prefer to find a new job. It is human nature to follow the path of least resistance. In this case, an attempt to educate is a waste of one’s and others ’time.

    If you do not have an extra person-year in stock, which is more than likely for commercial development, then the inexperienced team has little chance of smoothly evolving. It is better to immediately prepare to form a new team, presenting higher requirements to the applicants than before. Yes, thus increasing the budget. And this is not a inflation of the budget, it is a compensation of the debt created in the past by the manager: maybe in an attempt to save money, maybe due to lack of experience.

    Enumerating the technological changes that were to be made, I began with unit tests. Unit tests are also the cheapest implementation and maintenance opportunity to keep your project in order. And at the same time one of the most fundamental. Starting with tests is a good habit. With tests, you can start coding, using an unfamiliar library or language, setting a task, checking the implementation of anything. Tests are a way of thinking in which you formally describe the result before you do anything. If you hone this skill to the proper level, then getting any result will not cause you any difficulties, you will forget about the fear of a white sheet. And this applies not only to programming.

    SVN -> Hg -> Git


    Historically, the project used SVN. Merzhi was a difficult and responsible matter, accompanied by a sacramental "do not push it into the trunk yet, today it is deployed." Colleagues in other projects used Mercurial. For about a month we tried this version control system, but in the end we preferred the transition to the well-known and popular Git. As expected, most job seekers, when forming a new team, were more likely to be familiar with Git than with the other two hard currency. What is not a reason for the transition?

    CI & CD


    Windows Server -> Ubuntu
    Remote Desktop + manual update -> delpoy.sh -> Unix + Docker + TeamCity


    Copies of the demo and testing application were located on Windows Server. Server management and application updates were carried out manually by connecting to a remote desktop. About half a year it took me to convince the manager, and, through him, the customer, that the prerequisite for release in production should be the transfer of infrastructure to Unix. In parallel with this formal justification, in the process of searching for a second backend developer, I looked towards the candidates owning the administration of the LAMP stack. Fortunately, we managed to find a specialist with good skills in Bash and Unix: as a result, he became a 50% developer and 50% build and integration engineer on the team. By the time we went into production, we had a full CI and CD . Hi Rottenwood !

    This event, like others, is not a purely technical solution. Development methodologies and concepts influence other processes, do not forget about it. If the manager is used to leading a team for which the preparation of the release boils down to “** yak - ** yak and into production!” , It’s not enough to configure the agent in TeamCity. You will have to convey to the manager the realization that “you can’t just take and“ fix ”it in five minutes on the prod to the demo”. Yes, it will be uncomfortable at first. It will take the manager a month or more to get used to the fact that the deploy now is not happening in a kick, instantly, along with the fall of working production. Now this is a deliberate procedure, which takes 10 minutes, but is guaranteed to not drop anything, and will give a predictable result on any of the servers, no matter how many you have.

    For the customer, the arguments for the need to allocate resources and time for this were:

    • Reduced downtime in emergency cases - thanks to Docker we can quickly deploy to any virtual machine in any data center.
    • We can put the application along with the image and it can be run on any machine (such a case was also considered) without the participation of the development team.
    • Security - isolation of the container with the application from the host machine, simplicity and reliability of Unix for the server. Some points during the passage of the corporate security audit application were automatically closed.
    • Native environment for the stack used by the application, a wider range of applicable technologies for potential features. Great developer loyalty.
    • A tangible performance boost without additional configuration.

    Eclipse / NetBeans / Trial WebStorm / Brackets -> PhpStorm


    Another important event was organizing the acquisition of a PhpStorm license for the team and setting up a uniform formatting style in accordance with the PSR. I believe any organization can ( and should ) provide workers with normal tools. PhpStorm and WebStorm are now leading, in my opinion, the IDE in the market for supporting PHP / JavaScript / TypeScript. A good IDE can significantly increase both the personal effectiveness of the programmer and the team as a whole - it is easy to implement a single code style and various useful “gadgets” for working on a project through settings.

    Devprom + Excel + * .jpg-> Jira


    This transition was perhaps the most epic and long-awaited for us. Historically, Devprom was used. In a nutshell: I do not recommend this system to anyone. It was a revelation to me that paid software can be of such poor quality ! At random, the system could freeze, crash, contain explicit vulnerabilities. Each update, in addition to the patch of several SQL injections (and adding new ones, judging by the frequency of updates) introduced innovations in the location of GUI elements, so the familiar use cases had to be re-mastered.

    Since the project management system is the cornerstone of the planning and development process, the use of a “lagged” and unstable solution affected all other processes: people were too lazy to start bugs in the tracker (it was difficult and time-consuming), when planning, filling the iteration backlog turned into torture for the whole day , tasks were lost, analytics and evaluation tools were easier not to use, managers at every opportunity tried to push something to work around the tracker. Thus, the tool used was clearly detrimental to our processes, you must admit it is terrible.

    Thanks to Devprom and the ingenuity of the managers familiar to him, I observed some alternative management methods:

    • Work on pictures - after testing the output we have a folder with screenshots, named in accordance with the priorities. Developers have fun grabbing the pictures they like and “fixing” them, transferring pictures to the Done daddy. For an attempt to create a task for each picture in the tracker, you could get it by hand.
    • Email-driven managment : in a mailing list between team members, a letter goes in the table of high-level “features” for several days to a week. The initiative to use the tracker board for operational development management is punishable.
    • Excel planning - backlog compilation and evaluation using the legendary spreadsheet processor.


    The task tracker is no less important in development than the IDE. This is the cornerstone of the development process : any activity in the project must begin and end in it. No task, no code. Jira is arguably the best commercial solution on the market.

    PM-level


    If a developer carries out operational management in a team, much can depend on his decisions. His choice determines the success or failure of the implementation of individual parts of the product. Of course, this is most relevant for small teams: 2-4 developers, tester, analyst. In proportion to the increase in the number of different specialists - we introduce architects, administrators, QA-department - we must assume that the degree of personal responsibility of an individual participant in the process is reduced.
    But do not forget that there are at least two factors that you, as a technical specialist, on which you have no direct influence. At the same time, the very possibility of project success directly depends on them:

    • The total budget of the project and, accordingly, the staffing table. You can’t write a competitive analogue of the application if you have an order of magnitude fewer specialists. Further than the prototype, your “startup” will not fly away with enthusiasm.
    • The activities facing you in the food chain - customer and business analytics.

    If the project manager is on your side and can knock out additional costs from the senior management (or the customer, depending on the structure of your organization), if the manager is able to defend the importance of the necessary non-functional changes in the project and process to the customer, you have a chance of success. The rest depends on your perfection.

    If management is not ready to listen to the development team, there is no chance of recovery. You cannot directly influence the decision of a superior, or change the project management. But you must do everything possible to be heard, if you do not want to be responsible for the mistakes of others.

    Dev-level


    If your experience as a developer suggests that the project has problems, for example, a banal technical debt, you can rush your chest at the embrasure of refactoring ... And die, because the rest of the team will have time to "shit" from your machine gun more lines per minute than you can physically refactor and cover with tests. Get away from the development itself, look at the process from a bird's eye view: code reviews, secure branches and pool requests, statistical code analysis tools in your CI - there are many tools to prevent the spread of the problem. It is much more important to eliminate the cause of the problem; eliminating the symptoms is a secondary matter. And not the fact that you have enough time for the second, with most legacy you have to live a long time. The main thing is to prevent metastases.

    Sometimes the mistake is really in the genes. Then the best way is to cut off the low-quality genotype from your population and replace it with a fresh sample with the necessary dominant traits.

    After a year of refactoring, it seems to me that sometimes it’s worth evaluating developers not by the large number of benefits brought (functionality + the corresponding code base), but by the minimum amount of harm left, in the form of technical debt and unsupported or not tested code. Of course, your manager will have an alternative view of reality. But the reality of the development is such that it’s easy for the average developer to “gash a feature” of almost any complexity. It is much more important in the medium and long term that this addition does not degrade the architecture, is accompanied by not fragile and understandable tests, was designed with an eye to the principles of SOLID. In this regard, I would prefer one senior to two middle and two middle to four junior. The longer the distance you and your product have to go,

    Alas, it is almost never possible to assemble a sufficiently strong team in an acceptable time frame, even having the necessary budget. Qualified specialists, even in the most common technologies and frameworks, are sorely lacking. Building an industrial development process will help you offset this. Use possible utilities and techniques to analyze and control code quality. Having set an average or novice developer behind a streamlined conveyor, you will get a more predictable result and systematic development than letting the enthusiast with glowing eyes “commit” to the master. Your task should be to organize a stable process and facilitate the incorporation of participants into it, and not heroically eliminate the consequences of no less heroic dashing artisanal decisions.

    Analytics


    If you see a business analyst throwing raw requirements into development, don’t turn on your imagination and don’t start coding. Make a list of questions and all conflicting places and send him a letter. Or discuss all the dubious issues together over the printout. Coding, however, should start when you have so certain requirements in hand, in a file approved by the analyst, that the task of coding them can be delegated to any developer. I believe that the ideally posed development task, in addition to referring to relevant requirements, should not contain any “business logic”, in the worst case, a high-level description of the design patterns used, if the appointed developer is not yet familiar in detail with the components of the system where changes are supposed to be made.

    A banal truth that we forget daily: people tend to think their thoughts are obvious . Despite the average high IQ in our industry, telepaths, alas, are always on vacation. If you enter the telepath mode and do its work for the analyst, then you will be to blame for the following sins:

    1. With a probability close to one, the task will take you more time than you planned. Indeed, giving an assessment, the developer always voices the time for coding. The most experienced of us can take risks of debugging, testing, documentation maintenance, etc. But I doubt that even the best developers are able to accurately predict the time they need to work as a business analyst. The analyst will not program for you. And for the excess of time to answer you.
    2. You will be responsible for all your fantasies, since getting into implementation, they remain not described in the requirements and not agreed with the customer. Then you will have to convince everyone for a long time that this is not a bug but a feature, and ultimately rewrite this code, with the advent of real requirements. Well, if you are. It’s much worse to stumble on undocumented behavior and cryptic code with initials whose owner no one on the project remembers.
    3. You are depriving a good analyst guy of professional growth opportunities.
    4. You increase the entropy of the universe, offsetting the benefits of the division of labor.

    The results of oral discussions and correspondence should be recorded in the final requirements files. Develop a package of documents necessary for adequate development planning and planned coding. Determine which formats are needed to describe a particular part of the system. For example, for each screen form, the following set of documents may be required:

    • Visio file with functionality description, flowcharts of dependencies between fields
    • Excel file with validation rules and a description of the data types for each field
    • Layout PSD file, related sources (fonts, icons)

    Such a package will be in demand not only at the development stage but also at the following: acceptance testing and development of user documentation.

    It would be ideal to version these files in parallel with the source code in order to understand the current and required state of the system. Unfortunately, with most complex office formats, versioning by analogy with the source code in Git is not applicable.

    Business analysis is a process that is important to formalize and debug no less than the development itself. And since you are a kind of consumer in a developer-analyst relationship, it is your duty to help build these relationships for mutual pleasure.

    Estimate it


    If the manager forces you to evaluate the task for which the analysis was not carried out, put the top of the possible ones. For example, I use Fibonacci values ​​13 or 21 for such tasks, while for normally planned tasks the maximum value is 5. Thus, you clearly reflect the complexity, which at this stage cannot be estimated more accurately.

    The other extreme: set the minimum score. I use 1, although many optimists tend to make promises like "this can be done in 5/10/15 minutes." Yes, of course there are edits, the introduction of which will take a matter of minutes - not including the overhead of interacting with the tracker, hard currency, documentation, tests. In order not to upset the manager that the “small” fix takes an entire engineering hour, I can recommend related minor edits to be combined into one task.

    QA


    If you get a bug report in the form “Repair a form on page M” or one screenshot with a big bold question mark on an innocuous, familiar page, you are unlikely to be able to fix the problem. Formulate the format of the bug report in accordance with the features of your application. Show and tell everyone involved in product testing how to get debugging information necessary for correction, how to formulate. Do not attempt to reproduce the unplayable.

    Another nuance: if the team does not have a testing culture, the manager may believe that manual testing of the product is up to the developers. Your mission is to show him simple arithmetic: the developer’s hour is usually N-times more expensive than the “manual” tester’s hour. For several full days of testing by the developers in the team, the salary of the salary of the selected tester is easily burned. Do not forget about simple development.

    Testing- This is a process that should take place systematically and on a regular basis, and not an event like the April subbotnik. If your organization does not yet have QA, but you know what it is - bring it closer by all available means. While there is no qualified acceptance testing, the development team will turn out to be extreme in all detected bugs. If the testing is irregular, then bugs will be rare, but many are not the same. This means that an avalanche of bugs that are automatically detected will poison your life, and seriously harm the entire development process. In the mentioned project, it took me about a year and a half to knock out a staffing unit for a QA specialist and search for a suitable candidate.

    What are the risks of irregular and poorly organized testing:

    • If testing is rare, it will seem that the version is terribly “buggy”, blood-gut-plague, a-ah, everything is gone, we’ll drop everything, we all run to the fire !! 1 - this disrupts the planned development, sometimes completely paralyzing it. The risk of breaking the next deadline.
    • The simultaneous detection of a large number of bugs will be subjectively perceived as the fact that there are critically many of them in the (seemingly) finished version, which undermines your credibility as a developer.
    • In the case of manual, occasional, poorly organized testing, bug reports will be of poor quality - which means that they are difficult to reproduce, irrelevant, duplicated. Developers will spend extra effort and time trying to reproduce them. In my experience, I can say that, often, reproducing a bug takes no less than a fix. And doubly disappointing for the time spent, if failed to reproduce.
    • The time to fix a large number of bugs is more difficult to evaluate, compared with the regular daily inclusion of them in the plan. If the process is debugged, then, having the metrics of testing and development productivity, you can put the assessment of corrections in the long-term plan, in parallel with the rest of the development.
    • The chance to “fix” all the bugs that became known the day before the release on time will be very desirable for both the manager and the young developers. But no, it doesn’t. The release will either be of inadequate quality or broken.
    • Specialists performing case-by-case testing (visiting trainees, project managers, analysts, the developers themselves (God forbid!), A coffee lady, an accountant and plumbers) by definition: a). low-effective in this role, since it is episodic for them b). it is unlikely that they will act in concert according to any plan or scenarios c). give guaranteed low-quality reports

    To help organize healthy testing is in the developer’s own interest.

    The best is the main enemy of the good!


    If you are lucky and you debug the development to an ideal state, this does not mean that management and the customer will automatically become happy. Firstly, if PM, for example, chronically bends to the customer, taking the development commitments more than physically can be done without the knowledge of the team, then as the stability and productivity of the development increase, the obligations will also grow. Secondly, there will always be problems that have not been voiced before, which, if not owned by others, will receive the highest priority. Here the scheme is this: if the “fakapila” team used to have stability and new functionality, now the customer can consider the application to be slow and write it as an “epic fakap”, although initially there were no requirements for this. Or remember some Pet-feature that breaks all the existing logic and set it as a must-have, and count any counter-arguments as unprofessionalism and sabotage. Here everything depends on the adequacy of the customer and your managerial layer. In such situations, it remains only to consistently argue, appeal to logic or look for a more grateful organization.

    Another caveat to keep in mind: as the product grows, the cost of making changes inevitably grows . Is always. All the best practices and other lotions, which are described above, only reduce the growth rate of this cost. Thus, if a person did not have long experience working with teams that have well-established processes and do not understand these processes well, he can draw the following conclusion:

    - there was: no formalism, ** like in prod, the boys quickly did everything.
    - it became: a lot of formalism, some kind of CI / QA makes it difficult to quickly “release”, the guys slowly began to code ...

    At the same time, having no personal experience in projects of various sizes and teams, it eludes that between the “was” and “ became ”are:

    1. long time period
    2. a large slice of functionality implemented during this time
    3. unpaid debts from what was

    Therefore, for such characters it is important not so much to restore order - for a non-developer, any changes, except for functional ones, will be ephemeral and optional. An important justification with illustrative examples is important: what, how and why. That same aspect of the development profession that McConnell mentions.

    Only registered users can participate in the survey. Please come in.

    Are you looking for projects with similar challenges to solve then?

    • 11.2% Yes, I’m looking for, this is just my position / role in any project 55
    • 51.1% No, but getting into chaos - fighting as best as possible 251
    • 12.8% No, I’m going a mile and a half, someone else's mess to fix the business is not grateful. And if I get there, I’m rather looking for something more decent 63
    • 2.4% I am the same customer / manager, write the code, *** t! 12
    • 22.4% I myself am Chaos, I have come out of Chaos and defeating Chaos, you cannot defeat Me 110

    Also popular now: