The future of agile software development


    Software penetrates all the gaps of human society. We find out the weather via the Internet, and not through an ordinary thermometer outside the window. We are going to a new address with a navigator, and not looking for the G7 square on page 59. We turn on RunKeeper when we ride our bike to find out the average speed and show off on Twitter. We use software every day. Probably, we spend most of our lives embracing with our favorite gadgets and software, and not with our loved one.

    The problem is that no one knows how to actually write cool software quickly and correctly. Waterfall passed away safely at the turn of the century, and new development methods (agile) still cannot solve fundamental problems. We live in a very interesting time. At the moment, the entire software development industry is rapidly developing and the base for a qualitative breakthrough is being accumulated.

    It is clear to everyone that agile becomes mainstream and very soon individual mastodons with a predetermined life ending will work on the waterfall. Agile won the race. Microsoft is improving agile support in TFS and is using iterative development on many products. IBM Launches Agile Jazz Platform Even SAP implements agile (I'd like to see it)

    And here is a quote from Wolfgang Gatner, CIO of Deutsche Bank

    “We must move forward. Traditional methods are not fast enough. They are too complex and hamper innovation. This is clearly clear. ”

    At least almost all companies are beginning to realize that previous software development approaches do not work. They try to change things and agile is seen by many as the best solution at the moment. But the question arises, how radically improves agile software development? Improves. But does not solve all the problems. A terrible crap still comes out. As before, the development speed leaves much to be desired. As before, the quality of many decisions does not withstand criticism, teeming with bugs and makes itself felt at every convenient and especially not convenient case.

    Take at least the recent fall of Amazon servers, which lasted several days. Our server has been lying for almost a week! Skype crashes from time to time (especially after purchasing it by Microsoft). Everyone remembers dancing with a tambourine to make it work.

    Occasionally, problems arise with iOS. For example, my phone after the next update refused to go into standby mode and landed the battery in 8 hours, not to mention random clicks and calls. It was terribly annoying. I've been waiting for the fix for almost a month!

    Do the right thing


    Let's walk through the three fundamental problems of software development. The main task is to do the right thing .

    What do you mean right? It means necessary, solving specific problems, and solving these problems is good. There are several thousand project management tools on the market. How many of them are really good? 10? 20? hardly more than 20.

    Why were all these creepy monsters with a nightmarish interface created, which torture their users, torment their creators and do not want to part with life for good? Why were created these hundreds of rake-simple tools that do the same thing, differing in their color scheme, platform, and login form design? I have no answer. But it is obvious that the world is awash with software that no one needs, stupidly copied from other software, annoying users with its interface and holding everyone for fools.

    One of the first tasks of our industry is to learn how to make the right software. It is necessary to get into the hearts of people with the first release.



    Do things right


    What is the second task? The second task is to do things right .

    If we do the right thing, that's half the battle. But if we do it badly, the final goal will not be achieved. Yes, the user will be able to solve their problems, but only quickly, from the fall to the fall of the system.

    Poorly written software is strikingly two-faced. He seems to be saying that he is good and an assistant, but in fact it turns out that he is an old wreck, in which, under the mood, oil is mixed with antifreeze, air conditioning only works in winter, wipers only in good weather, and creaks at a speed of 130 half shaft. But he underwent excellent pre-sale training with the help of competent sales managers and quickly went under the hammer. And the fact that the air conditioner does not work, so who turns it on in the winter. Yes, and more than 120 you can’t drive on our roads, so this is not considered a bug.

    A few months ago I tried to use the OmmWriter program. Minimalistic interface for writing large amounts of text. Relaxing music. Full screen mode that hides everything except text. I liked the program and I started using it.



    Once I wrote a large article in almost one sitting. About 5 pages. Of course, I often kept it. And of course, I did not backup. The next day I came to work, opened a beech and OmmWriter happily hung. I killed the process, ran OmmWriter again, and opened the document. He was empty. Have you ever had moments when you thought you forgot to turn off the iron? So, a chill ran down my spine, the level of adrenaline increased and the feeling of irreparable fell behind me. With a shaking hand, I found the file in Finder and checked its size - 0 bytes. That was the last day I used OmmWriter. A program that loses my data loses my trust . I recently bought a very similar Byword program. She works like a clock. I am satisfied.

    Software written crookedly is extremely difficult to develop and change to the new realities of life. The development of such software is expensive and slow.

    At one time, we wrote TargetProcess quickly. Since it was written after work and on weekends, the main task was to test the idea of ​​how viable it was. Unit tests were few. The architecture was weak. For example, the O / R framework was chosen extremely unsuccessfully. No one thought about scalability and performance. But the main task was achieved - the concept has proved its viability. As a result, everything had to be rewritten from scratch and it took 8 months of full time. As a result, TargetProcess 2.0 appeared. Was it good or bad? On the one hand, we seem to have lost 8 months. On the other hand, we checked the concept and made sure that it works.

    Speed


    We are moving smoothly to the third problem - speed.

    Software development does not keep up with users. The complexity of the projects increases - the speed drops. The speed factor is becoming more and more important for any company. If we make software fast, we have the opportunity to try different options, respond to market changes, find the right path faster than others. If we make software slowly, we will not have a second attempt ...

    The industry has its own legendary long-term construction, such as Duke Nukem Forever.

    It started in 1996 and was completed in 2011. After several deadlines, the developers stopped making public assessments, but simply said that the game would be released "when it is ready." Unfortunately, Duke Nukem Forever was never able to meet the expectations of gamers. Gamespot gave the game a score of 3.5. The game was boring, unoriginal and outdated.

    What a slow country! - said the Queen. “Well, here, you
    know, you have to run as fast just to stay in the same
    place!” If you want to get to another place, then you need to run at least twice as fast.

    In fact, we need to run fast so as not to lag behind others. If we want to be the first, we must run faster, much faster.

    Any software development process should focus on these three problems, and in general the slogan of any good process is:

    Do the right thing correctly and quickly



    If we do the right product correctly, but slowly , we can miss the opportunities that exist on the market. We can be late and the reality will change so much that few people need our product.

    If we make the right product quickly, but incorrectly , we will face problems of product development. Either we will insert props and crutches and bury the product in 10 years, or rewrite it from scratch, which is expensive.

    If we make the wrong product correctly and quickly, then obviously few people need it and there will be no success. Unless we can understand why it is wrong and quickly change in the right direction.

    Ideally, we should get to the center.

    The future of agile development


    Here is a picture of the evolutionary development of agile processes in the near future. By and large, everything can be reduced to two main trends: speed and scale.



    We need to learn how to do the right thing correctly and quickly at any scale.

    This is the foundation.

    Why did I endure speed as the main entity? By speed, I mean the time it takes to solve a customer problem . What happens if we do the wrong thing? Will have to redo or completely abandon the idea. What will happen if we do wrong? Will have to redo to move on. All alterations affect the speed in the most negative way. Therefore, doing the right thing correctly directly leads to increased speed in the long run .

    Let's start with speed. Speed ​​is determined by two things: expertise and quick feedback.

    Feedback


    Why is feedback important?

    A Swedish psychologist named Anders with the original surname Erickson conducted many experiments related to training. In particular, he noticedthat many doctors lose their skills after graduation. However, surgeons are an exception to the rule. Surgeons have ongoing feedback and clear goals. Oncologists are getting harder. When an oncologist examines the images, he does not know for sure whether the cancer is there or not. He only finds out after a couple of weeks, from a biopsy. Or even later, through the years when it becomes clear that there is no cancer. Without immediate feedback, a doctor’s abilities worsen over time. Erickson has proposed a new model of teaching on old case histories. You can see the pictures, make a conclusion and immediately get an answer from the real medical history. In this case, the intensity of training increases significantly, one day you can consider more stories than several years of ordinary practice.

    Feedback fundamentally allows you to learn 2 things. Have we done what we need. Did we do it right. We want to know this as soon as possible. Ideally, immediately after completing a task. How to find out as early as possible that we are doing the right thing? Two solutions: sketches and prototypes, quick deliveries.

    Sketching Solutions


    Sketches are a powerful means of finding out requirements. But sketches should not be understood narrowly. A sketch is a quick way to emulate a real system. Sometimes a freehand sketch is enough.



    And sometimes it is necessary to build a not very simple model.

    A few decades ago, IBM decided to create a voice-based computer control interface. A huge number of users dreamed of ordering the operating system, and not poking at it with the mouse. The project was nearly launched, but decided to check, but will users really like it? The sketch was simple. They brought the user and put him on the computer. The user inspiredly gave commands, in another room there was a person who heard and executed these commands. The user saw the process and the result of the command on his monitor. Ingenious! As a result, most users did not like to control the computer this way, it was slow and not particularly effective. The keyboard and mouse won, the project was closed. It’s easy to imagine what tools would be spent on speech recognition algorithms, embedding voice commands in the operating system and so on. Just a sketch saved a lot of money and helped make the right decision.

    Sketching skills development is very important for any software development professional and often we miss this. How is the software being created now? Requirements are collected in one big backlog, important user stories are selected, and forward draw the UI and write the code. In the best case, a prototype will be made, which in the worst case will develop into a real system. This is definitely not enough. At the first stage, the team must make sure that they are going to do the right thing. Yes, this is a business task. But the business is poorly aware of how to make software or design a UI. He cannot solve such a difficult task himself.

    Rushing headlong into a new project and starting to do it right away is a practically guaranteed way to release a mediocre product that differs little from its competitors. If you have only one solution, the behavior of the entire team and the development process are as predictable as a pedestrian.


    Bill Buxton, Sketching the User Experience

    At the moment, there are practically no tools to create rapid prototypes of systems. All prototyping tools are flawed and it takes weeks to create prototypes of medium complexity. This is a very long time. In the coming years, tools should appear that allow you to create interactive prototypes in a matter of days . Such a drastic reduction in time will speed up feedback, test several alternative solutions and make sure that we are going to do the right thing.

    Continuous delivery


    The second way to expedite feedback is through fast deliveries. Delivering after each commit is just a super-radical idea that reduces feedback to the limit.



    For the end user, this is usually not of particular interest. But for the team, such an extreme change the development process, forcing a serious increase in the quality of the code. In order to provide a CD, a company has to work hard.

    The CD includes almost complete testing automation: unit tests, functional tests, performance tests, acceptance tests. Delivery itself must also be automated. This level of automation requires serious work and often restructuring the entire software development process, from setting requirements to the marketing department. That is, the problem of scale is affected, which complicates its implementation.

    Expertise


    Let's move on to more human matters. Software is written by people. Attempts to put software development on the conveyor were unsuccessful, which is not surprising. The software development industry itself is in childhood and has just celebrated the appearance of secondary sexual characteristics.

    While Everything is kept on people. A team of cool professionals will make normal software in the process of almost arbitrary curvature. A beginner team can make cool software only at times of total solar eclipse, even if they have extreme programming from top to bottom.

    People have the expertise and problem solving skills. That is what needs to be developed and improved.

    Domain knowledge


    Knowledge of the subject area helps to understand whether we are doing the right thing. If you know the device of the car, you are unlikely to agree to add a fifth wheel to it in the center. If you see a car for the first time in your life, nothing will jump inside you when you add a fifth wheel and you will do it ruthlessly and mercilessly.

    Domain knowledge is one of the major failures of all software developers . Of course, everyone somehow delves into it, but few people purposefully allocate time for study. When I worked on software in the insurance business, it only occurred to me a year later to read about the structure of insurance companies and find out how Carrier differs from Account in real life, and not just at the class level. But if it occurred to me, then none of those with whom I worked closely came up with it at all.

    Usually, it doesn’t matter for the developer how an entity lives in the real world. A set of business rules is important to him. It is important for him to obtain consistent requirements and implement them. It is important for him to make a beautiful architecture and write clean code.

    Here the initiative rests entirely with the company. It is her task to organize training in the subject area. For example, in our company we send people to conferences and strongly recommend reading books on agile project management, but this is not enough. The CE must clearly understand what we do, why we do it and why it is important now .

    A team where at least a few people have decent knowledge of the subject area can discuss its problems with the business on equal terms and come up with solutions. Confidence in such a team rises many times. The likelihood that the team will do the wrong thing is reduced. This means that the speed of development increases.

    Craftsmanship


    Closer to the heart of programmers is the concept of Craftsmanship . The essence of the concept is to improve the quality of decisions and the development of professional skills. Craftsmanship is almost completely focused on doing things right.

    The only way to go fast is to go right.

    Uncle Bob Martin

    To improve the quality of solutions, it is necessary to build the right architectures, have excellent test-coverage, and not complicate solutions for no reason.

    For professional development, attempts are being made to apply the techniques of training athletes or martial arts, such as Coding Dojoand Coding Kata. Of course, this is a big question, is it possible to apply such techniques to train programmers. But in any case, some training is needed. If you work on production code all the time, you can solve the same problems in the same ways and not develop at all. It seems that we really need classes aimed not at creating something specific, but at honing certain skills. The most effective method is possibly solving the same problem in different ways, using a variety of technologies. Continuous improvements to the current architecture are also good. We need to constantly question the current state of things and try to improve the solution, make it simpler, more flexible, more understandable.

    Obviously, increasing the complexity of systems requires a lot from people. Now you can’t just know C, data structures and algorithms. Now you need to know OOP, patterns, libraries, a bunch of related technologies and disciplines. The amount of necessary knowledge is increasing every year. However, the level of programming abstraction over the past few years has not changed much, while the complexity of the systems has changed significantly. The architecture of Facebook or Twitter is extremely complex , I think no developer there can imagine how the whole system works. So the programmer has no right to stop learning.

    Solution of problems


    Perhaps solving problems is the main skill of any person. If it is, you can easily fill in your own gaps and gaps in the development process in the company where you work.

    See how retrospective happens in almost any team. People gather, put forward some problems that often lie on the surface, and offer solutions that also lie on the surface. Rarely affected and resolved root problems. Very rarely, actually non-standard, cool solutions are offered. At such meetings, virtually no problem identification tools are used. Systemic thinking, TRIZ, brainstorming techniques - most teams ignore all this because they do not have the skills and knowledge. Programmers are not taught systemic thinking. Programmers are not taught TRIZ. Programmers are not taught brainstorming. Programmers are not taught to think outside the box .



    Techniques for solving problems are not studied either at school, or at an institute, or at a university. This is just unbelievable! One of the main skills in professional life is not mentioned in any lecture!

    Programmers themselves love to learn everything related to programming strategies and tactics, the rest of the things to mostdevelopers are of little interest. Programmers know the algorithmic way of solving problems, but practically do not know the heuristic, creative way. Tasks for improving processes are almost always complex and creative, which cannot be solved using algorithms. Architecture challenges also require creative thinking.

    Experienced trainers try to bring new practices into retrospectives, but they do not always succeed. Often they themselves are not experts in problem solving, often it is not possible to transfer their knowledge to the team.

    Agile hopes for self-organization in teams. So the team itself must identify and solve their problems effectively. Otherwise, it will inevitably step on the same rake. However, no agile methodology contains or promotes problem-solving tools.

    Interestingly, in the field of design and UX, people use these tools. There it is something commonplace, just another set of working tools. Developers often lack this.

    An excellent developer is required to have creative, innovative thinking. He does not have to be able to compose poetry, but writing coherent, good texts is imperative. Develop the right hemisphere, gentlemen!

    Scale


    Scale is a serious test for agile processes. Basically, the problem of scalability concerns the company as a whole and distributed teams.

    Company as a whole


    Agile works well at the individual command level. A single team can really seriously improve its effectiveness by switching to Scrum + XP or Kanban. But if you try to develop an idea for the whole company, very often it will meet with serious resistance and misunderstanding.

    In the first place, agile conflicts with the corporate culture of many companies. The basic principles of agile, such as transparency, trust, modesty - the business world is perplexed. Transparency? The trust? Modesty? Hmmm ... Political games, careerism, boasting and incompetence reign in many companies. For such companies, agile is perceived as a virus that threatens to break the existing system.

    Agile seeks to disrupt the homeostasis of a traditional company and brings to the surface all undercurrents, hidden imperfections, and everything that does not sink in water under normal conditions.



    Even if the company is doing well with transparency and trust, quite complex problems remain. HR should change its approach to hiring people and select those who are suitable not only for skills, but also for the culture of the company. The marketing department has to figure out what to do with frequent releases. The design department needs to understand how to include yourself in the development team. Top management needs to understand how to organize the interaction between many agile teams throughout the company. All this is very difficult and there are no answers to many questions at all. Large companies are just starting agile reorganization processes.

    Recently, UX teams and the development team have been actively joining. There is serious progress.

    As for the interaction of many teams - everything is much sadder here. Scrum of Scrum has shown its complete inoperability. There are no clear mechanisms yet, everyone steps on the same rake and invents his own bicycle.

    Distributed teams


    A long-standing agile problem is globalization. The development of the Internet has led to the emergence of distributed teams. There are companies with offices in dozens of countries around the world that have to synchronize the work of hundreds of people. Since agile was born at the level of co-located teams, he initially had serious problems with distributed teams. Continuous communication and reduced feedback is difficult when an OUNER product wakes up at 9 a.m., and developers sleepily appear at work at 4 p.m.

    The solution to this problem is complex and, first of all, should be implemented by increasing the saturation of communication channels. Direct continuous video communication between offices is the best solution. It is always nice to see today's mood on an unshaven face and to know whether a person can now be asked a question, or is it better to wait.

    Tools for remote pair programming, brainstorming and testing also appear and develop. Tools like TargetProcess are inevitable for any distributed team.

    Conclusion


    The main task of the software development industry is to get to the center. Learning to do the right thing right and fast.



    I am sure it is possible. To do this, it is necessary (in decreasing order of priority):
    • to study problem-solving techniques, such as TRIZ, to develop innovative thinking and systemic thinking
    • reduce feedback time in all ways
    • study subject area
    • learn how to scale agile-mindset across an entire company and to distributed teams


    PS Yes, I know that the article is great. But to beat her to pieces I do not see the point.

    Also popular now: