Scrum. From chaos to order and high productivity

About what Scrum is and with what it is eaten, millions of articles have been written. However, most of them assume that there is a certain vacuum before scrum, or vice versa a tough environment with PMBOK and other processes. Many authors write about the “zero sprint” in the beginning project, about choosing the perfect team, about choosing the length of the sprint, but I didn’t find a lot of articles about the implementation of Agile methodologies in the existing environment, in which there were no methodologies before, but there were already established traditions.

Two and a half years ago, when the team and I (more precisely, it was not there then) started developing our product, we did not think about the methodologies, processes, and other bureaucratic issues that seemed to us then unnecessary. Time passed, there were more products, the team grew. Gradually, everyone began to realize that some kind of chaos was forming, which was becoming increasingly difficult to control, and most importantly, which seriously limited our capabilities. In fact, imperceptibly for us, the situation was approaching critical.

Under the cut is a long real story of Scrum introducing into a development process that was going through hard times. I hope this story will be interesting to you and, possibly, will help you to decide or solve some problems.


To better assess the situation:
A little about who we are
We, the technical team of Sputnyx, are developing a set of systems for automating the work with online advertising. At the moment, we fully cover the needs of clients for managing contextual and targeted advertising, and also offer a number of small tools to automate routine tasks.
The team consists of five people (1 front, 3 backs and I am the product manager) + one DevOps engineer, providing us with first-class server configuration.
There is also a separate team to develop our RTB platform, but it does not appear in this story.


So the situation:


Autumn 2014. The team supports two small systems and is actively developing the other three, combined into one complex with a common architecture. In fact, this is not even a team. In the process of development, historically there were people responsible for each system and only dealing with it. Support for two small systems and the development of one is done by one programmer, two more systems are distributed among two other team members, and finally one front-end developer is developing a single interface that all systems can use.

The code base at that time was already hundreds of thousands of lines, there were minimal tests, no automation. Git-e used two branches of master on production and dev in development, naturally there were a lot of problems with large merges, etc. Gradually, the speed of introducing new features dropped to almost zero. The management asked me to somehow evaluate the terms, but in fact, any estimates were a finger in the sky.

Another big problem was that we could not even somehow balance between projects and priorities. When trying to allocate more people to system A, suspending, for example, the development of system B, we were faced with the fact that the programmer absolutely did not know someone else's code, had little idea of ​​the business tasks of the system and, in fact, slowed down development for a long time.

Why was everything so bad? Everything is simple. When we started, the main thing was to do everything quickly, and as you know, quality always suffers in the pursuit of speed. At heart, everyone understood that this was wrong, but it seemed that there was no time to think about it. Yes, we ourselves are to blame, but that is, that is.

It was clear that something had to be changed.

Start


The idea of ​​using Scrum has long been in the air. Its main advantages were exactly what we needed: a short release cycle, flexible backlog, quick feedback. However, there were problems. Firstly, we have at least one complex, but of different systems, each with its own backlog, its own priorities and there is no way to provide each system with its own team. Secondly, programmers are a very inert class of employees, they are used to working as before, and although they themselves understood that not everything was smooth - they were afraid to radically change something. Thirdly, Scrum is not only the project management, it is autotests, CI, clean code and much of what we did not have at that time.
But the time has come, it was clear that it was no longer possible to procrastinate, and we began to prepare to globally change the way we work.

Implementation


To begin with, it was necessary to solve the main problem, without which no next steps seemed possible.

Exchange of knowledge
We have been preparing for this for a long time. We completed all the major tasks that were started at that time, made descriptions of the business functions of the systems, and each developer prepared a story about thin or non-obvious places in his code.
When everything was ready, for two weeks we suspended any new developments, only a bug fixing and only in extreme cases. For two weeks, each member of the team presented their code, and I talked about user needs in great detail.
For two weeks, people studied the code, tried to fix "alien" bugs and asked a lot of questions.

When this was over and everyone at least had some idea of ​​what each project was talking about, it was time to change the process.

It should be mentioned here that at the beginning of everything I was both a Scrum master and a Product owner. I perfectly understood that this was not right, but at that moment, it was simply impossible to work otherwise. I was the only one from the team who knew the principles and rules of scrum, but I also had to present the product.

Process

If everything went relatively well up to this point, then the problems started. The programmers generally supported Scrum, but when it came to planning, they started to object like “Why spend so much time talking?”, “We’ve been sitting in a chat room all day,” but of course everyone was shocked when I spoke about Daily Scrum - “Chatting every day is terrible,” “Everyone can watch everything in a jig / bit bag”, “If anyone is interested, he will ask.”
It was clear that something needs to be introduced gradually, and the point here is not that the process was imposed “from above”, no, people wanted changes, but this was beyond their strength.

We started by introducing sprints and yes, planning.
We combined the backlogs of all subsystems into one and tried to prioritize. Our first user-story was an old task, they could be purely technical and so on. Evaluated in story points, based on the scale of degrees of two, but of course, the initial ratings varied significantly. Our first sprints lasted a week, some did not approve of such short sprints, and such frequent “chatter”, but those who supported me understood that even for a week we couldn’t plan everything exactly yet, there was no question of large sprints .

We started our first sprint on Wednesday evening and decided that a retrospective would be in a week on Wednesday morning, so it would take half a day, then planning and a new sprint. Of course, there was no question of any demos then, we essentially had nothing to show, moreover, this is another point of increasing the emotional load on programmers.
So, on Wednesday morning, coming to a retrospective, we all together realized that we had a few more sprint tasks in our work that “left to finish a little” (in fact, a little could mean a whole day of work). Here one more misunderstanding was revealed, in retrospect we had to discuss problems and opportunities to improve our process, but programmers unanimously declared that “there are no problems”. They did not feel that they did not have time, but today they are discussing everything, and then they will finish it. There was no sense of cyclical process.

For several weeks we continued to work in this mode, I tried to draw attention to the fact that the sprint is just an iteration, and not just a segment on the way. In the end, a less good solution was found. We divided retrospective and planning and put them apart on different days, on Tuesday retro, on Wednesday planning, between these meetings, programmers solved different RND problems, those that were still too early to evaluate, because it was not clear what to do. But most importantly, between retro and planning it was forbidden to work on the tasks of the past sprint. At first, I didn’t like this either, because the incomplete code remained and was forgotten, however, this measure paid off. The team began to realize that the tasks that were not completed in the sprint were not really completed, they could be forgotten, left indefinitely, etc.

In the meantime, it was necessary to pay attention to working with the code. Previously, tests were written only for key functions and, in fact, they covered an extremely small part of the code. From this moment we decided that every piece of code that affects the problem to be solved should be covered with tests. Here, too, there were moments, for example, the first five sprints in planning, it was often necessary to remind when evaluating "And with tests this task will take how many points?". But it was still implemented quite simply, everyone understood too well why this was necessary.

Another big misfortune was that, despite two weeks of study, people still did not know well “foreign” projects.
Firstly, in planning the task for project A, the person who developed it rated 4 points, and the others 16 and 32. It took a long time to explain that the assessment should take into account the fact that a person who was previously with this project may have a task did not work.
Secondly, the developers began to actively take advantage of the fact that Scrum invites team members to choose their own tasks. Thus, the programmer usually tried to take on himself a task according to his own project, so as not to bother with someone else's code. For several weeks, we had to make a rule that you are forbidden to take a task from "your" project, if there are others on the board. There was dissatisfaction, but I must say, it quickly came to naught as soon as people had studied the entire code enough.

But of course the main thing, and certainly the best thing that happened during this transition, is that a few good guys who used to just work at the same table finally began to become a team. Prior to that, they had only gone to dinner together, and even then rarely, and then real mutual assistance, joint work and responsibility began to arise.

Development


Slowly but surely, we were moving towards the right and good process that helped us work.
After a couple of months, we introduced the Daily Scrum (stand-up), assigned it at 12:00, because by that time, usually everyone was already at work (we have a very flexible schedule). For a long time at the stand-up, everyone simply called the tasks that they worked on, and this was not very useful, but a certain adaptation period passed and the team began to really exchange useful, up-to-date information about problems and progress.
Sprints became two-week, we learned how to better evaluate tasks and were able to afford it. Estimates on planning began to coincide in most cases, and when they did not coincide, we could discuss everything and, most often, come to a common opinion.
The percentage of purely technical tasks in the backlog gradually began to fall. And increasingly, the task was User Story.

The weak point in the process was still a retrospective, on most of them the team could not offer topics for discussion. “We are doing well” is a fairly common statement. Acting as a Scrum master, I read a large amount of materials on this topic and began to use some techniques, but there is still a big difference between theory and practice, for example, the standard method for finding the root problem is very good, but in practice, it showed that we have a problem in the indivisibility of some tasks that we already knew about, but not a single cunning method could help us solve this problem. However, nevertheless, over time, the guys began to show an initiative, which certainly was extremely useful.

We started using Git Flow, set up Jenkins to run auto-tests at every commit, began to measure code coverage and try to improve it. The number of hotfixes, which in the beginning could occupy the floor of the sprint, began to decline. The quality of the product, and most importantly the mood of our users, has improved significantly. We analyzed where we have weaknesses in the development process and figured out what to do with the long Code Review.

Our days


More than six months have passed since we started this epic. Today we are still developing and are going to do it forever. However, we can already say about the results.

  • We work as a team on all tasks, which means that if someone gets sick, no project will be left without support.
  • The development speed in the number of implemented features over the period of time has increased several times, when compared with last fall.
  • The number of bugs and hotfixes, at the moment we have from zero to one per sprint, which indicates that the quality of the system has increased by an order of magnitude.
  • Over the past six months, we have raked backlog from accumulated tasks, removed unnecessary ones and now this is a full-fledged dynamically changing list of features that we will really implement in the next quarter.
  • In addition, thanks to realistic estimates, we, knowing the speed of the team, can build longer-term plans and calculate what we’ll do about the next six months.


We still have a lot of work ahead of us. Recently, we finally got to the point where it’s time to introduce a demo, let's see what this leads to. In addition, we understand that we are still far from ideal and can work an order of magnitude faster and are trying to achieve this. However, the main thing that I can say that none of the team regrets that we started this, and despite many difficulties, many of which may have also been related to our inexperience in agile processes, we still achieved order in processes and actually escaped from a situation in which we were led by the pursuit of speed and fear of the bureaucracy.

Thanks for reading, I will be glad to answer any questions, as well as accept any criticism.

Also popular now: