Big big data kitchen. Part 1

    It is time to share our experience of organizing the development process in the fashionable topic of "Big Data." In the telecommunications industry, Big Data has considerable hopes for new niches, products, and, consequently, revenues. True, many telecommunications companies prefer to buy ready-made solutions in the field of Big Data, rather than developing their own expertise. Since 2013, MegaFon has taken a different path, relying on a team of strong Big Data specialists capable of effectively solving very difficult tasks.



    There is a law



    Before you dive into Big Data, you should familiarize yourself with the legal framework that governs their processing. The federal laws “On Communications” and “On Personal Data” prescribe in detail the data processing procedures that are accumulated by the mobile operator. This information, in accordance with the law, must be protected - therefore, information about real clients and their transactions cannot be used to debug and test the code just written.

    From particulars to general


    When “internal clients” and job performers sit at adjacent tables, this provides instant and natural feedback. We add here not always a clear statement of tasks, the desire of the customer to make adjustments "on the go", and we will see that SCRUM becomes the best method of work. He served us faithfully for almost 12 months until the end of 2013. Development went quickly, while we had the opportunity to experiment and produce an effective product. A single team of customers and developers in full force participated in the discussion of tasks, in the decomposition of requirements for technical components. As a result, customers saw how their ideas were implemented in code, and developers solved interesting problems by combining and performing the functions of analysts, architects, and designers.

    Meanwhile, the tasks from the customer were multiplying, requirements were becoming tougher, backlog was accumulating, technical debt was growing. In addition, the requirements for reliability and performance of already implemented solutions have been tightened. It was then that SCRUM began to fail: sprint planning became so long and complicated that it disrupted the team’s rhythm. Nobody wanted to take large and complex tasks into the sprint; I also did not want to cut tasks. For a large number of tasks, the team stopped seeing the big picture, momentary tasks began to distract people and reduce productivity. The time has come for the Kanban method.

    We rebuilt the workflow, again engaging the “internal customers” themselves in the discussion, and established a constructive dialogue: tasks were evaluated all the time, and not just during sprint planning. At the same time, there was a lack of an analyst who would filter the flow of customer requirements, and the size of the team somewhat did not correspond to the volume of tasks. As a result, we invited trainers, we needed a consultation of people who made training in lean-technologies their profession. After communicating with them, we came to the process scheme that we use now.

    Error handling



    Perhaps the main problem was a large number of tasks (regardless of the amount of work to solve them), which extended the deadlines. The customer was dissatisfied, the transparency of the process left much to be desired. Another problem appeared: feedback from the customer did not work well. He was not interested in how his tasks are decomposed, in what order they are taken into work, what and when he will receive the output. Hence the lack of understanding of the actual timing of the work. The third problem was that the developers themselves began to lose their understanding of how a particular task affects the overall goals of the team.

    Remaining committed to a flexible approach (which ensured speed of work, flexibility and prompt response), we changed our process this way:

    • 1. Formalization of the requirements collection process, backlog management

    We agreed on the general rules for maintaining and filling out the backlog, when all requirements are collected, and the analyst places emphasis.
    This gives customers confidence that their tasks will not be lost, and developers the opportunity to receive correctly described tasks with a pre-set priority (in the future, this will help to avoid unnecessary iterations to redo the functionality).

    • 2. Story Mapping

    We began to actively use User Story Mapping, this tool has become a good tool for involving customers in the discussion of tasks and their decomposition. For all participants in the workflow, it has become clearer what is being done to perform a specific task and how this decision will affect the final product.

    After successfully applying Story Mapping, we come to the Impact Mapping tool. It will allow you to globally evaluate the impact of each refinement on common goals and to emphasize.



    • 3. Introduction of service classes

    We identified several areas within the development process and divided into classes all the services developed by different teams. This allowed us to isolate the development teams, thanks to which the programmers were no longer distracted by the momentary tasks of other projects, and customers and analysts understood the capabilities of each development area and set their goals more efficiently.

    • 4. The exact "Kanban"

    Trainings were held for all employees (customers and R&D), at which they were told about the methodology, basic principles of work and tools to use Kanban with maximum efficiency.



    • 5. Limiting the number of tasks that are simultaneously in work, including by service class

    This key practice of Kanban has allowed us to increase the transparency of development processes for customers. We calculated the average time to complete the tasks and calculated the maximum number of tasks in each direction, which is necessary to meet the requirements of customers. Therefore, in order to guarantee the quality and terms of work, we consider only the tasks that we can handle exactly.

    • 6. Weekly meetings with customers

    We could not completely abandon the weekly planning, therefore we left weekly meetings with customers at which the statuses of development stages are discussed, it is determined which tasks are transferred from the backlog to ToDo, and completed and tested work is accepted.

    Key meetings :
    • Planning
    Participants: Customers, Leaders
    Purpose of the meeting: filling TODO from Backlog
    Weekly
    • Acceptance
    Participants: Customers, Leaders, Team
    Purpose of the meeting: Accepting the results of the team
    Weekly

    • 7. Daily meetings (stand-up meetings) for development teams

    We began to arrange weekly meetings for the whole team. They were arranged earlier, so that the whole team was aware of the general processes. Now emphasis is placed on technical tasks, it is determined which of the developers will conduct specific tasks, which previously completed work is being closed, development plans are being formulated. It has become a really effective tool.



    To be continued.

    Also popular now: