June 13, 2019 at 14:11

Performance Review and Identification of Secret Knowledge (review and video report)



    On April 26, at the KnowledgeConf 2019 conference, the report “Performance Review and Identification of Secret Knowledge” was delivered. Usually we talk about technology, however, in order to develop as a company, we are far from doing just that. This presentation, dedicated to engineers and their development, is a good example of this. If you are a team leader or think about how to ensure the growth of employees in the team, this article (and the report itself) may be useful.

    By tradition, we are pleased to present a video with the report (50 minutes, much more informative than the article), and below is its squeeze in text form, enriched with some details that were not heard in the report itself.

    Why Performance Review?


    Prior to Flant, I was the CTO at the development company. I wanted the team to grow, become able to solve more and more complex tasks, and employees improve their skills. And for this, two things were required:

    1. Transparent growth rules.
    2. Feedback.

    It was necessary to get some rules by which an employee could develop in the company and by which it was possible to give him timely feedback in order to help with problems along the way.

    Is building a career ladder easy?


    As it turned out, no. Even if you have job descriptions of a higher level employee, you can’t just take a set of keywords from it and say that these are necessary steps to enhance your career. If you tell an employee: “Learn Symfony 4, PostgreSQL, and MySQL,” it won’t help him much. Do not teach him the whole of MySQL! Details are important, so some means of measuring a person’s skills will be needed ... something like a ruler .



    Imagine a ruler, but instead of divisions, it has qualitative levels of skill that can naturally be distinguished. If we had such a line, we could tie real tasks, real experience and specific training materials to it. So?

    Unfortunately, no: in practice, such a line turns out to be a separate topic for the “holy wars”, since it is impossible to say with mathematical precision whether a person knows how to implement complex business logic or not. But there is good news: in the same practice, mathematical accuracy is not so necessary . You can use the method of expert evaluation: to give the most experienced and most immersed in the situation the employee the opportunity to make a decision about belonging to the level of skills, and from this already build on.

    We will have to turn a blind eye to poor accuracy, to the mediocre scalability of the method and much more, but at the same time we can admit that this method copes quite well with its task. Not only that - to this day it is used very much where. Often a team leader can very well act as such an “expert” in his team, and this is approx.

    Is it easy to give feedback?


    It sounds logical that you need to periodically (for example, once a quarter) meet with an employee and give him feedback. And try to organize it so that the process is filled with meaning, and not a cargo cult. It seems like a good idea to take our “lines” with skills:



    ... organize an N-dimensional space from them and tell the person where he is in this space, and then agree on where he will move next: it’s



    hard to imagine an N-dimensional space, so we can use these goals table. The rows in it will be our skills, and in the columns will be the level of development of the skill. If we hold two such sessions, we can see approximately the following picture:



    ... which, it would seem, will convince us of the fidelity of the chosen path and the chosen tools - after all, people are developing. And when the critical mass accumulates changes, you can raise the employee’s salary and change the nameplate for his position, for example, saying that he is now Middle PHP Developer.

    But how does everything happen in practice?


    In theory, it sounds cool and transparent. I made attempts to start the mechanism in this form - and they were ... frankly, not really. Known to me, the attempts of my colleagues in the workshop were not inspired either. The main problems are:

    • Not all employees respond equally well to feedback . It is easy to find yourself in a situation where the question of lack of authority arises sharply, and in such conditions it is difficult to give an expert assessment.
    • Highlighting the key skills that are assessed in practice turns out to be a non-trivial story - especially if you are a person “from the plow”. It is difficult to disengage from the details and strike a balance between accuracy and generalization.
    • The presence of such a “feedback” and “ladder” did not correlate with the motivation of employees. If they developed, then rather contrary to the system than thanks to it.

    Flant's Need for Performance Review


    When I came to Flant, I saw a number of problems that made me think about Performance Review, namely:

    • Engineers did not have an understanding of why they should grow, in which direction they should develop. This question was sometimes voiced, but an intelligible answer could be obtained quite rarely.
    • Engineers did not feel involvement in the company, did not feel the return from it and its participation in their career.
    • The company began active growth, and in these conditions it is especially necessary to build understandable and working mechanisms to turn novices into experienced engineers and managers.

    Therefore, I really wanted to highlight the skills that are needed for each of the positions, and create conditions for career growth.



    But everything was not so obvious ...

    Looking for a list of skills


    It turned out that no one can highlight the requirements for the skills of DevOps engineers and generally highlight any “skill levels”. The closest description of this position is in the immortal Robert Heinlein:

    Any person should be able to change diapers, plan invasions, slaughter pigs, construct buildings, control ships, write sonnets, keep accounts, build walls, set bones <...>, program computers, cook deliciously, fight well, die worthy.

    That is, team leaders expected and raised their people as fullstack engineers, able to correctly find out the details of the task and solve all technical issues, and bring it to production.

    Of course, it inspired me. It’s great to work in a company where everyone is a universal fighter, a versatile personality and an excellent engineer! It was only darkened by the fact that everything looked like a huge dark cave with secret knowledge, into which it was terrible to enter . I did not want to doom the engineers to such trips.

    As you might have guessed, dozens of technologies with hundreds of nuances were under the hood. We do not limit our customers in choosing the technological stack for their applications, so our engineer is faced with a wide variety of databases, queue servers, frameworks and languages, diagnostic, development and debugging tools, as well as features of different data centers and platforms.

    Over the past 5 years, I have witnessed a fundamental change: if earlier it was almost considered the norm when the technical director stated: “We are working on platform X with the database Y and we will not work on anything else,” now some service stations may even let go stack control, relying on microservices and the ( rather controversial ) slogan: "If it breaks, it's easier to rewrite everything from scratch."

    Compose in these conditionsa list of technologies for evaluating employees is not a trivial task. And to keep it constantly up to date is too laborious and difficult to organize. Therefore, we tried to find a solution and want to share it.

    Rethinking PR


    We decided to try a different path, but first, rethink our actions and expectations. By launching Review, we would:

    • ... tried to make performance measurements of employees;
    • ... talked to them;
    • ... they expected that in the end they would become happier, better understanding their prospects and their expected career growth.

    We invested about a month and a half to rethink these points and below I will tell you what came of it.

    Performance rating


    I happened to look at something like this in an attempt to understand what might be wrong here:



    And at some point an interesting thought came: an excess amount of information is shown here. In fact, specific “divisions” are not so important - the picture can be simplified to this:



    We are interested in the speed of development, and not the fact that a person has reached a specific level. Was there any development at all? Has a lot of people learned or little? That's what a real indicator!



    Investigating the secrets of the skill list


    Earlier, we came to the conclusion that the list of skills is a secret knowledge for us: engineers are able to do something, do something every day, but it is difficult to formulate exactly what.

    One way to reveal secret knowledge is “in the wake”, that is: fixing what is happening, then to analyze it.



    Just as we can follow the footsteps in the forest that squirrels, foxes and bears are found there, we will try to reveal secret knowledge ...

    Pilot project


    Timlid Peter kindly agreed to participate in the pilot project to launch Review in an updated form. Peter delved into ideas, and we planned meetings with employees. As a result of self-preparation, Peter formulated this kind of feedback about one of his engineers - Leonid:



    This is an example of poor-quality feedback: if you heard about yourself in such formulations, it would hardly help you much. Even if everything is more clearly formulated in the head of the timlid (which is completely not a fact), recording information in this form is not very useful, because after six months such “traces” will not bring any benefit - even the author will not remember what it is about.

    We would like the feedback to be as objective as possible, polite and rather help to learn more about each other than to be a means to “motivate”.

    Leading PR training


    As a result, we formulated the following rules for preparing for PR for team leaders:

    1. Before each review, you need to allocate from half an hour to an hour and a half to prepare. It is better to prepare the day before, and not on the same day.
    2. Go through all trackers (ticket systems, instant messengers, karma systems, etc.), even if the leader is too lazy and he resists, appealing to his beautiful memory. View notes from a previous review.

    3. Write out a number of points for discussion. These abstracts must contain:
      • Facts, with justification in the form of links to trackers;
      • The personal attitude of the reviewer: an explanation of why he (a) highlighted (a) this item, and what is so important in it.



    By the way, we have posted examples of bad and good wording here .

    PR process


    The review itself is always full-time . Despite the fact that we work remotely, we can’t talk about any review through documents / questionnaires / services. In our case, communication goes through Google Hangouts and there is a team leader, engineer, HR and, if necessary, other interested parties.

    A review goes through several stages:

    • Talk about the distraction to tune in to a conversation with each other.
    • The story that a person did good , for which I want to thank him and explain why this is important. After all, we want that good to be done further?
    • Denote unambiguously bad : violation of agreements, discipline and other things that you consider unforgivable and that are indisputable.
    • Discuss incomprehensible or controversial according to the following rules:
      • Identify the facts and personal attitude to these facts.
      • Ask a general question (like “What do you think about this?”) To get more information from the employee about the problem.
      • To ask the question: does the employee think that there is a problem? It may turn out that for some personal reasons he does not consider this important. We need to make sure that he shares our perception.
      • To ask the question: what exactly is the employee himself going to do to solve the problem?
      • If an employee does not share your problem, offers inarticulate solutions - it is highly likely that he will not do anything with it. And it makes no sense to put pressure on him in this situation - it’s better to try to understand why this is the case.
    • Ask for feedback from an employee. Did his life in the company during this cycle match his expectations? If you were really able to build a dialogue before, then at this stage you will get feedback, not a slurred moo, as is often the case.
    • Talk about salary : do you raise it, and if not - what exactly needs to be fixed in order to raise it. The issue of salary is very important and complex and in general is the main motive for an employee to participate in Performance Review. There is a suspicion that it makes little sense to try to start some review processes, if this does not affect the salary, but this is not 100% ...

    Total:



    First year results


    The analysis of the records accumulated during the first year helped us:

    • to open the veil of secrecy of the identity of our engineers;
    • replenish the knowledge base with instructions and information that cause the most problems for employees;
    • improve the climate in teams.

    Engineers began to grow much more tangibly and quickly , they had an understanding of the direction for building their career.



    Other significant moments:

    • Beginners began to solve problems in a more constructive way when they adapted: managers suddenly found for themselves that they could talk to the newcomer about problems ... and he would be corrected!
    • It became clearer what kind of employees we are waiting for in the company, so the requirements for hiring were adjusted.
    • It became more obvious to the employees themselves what they might want and what development prospects they have.
    • Sketches appeared on the matrix of competencies, and there is also a mechanism for its constant updating.

    Engineer portrait


    It may seem obvious that an engineer, for example, should really love what he is doing and enjoy working ... Anyway, here are four skills that turned out to be the most demanded (by a wide margin):



    Other important skills were:



    Let's try to find out in the second year


    Meanwhile, an attentive to details reader could notice that we didn’t measure performance very much ... Perhaps, over time, the list of skills will settle down and a suitable competency matrix will be formed, or we will introduce some other rating system.

    In addition, right now we are looking for ways to stream the training of conducting a review among managers - we still do not have a ready-made solution.

    A separate complex issue that is not so easy to formalize is the relationship between the assessment system and the issue of employee salaries. It seems to us that this issue is directly determined by the business model of the company, and the solution for one business may not be very suitable for another. (We already talked about our model on the hub: we are built almost like a franchise, and as the team's efficiency and competence grows, it gets money for salaries.)

    Videos and slides


    Video from the performance (~ 50 minutes):



    Presentation of the report:



    PS


    You might also be interested in the following publications:


    Among other reports on our blog:

    Tags:

    Also popular now: