Trolling Detector: How We Improved Productivity with a Speech Analyzer



    In the modern world, corporate trolling is just as common as spam. In the rating of “time eaters”, trolls are in the first place, and the point is not only that there are too many of them divorced, but also that, unlike spam (against which very effective tools have already appeared), corporate trolls mimic for ordinary office workers, and topics for trolling are based on what, in fact, is the work. Therefore, it is sometimes very difficult to distinguish the employee’s simple desire to “troll” from the substantive productive dispute.

    The Mail.Ru Group internal resources efficiency department has found a way to distinguish trolling from productive communication: we created the AntiTroll hardware-software complex, tested it and made sure of its effectiveness. Details under the cut.

    First of all, one cannot fail to mention that communication with the troll reduces the effectiveness of not only the troll itself (which, like a swamp, is becoming wider with fresh water pouring into it), but also of the employees who participate in the discussion. Therefore, a decrease in effectiveness is often of an epidemiological (viral) nature.

    In order to determine the level of trolling, we plunged into discursive analysis, studied the language used by our team members (we wrote about the terms used by our developers here .

    During this analysis, both the form of the language and its function were considered. As a source The material was taken in colloquial speech and written texts in which we identified the linguistic features of the perception of various texts and types of oral speech.

    The analysis of written texts included the study of the development of the topic and the semantic connection (coupling) between sentences. The guys from Mail.Ru Search helped us a lot in this direction.



    The analysis of colloquial speech was based both on the above aspects and on the practices of step-by-step interaction, opening and closing sequences of social interactions or the structure of narrative.

    The working group developing the AntiTroll mathematical apparatus had to seriously study psycholinguistics, sociolinguistics, sociology, hermeneutics, social psychology and even anthropology (as it turned out, such parameters as gender, height and acoustic properties of the articulatory (speech) apparatus should also be taken into account when processing information )

    In parallel with the development of the mathematical apparatus, we collected data for analysis.
    Our usability laboratory was connected to this. After a short period of adaptation, the experimental group of the game development department employees started talking about their common project, and with the help of a decoy troll, the discussion moved from the stage of effective communication to the stage of low-efficiency trolling on a traditional hollywood topic (for example, Android vs iOS or Windows vs Linux) .

    Gigabytes of data were collected by the registrars of pulse, respiration, activity of the cerebral cortex, microphones and video recorders. This initial information was fed to a script written on the basis of the above mathematical apparatus.

    The result of the script should be a single number - the level of trolling.

    After 28 builds of the script, we were close to abandoning this idea: the script produced some strange values, rather resembling the result of the random number generator. And then we decided to add another weight vector to the mathematical model: “level of emotional involvement”. It is precisely called the “subjective appraisal of existing or possible situations.” And suddenly, on the 29th assembly, “it” worked!

    The script began to determine the trolling level quite accurately, and when someone moved from the stage of productive discussion to the stage of verbal procrastination, the numerical value of the trolling level steadily grew.

    Over the past 7 months, we tested this system in 5 departments of Mail.Ru Group, gradually adjusting the script, and at the moment we have achieved the accuracy of determining trolling in 96%.

    It is interesting that during testing, an analogue of the "Ballmer peak" , known to most habrayuzers, was revealed. Up to a certain value, trolling almost does not affect productivity, but when the value of X is reached, developers have a sharp increase in productivity (the number of commits increases, the number of bugs decreases). However, when threshold X is exceeded, productivity drops sharply.

    Whether this discovery has practical application is not yet clear: it is obvious that it is quite difficult to maintain the level of trolling in the work team at the required level to achieve this peak.

    The next step was the creation of the AntiTroll Alarm! Hardware and software complex:



    The principle is quite simple: a microphone placed in the meeting room transmits an audio signal to a constantly running server with AntiTroll script, and if a high level of trolling is detected, then to Arduino, connected to the server via USB , a signal comes on that the “AntiTroll Alarm!” relay is on. As soon as the trolling level in the discussions rolls over, a siren and a flashing light start up:



    While we tested our new system in several units for three months, we already see positive results: the number of commits increased by 23%, the number of bugs found by testers decreased by 14%.

    In the near future, we plan to introduce the system in all divisions, and then completely make mobile applications that can be used not only by us, but also by colleagues in the market.

    Just imagine - before the planning session, put the phone in the center of the table, turn on our application, and don’t worry about missing the moment when your discussion goes into a non-constructive direction ...

    The source code will later be posted on github here .

    Also popular now: