Visiting Odnoklassniki

    Recently I managed to visit the Odnoklassniki company’s Riga office - I will talk about the office a bit later, and in this post I want to retell conversations with a number of company employees. I'll start with Elina Parkhomenko , she is the director of product development.

    image


    Elina and I met in a video conference room. At the very beginning of the conversation, she emphasized that she was not involved in deeply systemic things - she was engaged in setting the development vectors of the project, determining the work of various services, and directing the development. However, Habr’s audience still didn’t ask purely technical questions :)



    imageHonestly, I now have almost no time for “meetings” on social networks, but before I left for you, I went to the site and was very surprised at how it has changed since I used to. The site has become more sophisticated, "overgrown" with all kinds of chips - what was the path to such solutions?

    Elina: I came to work for the company just when everything was “simple” on the site, but you need to understand that without complicating the interface, the appearance of innovations is often simply impossible. In general, we try to maintain the project development course and the general concept - there are several aspects.

    First, the social graph is important. All that is not done is done for the sake of users, for their interaction with each other, to strengthen ties within the graph. It is important not only the development of the graph, but also its growth - the priority is to attract new people from outside. People are the most important.

    The second - it is necessary that these people have fun and interesting, that they have an incentive to enter the project.

    Accordingly, all services are developed from these two considerations.

    imageAnd what, in your opinion, keeps people on the site?

    Elina:As practice shows, users are most in demand face-to-face communication with other people, less often - communication in groups. And in this regard, we are trying to make it convenient for users to communicate in one form or another - here you have personal messages, discussions, “guests” and even branded “ratings” (a favorite feature of users), which often lay the foundation for new dating. These are the features of the site that we are trying to improve and make more convenient - recently, for example, they made messages “instant” so that users receive messages immediately (and, accordingly, stayed on the site), and not ex-post. From the same “speedy” considerations, we made a toolbar that always appears at the top of the browser - a kind of user dashboard.

    The same with the content part of the project - people upload photos for the sake of feedback (for example, comments and ratings), so it was logical to make a functional upload of photos and opportunities for their subsequent discussion, without any restrictions.

    That is, by and large, we help to realize the natural need for communication. And when users are comfortable and interesting, they remain on the site.

    imageHow do you imagine the audience of the project when you are developing this or that service?

    Elina:It is very important for us that the service is understandable even to the simplest person, not to mention advanced users. Therefore, all the services that we launch always have basic functionality that is understandable to absolutely everyone. This already in the future it “grows in ruffles” for narrower audiences, but initially - everything is for everyone.

    Although, as a rule, we do not make radical changes and we haven’t had any stories like “return the wall”. Therefore, users are quite calm about all the innovations.

    By the way, when we launch a new service, at the testing stage we have a special button “share feelings” - any user can use it to express their opinion on any aspect of the innovation. All appeals are tracked and taken into account.

    And they write a lot?

    Elina: If you don’t like something, then a lot. But this is vice versa good - constructive criticism is not superfluous, and we always listen to it. There are those who are dissatisfied with absolutely everything, and sometimes vice versa - they love us no matter what we are.

    imageI heard that over the summer you had more than 50 various releases - how is the process of creating new site features going on? If someone came up with something interesting (beyond the scope of the current project development course) - what will be the fate of his idea?

    Elina:We have several managers, each of which, as a rule, has 4-5 interconnected (ideologically or functionally) projects. A group is assembled to create each innovation - programmers, designers, testers are taken. When the team is formed, teamwork begins: the idea is discussed, finalized, visualized and begins to be implemented.

    If the innovation is really interesting, we always find a place for it in terms of project development. Anyone can offer a feature - a user, a tech support employee, or even an administrator.

    imageHow are new services tested? Do you collect any statistics?

    Elina:First, everything runs on a virtual machine - with the joint efforts of the company we test the service for strength, look for bugs, fix them, share our impressions. We make the appropriate amendments.

    Closer to the release, a test group joins to test the load and perception of the product. And only then - on production. Sometimes innovations are launched as an experiment, that is, for a small part of the audience or for the whole, but for a short time. After that, we carefully study the statistics, loads, reaction in the support service. If there are no problems, then we launch it completely - the system is very “live”, so it is important not to break its components or the connections between them. Later we monitor the effectiveness of services.

    imageWhat kind of violations do you encounter most often with spam? How do you fight?

    Elina:On the contrary, we now have very little spam - either we really defeated it, or THEY are still on vacation;) But, unfortunately, we can’t talk about methods of dealing with spam.

    The most common violation is all kinds of groups spreading something malicious. But we managed to defeat them with our response mechanism - a symbiosis of user moderation and developed algorithms. Now these groups simply do not have time to “blossom”.

    imageThat is, you do not have an army of moderators?

    Elina:Right. Social responsibility for Odnoklassniki means a lot - we, of course, have moderators, but they simply would not have done the whole amount of work. Users themselves, who have simply unique responsiveness, come to the rescue in this matter - apparently, people want their home to be cleaner. So, for example, all uploaded photos are sent for user moderation to the N-th number of people who are very likely to filter out all the "extra" - now all (public) photos are moderated. Popular moderation is a very important component on our project and in the photos section it works like a clock.

    imageAnd how many users are involved in this?

    Elina: Over 5 Million. Imagine if several moderators did this work ...

    imageSurely on your way there were “fakapy” or vice versa, “surprises of good luck” - share some story?

    Elina: The story of the abolition of paid registration comes to mind, when we paid all the users who paid for it free of charge the opportunity to put a rating of “5+”.

    Then the “wave” covered even uncommon profiles - users began to scamper around the site, visiting each other and putting the highest score to the right and left, literally everyone. Of course, we expected to increase activity, but after an hour due to the high load, we were forced to withdraw the "freebie" from a number of users and began to distribute it in portions, with a certain interval. What, as you know, caused the explosion of applications in the support service: "why he was given fives, but I don’t ?!".

    imageAnd by the way, what was the main reason for introducing paid registration? Karma fell victim to banal monetization or was it a fight against spam?

    Elina: Before that, there was a lot of spam, fake profiles, bots and other fraud - no one knew how to deal effectively with this. After the registration was closed, we came up with and implemented a number of algorithms that allowed us to get rid of unscrupulous accounts. Well, of course, they came up with a lot of innovations so that it would be interesting for newcomers to come - all this, after the registration was opened, ensured the intensive growth of the project. Now we have more than 102 million "confirmed" (by phone) grateful users who can safely communicate with each other.

    imageCan I see the statistics somewhere?

    Elina:Yes, we are one of the few who have statistics open on the same LiveInternet.ru. On average, it is 20-23 million visitors a day, the bulk of which come from bookmarks.

    image

    imageTell us a few words about the life of your employees?

    Elina: Everyone’s working day starts at different times - the people who usually work in the group agree on what time they will arrive at the office. As a rule, it’s 10 in the morning, although UI employees arrive at 8.

    About 100 people work in the Riga office, 25 of which are programmers. The youngest employee (he is a developer) has recently turned 20 years old, the oldest (manager and designer) almost 40; the average age is up to 30. Everyone speaks Russian.

    Internal culture is not much different from other companies, although there are some features. So, for example, our employees take the initiative to celebrate any events - whether it’s buying a car, birthday or the end of a trial period. The probationary period, by the way, is 3 months, but if we see that a person is good and is doing his job, then we do not wait.

    imageAre there any female developers?

    Elina: Well, where, where, and we have, there’s more than one!

    imageWhat, directly write the code? What language do they write in ?!

    Elina: Like everyone else, in Java! By the way, especially a lot of girls are testers, and they are very good at doing their job.

    image

    imageAnd how is the integration of new people into the company?

    Elina:Regardless of which office (Riga, Moscow or St. Petersburg) we are talking about, newcomers undergo some kind of training. Various specialists (as a rule, leading developers, backend programmers and UI designers) tell important features of the project - about the basic principles, to the architect, design - we don’t throw anyone under the tanks. Moreover, such training can be carried out between offices - we constantly fly to visit each other.

    imageInteresting. And finally - maybe tell us something from the plans for the future?

    Elina: I can’t say anything concrete - while we are working on optimizing the project so that everything works for everyone quickly. We are developing a new mobile application. Well, in a relatively short time we plan to make a complete redesign.

    Architecture


    After a conversation with Elina, we met with Andrei, the project’s system administrator — he told us about the changes that have occurred since the March publication. If you missed, I advise you to read it, if you forgot to refresh it; an interesting article - Architecture and the Odnoklassniki project platform

    The main changes (>> it was / - it became):

    >> Up to 2.8 million users online at peak hours
    - Now this figure has increased and amounts to about 3.4 million.

    >> 7.5 billion requests per day (150,000 requests per second during peak hours)
    I will not say in billions from memory, but now the number of requests per second has increased to about 250 thousand.

    >> Network traffic at rush hour: 32 Gb / s
    At the moment, this figure has grown to 52 Gb / s

    >> 2,400 servers, storage systems
    Now the number of servers (and storage devices) has exceeded 3,100. Moreover, if there were three data centers, now four are all located in Moscow. Channels between data centers have also been expanded.

    imageHave server configurations been updated?

    Andrew: The servers themselves were not updated, they increased their number. The average configuration is a dual-socket system with quad-core processors, 1-2 disks and a memory capacity of 8 GB or more. The backup concept has not changed.

    They began to use more memory and more cheap iron. Especially for storing binary data, as launched a new software solution (we talked about it in detail on Highload).

    From memory: now a maximum of 96GB on the system that serves the user tape. We install and test 128-192GB on the same system.

    Regarding the question about NO-SQL:
    1. Recently launched a cluster on "Cassandra" for the photo evaluation service.
    2. Launched " Tarantool " + "Voldemort" for the messaging system, where Tarantool - in memory storage, Voldemort - transport, replication, cluster.

    Caches: As a cache, its own Java solution is used. Data is stored outside the Java Heap using “sun.misc.Unsafe”, which avoids problems with the GC.

    Support


    The next employee of the company with whom we managed to talk was the girl Irena , she is the head of the technical support service.

    image

    Until recently, the support received more than 25 thousand calls per day - a huge amount of work. Moreover, 70-80% of the total number of applications were problems with a password - when someone forgot something and cannot remember. For comparison, there are about 150 applications per day on Habré, but there almost all issues require an individual approach.

    Help reorganization and writing a detailed FAQallowed to reduce the number of hits by almost 5 times, while the number of "password recovery" increased. Now the volume of work is 7-8 thousand applications per day, which is also a lot - all the problems of users are solved by about 30 employees of the company, divided into two "lines". The first line (7-8 simultaneously working employees) is located in Ivanovo and takes on the main blow (applications that can be answered stereotyped), the second line is located in Riga and is responsible for all other applications.

    It is noteworthy that among the employees there is a person (girl) who is engaged exclusively in ... pornography) More precisely, her removal from the resource - yes, it’s about the very social responsibility that Elina spoke about. As I understood from a lively conversation, only the lazy one didn’t joke on this topic ... but you must admit that on one (or maybe not one) of Fridays, this girl could make an interesting presentation about her not very dusty work :)

    Tired, but satisfied, we flew home. When I climbed into Habr, there was a desire to rewrite all the help on the site, which I could facilitate the work of our support team and help new users to get comfortable - I will try to do this in the near future.

    To be continued.

    Also popular now: