Do not put off in the mailbox: b2c-messenger 2GIS



    In September, a new feature appeared on 2gis.ru - a b2c messenger for communicating with organizations. Chat is very convenient when searching for a product or service: you can write to several companies at once, you do not need to listen to the voices of robotic answering machines or wait on the line until the operator determines the price or balance of the desired product. Select a company, click on the message icon on the company card, and a chat will open.

    To make a messenger, we had to figure out a little how the chats work and what the “big brothers” like WhatsApp or Telegram have under the hood. It turned out, everything is not so scary.

    Why 2GIS messenger


    Perhaps it’s worth starting not with the messenger, but with how we came to this idea. We used to show company email in the directory. Then we added a contact form with the company directly from the directory. They write less than they call, but the length of the chain of letters is an average of 2 to 10 messages. That is, the messenger is a logical development of the existing ability to send letters to the company.

    How does it work now




    Better yet, just try to ask us a question . We are in Novosibirsk, and when in Moscow at 18:00, we have already 22:00.

    Quick review: choose a bike


    First, we looked at BaaS (Backend as a Service), such as quickblox.com, backendless.com, sendbird.com, etc. All of them provide tools for adding standard chat functionality to mobile or web applications.

    Implementation of the main (sending email notifications in the company) and more advanced scenarios based on the existing BaaS solution will require us both to develop our own backends for integration via webhooks and extra effort to integrate with the vendor API. Therefore, BaaS did not suit us. In addition, we wanted to have full control over the features, that is, not to depend on third-party solutions. In the end, we are engineers and decided to experiment to get new experience for us.

    And how to deliver messages?


    To make your backend for the messenger, you first need to deal with a bunch of questions: how to transfer messages from the client to the server and back, where to store them, how to deal with a bunch of simultaneous connections, how to scale the service. We thought, googled, collected ways of delivering messages to a tablet for comparison.

    Table
    Sorry, but we could not make out the tables in the Habra Editor beautifully.
    WayDescriptionTransportProtocolBrowser Support
    WebsocketsThe client opens a permanent connection to the server using the WebSockets API .Websockets APITCP (HTTPhandshake required to open a connection)caniuse.com/#feat=websockets
    StreamingThe client opens a permanent connection to the server.XMLHttpRequest (multipart onload), XMLHttpRequest (onprogress), Iframe tagHTTPcaniuse.com/#feat=streaming
    Server sent eventsThe client opens a permanent connection to the server using the Server Sent Events API .API Server Sent EventsHTTPNo support in IE, caniuse.com/#feat=eventsource
    PollingThe client periodically polls the server for new messages.Any availableHTTP
    Long pollingThe client opens a long-lived connection to the server, which does not close until a new message or timeout expires. Immediately after closing the connection, the client opens a new one.XMLHttpRequest
    Script tag
    HTTP
    Browser Plugins (Java / Flash)The client opens a permanent connection to the server using the browser plug-in API (Java or Flash).Plugin APITCP / UDP


    Most modern chat applications use one of two methods: long polling or websockets. Websockets has a number of advantages over long polling (full-duplex connection, the absence of unnecessary traffic when reconnecting), it is widely supported in browsers and opensource libraries, therefore it seems to be the best choice for our task. There are also disadvantages: differences in implementation in different browsers, more complex logic on the server.

    The most common open general-purpose protocol for transmitting data is XMPP.
    Positive sidesNegative sides
    IETF Open StandardRedundancy of transmitted information
    A large number of opensource implementationsXML
    Wide possibilities for customizationIt will require refinement to the specifics of our task.
    Overview of great messengers
    MessengerDelivery methodProtocolServerStorage system
    WhatsappWebsocketsWe started with XMPP, but then switched to the in-house protocolThe entire server infrastructure is built on Erlang + FreeBS, this allows you to keep up to 2kk TCP connections on one server. We started on ejabberd, but very much modified it later for ourselves. Yaws, lighttpdDo not store message history on the server, messages are deleted from the server after receipt by the client, mnesia
    LineNo web version, only Chrome appThrift for mobile clientsJava, C ++, NginxWe started with a cluster of three Redis instances, after the explosive growth of the user base, cold data migrated to Hbase: message history and user / group / contact history, MySQL for backups and analytics
    ViberNo web versionIn-houseC ++ hosted in AWSWe started with a self-written in-memory repository written in C ++, then we switched to Mongo and Redis as a cache. Mongo did not work for performance, eventually migrated to Couchbase
    Facebook messengerLong pollingIn-house json-based for the web, Thrift for mobile applicationsErlang - message queues. C ++ - presence information service, message history repository. PHP is a frontend that handles all user requests except long polling. Services communicate with each other through Thrift
    Queue on MySQL, HBase
    SlackWebsocketsIn-house json-basedJava messaging server, LAMP for core app / APIs hosted in AWSRedis, MySql, Apache Solr

    However, having studied the solutions of various "large" messengers, we did not find a single significant success story using XMPP without its subsequent refinement and customization. Upon reflection, we decided to develop a simple json-based protocol that initially takes into account the specifics of our task.

    Technological stack


    It so happened historically that our main backend service - API 2GIS - was written in PHP5. Two years ago, we decided to leave PHP, migrate to Scala and Go. Go makes it very easy to build fairly complex concurrent programs. This became a decisive factor when choosing it as the main technology for implementing the messenger. And we already had some development experience on Go.

    So, we write the backend on Go, and the frontend on React + Redux. We chose RabbitMQ as the messaging system; for data storage we use Redis and PostgeSQL. We pack the application in a Docker container and deploy it via Gitlab-CI to the Deis platform.

    As I said above, we wanted to use websockets, but when it came to implementation, we slightly rethought the solution. The fact is that we wanted to release a feature as quickly as possible in order to test the hypothesis (the notorious MVP). To speed up, we decided to split the logic into using web sockets to send data from the client to the server and the simplest REST API in the other direction.

    What about notifications?


    Suddenly, notifications turned out to be the most difficult to implement. It would seem that everything is simple: send pushies to the phone, letters to mail, display notifications in the browser. But our task was to make sure that all notifications were read as soon as possible and at the same time did not turn into a spam stream.

    There is a solution called a cascading notification system. For example, we have three main channels: notifications on 2gis.ru, letters and push to phone.

    In this case, the notification system takes the form:

    - we check whether the user is online → if so, then we send him a notification in the browser;
    - if the user is not online, check if the phone is attached → if so, try to send a push to the mobile phone, show a notification in the dialogs in the browser, do not send the letter;
    - if this did not work, then we will send a letter;
    - if the notification concerns the company, we observe the reaction for some time, and if there is no response after the notification in the browser and push, then we will send an email anyway.

    In fact, we are not sending push notifications from this scenario right now, but very soon we will learn how to do it.

    Plans


    In the near future - add the messenger functionality to 2GIS mobile applications, finish the basic functionality (the ability to attach an attachment to the message, browser push), implement advanced communication scenarios (for example, a request to several companies at once).

    So far, companies are not used to the messenger, and not everyone responds quickly to messages. But we believe that the interaction of people and companies through text has a great future.

    Also popular now: