Do not put off in the mailbox: b2c-messenger 2GIS
In September, a new feature appeared on 2gis.ru - a b2c messenger for communicating with organizations. Chat is very convenient when searching for a product or service: you can write to several companies at once, you do not need to listen to the voices of robotic answering machines or wait on the line until the operator determines the price or balance of the desired product. Select a company, click on the message icon on the company card, and a chat will open.
To make a messenger, we had to figure out a little how the chats work and what the “big brothers” like WhatsApp or Telegram have under the hood. It turned out, everything is not so scary.
Why 2GIS messenger
Perhaps it’s worth starting not with the messenger, but with how we came to this idea. We used to show company email in the directory. Then we added a contact form with the company directly from the directory. They write less than they call, but the length of the chain of letters is an average of 2 to 10 messages. That is, the messenger is a logical development of the existing ability to send letters to the company.
How does it work now
Better yet, just try to ask us a question . We are in Novosibirsk, and when in Moscow at 18:00, we have already 22:00.
Quick review: choose a bike
First, we looked at BaaS (Backend as a Service), such as quickblox.com, backendless.com, sendbird.com, etc. All of them provide tools for adding standard chat functionality to mobile or web applications.
Implementation of the main (sending email notifications in the company) and more advanced scenarios based on the existing BaaS solution will require us both to develop our own backends for integration via webhooks and extra effort to integrate with the vendor API. Therefore, BaaS did not suit us. In addition, we wanted to have full control over the features, that is, not to depend on third-party solutions. In the end, we are engineers and decided to experiment to get new experience for us.
And how to deliver messages?
To make your backend for the messenger, you first need to deal with a bunch of questions: how to transfer messages from the client to the server and back, where to store them, how to deal with a bunch of simultaneous connections, how to scale the service. We thought, googled, collected ways of delivering messages to a tablet for comparison.
Table
Sorry, but we could not make out the tables in the Habra Editor beautifully.
Way | Description | Transport | Protocol | Browser Support |
Websockets | The client opens a permanent connection to the server using the WebSockets API . | Websockets API | TCP (HTTPhandshake required to open a connection) | caniuse.com/#feat=websockets |
Streaming | The client opens a permanent connection to the server. | XMLHttpRequest (multipart onload), XMLHttpRequest (onprogress), Iframe tag | HTTP | caniuse.com/#feat=streaming |
Server sent events | The client opens a permanent connection to the server using the Server Sent Events API . | API Server Sent Events | HTTP | No support in IE, caniuse.com/#feat=eventsource |
Polling | The client periodically polls the server for new messages. | Any available | HTTP | |
Long polling | The client opens a long-lived connection to the server, which does not close until a new message or timeout expires. Immediately after closing the connection, the client opens a new one. | XMLHttpRequest Script tag | HTTP | |
Browser Plugins (Java / Flash) | The client opens a permanent connection to the server using the browser plug-in API (Java or Flash). | Plugin API | TCP / UDP |
Most modern chat applications use one of two methods: long polling or websockets. Websockets has a number of advantages over long polling (full-duplex connection, the absence of unnecessary traffic when reconnecting), it is widely supported in browsers and opensource libraries, therefore it seems to be the best choice for our task. There are also disadvantages: differences in implementation in different browsers, more complex logic on the server.
The most common open general-purpose protocol for transmitting data is XMPP.
Positive sides | Negative sides |
IETF Open Standard | Redundancy of transmitted information |
A large number of opensource implementations | XML |
Wide possibilities for customization | It will require refinement to the specifics of our task. |
Overview of great messengers
Messenger | Delivery method | Protocol | Server | Storage system | |
Websockets | We started with XMPP, but then switched to the in-house protocol | The entire server infrastructure is built on Erlang + FreeBS, this allows you to keep up to 2kk TCP connections on one server. We started on ejabberd, but very much modified it later for ourselves. Yaws, lighttpd | Do not store message history on the server, messages are deleted from the server after receipt by the client, mnesia | ||
Line | No web version, only Chrome app | Thrift for mobile clients | Java, C ++, Nginx | We started with a cluster of three Redis instances, after the explosive growth of the user base, cold data migrated to Hbase: message history and user / group / contact history, MySQL for backups and analytics | |
Viber | No web version | In-house | C ++ hosted in AWS | We started with a self-written in-memory repository written in C ++, then we switched to Mongo and Redis as a cache. Mongo did not work for performance, eventually migrated to Couchbase | |
Facebook messenger | Long polling | In-house json-based for the web, Thrift for mobile applications | Erlang - message queues. C ++ - presence information service, message history repository. PHP is a frontend that handles all user requests except long polling. Services communicate with each other through Thrift | Queue on MySQL, HBase | |
Slack | Websockets | In-house json-based | Java messaging server, LAMP for core app / APIs hosted in AWS | Redis, MySql, Apache Solr |
However, having studied the solutions of various "large" messengers, we did not find a single significant success story using XMPP without its subsequent refinement and customization. Upon reflection, we decided to develop a simple json-based protocol that initially takes into account the specifics of our task.
Technological stack
It so happened historically that our main backend service - API 2GIS - was written in PHP5. Two years ago, we decided to leave PHP, migrate to Scala and Go. Go makes it very easy to build fairly complex concurrent programs. This became a decisive factor when choosing it as the main technology for implementing the messenger. And we already had some development experience on Go.
So, we write the backend on Go, and the frontend on React + Redux. We chose RabbitMQ as the messaging system; for data storage we use Redis and PostgeSQL. We pack the application in a Docker container and deploy it via Gitlab-CI to the Deis platform.
As I said above, we wanted to use websockets, but when it came to implementation, we slightly rethought the solution. The fact is that we wanted to release a feature as quickly as possible in order to test the hypothesis (the notorious MVP). To speed up, we decided to split the logic into using web sockets to send data from the client to the server and the simplest REST API in the other direction.
What about notifications?
Suddenly, notifications turned out to be the most difficult to implement. It would seem that everything is simple: send pushies to the phone, letters to mail, display notifications in the browser. But our task was to make sure that all notifications were read as soon as possible and at the same time did not turn into a spam stream.
There is a solution called a cascading notification system. For example, we have three main channels: notifications on 2gis.ru, letters and push to phone.
In this case, the notification system takes the form:
- we check whether the user is online → if so, then we send him a notification in the browser;
- if the user is not online, check if the phone is attached → if so, try to send a push to the mobile phone, show a notification in the dialogs in the browser, do not send the letter;
- if this did not work, then we will send a letter;
- if the notification concerns the company, we observe the reaction for some time, and if there is no response after the notification in the browser and push, then we will send an email anyway.
In fact, we are not sending push notifications from this scenario right now, but very soon we will learn how to do it.
Plans
In the near future - add the messenger functionality to 2GIS mobile applications, finish the basic functionality (the ability to attach an attachment to the message, browser push), implement advanced communication scenarios (for example, a request to several companies at once).
So far, companies are not used to the messenger, and not everyone responds quickly to messages. But we believe that the interaction of people and companies through text has a great future.