August 18, 2017 at 11:02How we switched from InboxSDK to Gmail.js
To develop our customer support solution, we chose the Gmail service, as it is one of the most popular email clients. And to expand its capabilities - the finished library InboxSDK. At the time of development, it possessed the functionality we needed, and such a solution helped us quickly enter the market with the first version of the product and gain a user base. On the other hand, InboxSDK is a closed library from a third-party developer and it had drawbacks that had to be solved in the future.
Deskun is implemented as an extension for Google Chrome and Yandex.Browser. It fully operates on the basis of the Gmail mail service. For our part, it was required to choose a library that could expand the functionality of mail and work with its client part. We settled on InboxSDK (ISDK).
The key feature of Gmail, for which users love it so much, is the automatic grouping of letters by dialog. At the internal level (on Gmail servers) each message has its own identifier and the identifier of the dialogue to which it belongs. But the problem is that in the web client, global message IDs are not contained anywhere in the page code, and global dialog IDs are contained only in the hash of the open dialog URL. Instead, the HTML elements of each message and dialog are marked with local IDs of the form ": a5,: b6, etc." received by the main (rather large and obfuscated) Gmail script. Since local IDs vary from page to page and from session to session, for the normal operation of extensions dealing with Gmail, it is necessary to quickly establish a connection between local and global IDs.
Despite the fact that ISDK contains a large number of different methods for working with Gmail, this library has one unpleasant feature - it is developed by Streak, which, in turn, provides its own CRM system inside Gmail. That is, it is an indirect competitor to Deskun.
Developing your own closed project on the basis of another closed project is at least wrong. Especially if these projects are similar and you can always be denied access to the API simply because you are competitors. Even if this does not happen, then you at least have no control over changes and availability. The library developers are in no hurry to help write third-party services based on their library, and the changelogs of the new revisions do not carry any useful information. Even the InboxSDK website has hardly been updated since launch. And you can forget about fixing bugs at the request of users or adding features that do not relate to the needs of Streak itself. First of all, Steak makes a library for himself.
First problem. For the library to work, binding to some specific and stable (stable - which do not change when switching views, changing states, etc.) classes is used. For example, in some cases, keystrokes on focused elements are emulated. And any change in the Gmail interface can disrupt the library itself and third-party developments based on InboxSDK.
Second problem. Since the library is connected remotely using the bootloader, there may be interruptions in the delivery of content over the CDN network. For some time we tried to understand what was going wrong, and then we found out that the library, on the basis of which our plug-in works, simply does not load for some users. At some point, it turned out that already 10% of our customers could not work with the service. If there were problems loading InboxSDK, then all platforms built on this library stopped working. It is important to understand that the problem can be of a point-like nature, and it can not always depend on the hosting. And on our part, it can be completely impossible to fix the source of the problem and could only wait.
Third problem. InboxSDK is not an open library. Publicly declared functions include a maximum of one third of all available. This is approximately 400 Kb of packaged code, which is forced to be parsed literally right after parsing the main Gmail scripts. Mail.google.com loading time on slow machines is critical.
Fourth problem. Changes to the library are made not just regularly, but several times a week. Unfortunately, most of these changes do not contain bug fixes and do not add new features, but are aimed at the needs of the main Streak developer. There was a situation when, due to one update, there was a conflict with the code in the client part of our plugin. In December, a new minor (shadow, without changelog) version was released for InboxSDK, in which the recipient insertion function was changed. It was implemented in such a way that after adding recipients there was a trigger to save the draft. Saving itself was done in a rather interesting way - it was just a simulation of pressing a certain key on the input field. Before all the changes, pressing was implemented by "Enter", but it was in that change that they began to press the symbol with the code "190" (¾). In our case, when opening a response form in tickets, recipients of this ticket are automatically substituted. Thus, a conflict turned out: during the response in the ticket, the symbol ¾ was substituted in the field of recipients.
Switching from InboxSDK to gmail.js
As a result, we gradually decided to switch to something more open, and the Gmail.js library (GJS) became the foundation for a future solution. Unlike InboxSDK, the Gmail.js library is completely open. Its development depends on the community and the functionality can develop at a very good pace.
During the transition, we encountered two main problems:
Firstly , for the functioning of the Gmail.js library, access to the window object of the page is required, in which Gmail stores some technical information. The extension in its "sandbox" has access only to the DOM of the page, but not to its environment.
Secondly , in order to completely exclude changes to the client part of the plugin, you need not just a wrapper, but you also need to add functions that are not in the library itself.
There is no perfect solution for the first problem. All of them are associated with some inconvenience - implementation by adding your script directly to the site page
We decided to implement a bridge that, using GJS, reports specific events directly to the plugin. In the current implementation, through the injection technology, the GJS library and our bridge are being introduced into the page. Thus, GJS works in the right environment. For all extensions, the Google Chrome browser provides some set of methods for working with it. Accordingly, if you make an extension through direct injection (like the GJS library), then it will lose all the possibilities of working with API extensions, so very important and necessary functionality may be lost.
To solve the second problem, it was necessary to analyze the behavior of not only InboxSDK, but also the operation of the Gmail client side. Now in our extension there are places in the code that are tied to the attributes and classes of the inbox: in some cases they additionally expanded the capabilities of the library, and in some they corrected some kind of inbox errors. Therefore, in order not to disrupt the operation of the client part of the plugin, it was necessary to add the necessary classes and attributes in the same way, add them to certain elements, according to certain events and under certain conditions. It has already been mentioned that there is no API for working with the client part of Gmail. But there is stored a lot of necessary information. For example, user settings, a list of available email aliases,
We managed to make the transition from one library to another without changing the logic and code in the client part of the plugin. The final solution is not just using the GJS library, rather we created our own hybrid library ISDK and GJS. Gmail.js served as the foundation, but functionally our development is similar to InboxSDK. If the Habrahabr community is interested, and there will be many requests to share our code, then we will put it in order and put it out.
Why did we choose InboxSDK?
Deskun is implemented as an extension for Google Chrome and Yandex.Browser. It fully operates on the basis of the Gmail mail service. For our part, it was required to choose a library that could expand the functionality of mail and work with its client part. We settled on InboxSDK (ISDK).
The key feature of Gmail, for which users love it so much, is the automatic grouping of letters by dialog. At the internal level (on Gmail servers) each message has its own identifier and the identifier of the dialogue to which it belongs. But the problem is that in the web client, global message IDs are not contained anywhere in the page code, and global dialog IDs are contained only in the hash of the open dialog URL. Instead, the HTML elements of each message and dialog are marked with local IDs of the form ": a5,: b6, etc." received by the main (rather large and obfuscated) Gmail script. Since local IDs vary from page to page and from session to session, for the normal operation of extensions dealing with Gmail, it is necessary to quickly establish a connection between local and global IDs.
Despite the fact that ISDK contains a large number of different methods for working with Gmail, this library has one unpleasant feature - it is developed by Streak, which, in turn, provides its own CRM system inside Gmail. That is, it is an indirect competitor to Deskun.
InboxSDK Issues
Developing your own closed project on the basis of another closed project is at least wrong. Especially if these projects are similar and you can always be denied access to the API simply because you are competitors. Even if this does not happen, then you at least have no control over changes and availability. The library developers are in no hurry to help write third-party services based on their library, and the changelogs of the new revisions do not carry any useful information. Even the InboxSDK website has hardly been updated since launch. And you can forget about fixing bugs at the request of users or adding features that do not relate to the needs of Streak itself. First of all, Steak makes a library for himself.
First problem. For the library to work, binding to some specific and stable (stable - which do not change when switching views, changing states, etc.) classes is used. For example, in some cases, keystrokes on focused elements are emulated. And any change in the Gmail interface can disrupt the library itself and third-party developments based on InboxSDK.
Second problem. Since the library is connected remotely using the bootloader, there may be interruptions in the delivery of content over the CDN network. For some time we tried to understand what was going wrong, and then we found out that the library, on the basis of which our plug-in works, simply does not load for some users. At some point, it turned out that already 10% of our customers could not work with the service. If there were problems loading InboxSDK, then all platforms built on this library stopped working. It is important to understand that the problem can be of a point-like nature, and it can not always depend on the hosting. And on our part, it can be completely impossible to fix the source of the problem and could only wait.
Third problem. InboxSDK is not an open library. Publicly declared functions include a maximum of one third of all available. This is approximately 400 Kb of packaged code, which is forced to be parsed literally right after parsing the main Gmail scripts. Mail.google.com loading time on slow machines is critical.
Fourth problem. Changes to the library are made not just regularly, but several times a week. Unfortunately, most of these changes do not contain bug fixes and do not add new features, but are aimed at the needs of the main Streak developer. There was a situation when, due to one update, there was a conflict with the code in the client part of our plugin. In December, a new minor (shadow, without changelog) version was released for InboxSDK, in which the recipient insertion function was changed. It was implemented in such a way that after adding recipients there was a trigger to save the draft. Saving itself was done in a rather interesting way - it was just a simulation of pressing a certain key on the input field. Before all the changes, pressing was implemented by "Enter", but it was in that change that they began to press the symbol with the code "190" (¾). In our case, when opening a response form in tickets, recipients of this ticket are automatically substituted. Thus, a conflict turned out: during the response in the ticket, the symbol ¾ was substituted in the field of recipients.
Switching from InboxSDK to gmail.js
As a result, we gradually decided to switch to something more open, and the Gmail.js library (GJS) became the foundation for a future solution. Unlike InboxSDK, the Gmail.js library is completely open. Its development depends on the community and the functionality can develop at a very good pace.
During the transition, we encountered two main problems:
Firstly , for the functioning of the Gmail.js library, access to the window object of the page is required, in which Gmail stores some technical information. The extension in its "sandbox" has access only to the DOM of the page, but not to its environment.
Secondly , in order to completely exclude changes to the client part of the plugin, you need not just a wrapper, but you also need to add functions that are not in the library itself.
There is no perfect solution for the first problem. All of them are associated with some inconvenience - implementation by adding your script directly to the site page
scriptwith postMessage, Custom Event, or writing directly to the DOM. We decided to implement a bridge that, using GJS, reports specific events directly to the plugin. In the current implementation, through the injection technology, the GJS library and our bridge are being introduced into the page. Thus, GJS works in the right environment. For all extensions, the Google Chrome browser provides some set of methods for working with it. Accordingly, if you make an extension through direct injection (like the GJS library), then it will lose all the possibilities of working with API extensions, so very important and necessary functionality may be lost.
To solve the second problem, it was necessary to analyze the behavior of not only InboxSDK, but also the operation of the Gmail client side. Now in our extension there are places in the code that are tied to the attributes and classes of the inbox: in some cases they additionally expanded the capabilities of the library, and in some they corrected some kind of inbox errors. Therefore, in order not to disrupt the operation of the client part of the plugin, it was necessary to add the necessary classes and attributes in the same way, add them to certain elements, according to certain events and under certain conditions. It has already been mentioned that there is no API for working with the client part of Gmail. But there is stored a lot of necessary information. For example, user settings, a list of available email aliases,
Total
We managed to make the transition from one library to another without changing the logic and code in the client part of the plugin. The final solution is not just using the GJS library, rather we created our own hybrid library ISDK and GJS. Gmail.js served as the foundation, but functionally our development is similar to InboxSDK. If the Habrahabr community is interested, and there will be many requests to share our code, then we will put it in order and put it out.
