Cross-browser web extension for custom scripts Part 2
In this article, I continue the cycle of publications, in which I want to talk about my experience writing a web browser extension. I already had the experience of creating a web extension, which was installed by about 100,000 Chrome users who worked autonomously, but in this series of articles I decided to delve into the process of developing a web extension by tightly integrating it with the server part.
Part 1 , Part 3 , Part 4
At the very beginning of my idea, I was thinking how to save data obtained as a result of the script. The possibility of using locaStorage or similar database solutions on the client side came to mind. As the scheduling code progressed along this path, it turned out that downloading files and storing them on a user's computer is not a good idea, since the user will be tied to one computer.
Therefore, it is necessary to save user scripts, data obtained as a result of server-side scripts execution. Uploading user files should also occur through the server side. The user must register, reset the password, etc. directly through the web extension interface, because registering through a separate page would not allow you to comfortably interact with the web extension.
Thus, the use of XHR with the RESTful API was chosen. This option also allows you to use the same code in the future to build a pipeline data delivery system. For example, the retrieval of web resource positions in search engine results can be scheduled in a daily cycle. After that, the third-party application requests the data obtained as a result of the script's work using the API and builds a graph showing the tendency of the position to change in the search results.
Many frameworks in different programming languages have libraries, modules and packages for the rapid development of RESTful server applications. For about 12 years I have been working with different PHP-based frameworks, and for the last 3 years I have also been working with Meteor.js. This framework runs on top of node.js, so I chose it for the server part because of its high performance (112 milliseconds on average, to write one line of data received from the script) and a rich set of ready-made packages for building the application.
As a RESTful library, I chose the github.com/kahmali/meteor-restivus package , which, in other matters, needed to be patched, since it worked incorrectly with hook authorization and protection against the endless selection of passwords.
Restivus has good functionality for describing the RESTful API and supports operation in an authorized or anonymous way. To confirm the authority to execute a method-specific user using a RESTful address, token and userId are used. These parameters can be stored in localStorage web permissions and used when calling XHR.
For the administrative zone, I chose an extensible and highly configurable package github.com/yogiben/meteor-admin , which allowed me to reduce the time to create many of the same type of pages for the needs of the security team.
The interface plays a crucial role in the choice of the user of a product. Convenience and minimalism were chosen as the main criteria for the web extension interface. Later on, the aesthetic side was added to make the whole application complete.
Semantic-ui was chosen as the css / html framework. At the same time, I refused additional npm packages and webpack-type assembly mechanisms to minimize dependencies and the size of the web extension. The code is written as transparently as possible, using only three js libraries and third-party files.
To start the process a list of required pages was compiled. This list has been grouped pages to work with a user account.
At this stage there was a problem with a one-time password reset link. In a normal application, the user can follow the link, reset the password, and then enter the application. A web extension can “intercept” links using navigator.registerProtocolHandler and authorize a user in a web extension, but not all browsers support this. Therefore, it was decided to reset the password by reference, and then the user can enter the web extension with the help of a new password.
Below is a form for registering a new user in a web extension.
Here you can see elements for social marketing, such as the Invite code and links for distribution on social networks next to the support element.
After registration, or after authorization, the user can edit his profile directly in the web extension. This qualitatively improves the current solution, because it removes the need to work with a separate website. The user can perform all actions in one place.
All screens are divided into tabs, which allows you to quickly move between screens and plays the role of a kind of menu. In some cases, for a more visual presentation, the minimization of elements for the current iteration is deliberately denied. For example, the signatures of the buttons are placed on the buttons themselves, although in the future we plan to replace the vertical list of buttons with a horizontal one and take out signatures into tooltips. This is done to maximize the simplicity of the web extension presentation for new users. So made the main screen with a list of scripts for the current iteration.
Each user script must be created by someone. As mentioned earlier in the web extension can be as public scripts, as well as private. To add scripts use the following simple form.
Further, the user can set a sign of the publicity of the script, which will enable other users to use the experience of the web extension community.
Public scripts have a number of restrictions. For example, they cannot be used for autoloading on the page, in the task scheduler and have hotkeys. In addition, each public script has the ability to “edit”, during which the user can remove the sign of publicity and save it in a private state as its own script. These additional steps are necessary to protect against unscrupulous developers. When “editing” a public script, the user will be required to review the code and decide on its use for their needs.
Downloading files in the web extension is implemented by reading the contents of the file into the buffer, encoding and then sending it via the XHR channel to the server. On the POST server, the request is processed and the file is sent to the cloud storage. In scripts, users can read downloaded files via GC.loadFile ('filename.ext'); , function from the internal library, about which a separate article will be written.
Each script can record the data from the calculations by calling the function of the internal library. Each call will write one row to a cell with the same name for all rows. This way you can write csv files. The screen below shows the interface for working with the output data. The user can download data directly from the web extension, send the generated file to his email, get a link to the API for use in a third-party application, or delete the data.
All actions that change the state of the script, such as a delete action, require confirmation. In the web extension there is a restriction on the use of pop-up or confirm windows, since it itself has an interface running in pop-up. Therefore, a confirmation word input mechanism is used.
In the next article I will talk about "pitfalls in the implementation of the interaction of web expansion and server-side . "
Part 1 , Part 3 , Part 4
Choosing a server-side framework
At the very beginning of my idea, I was thinking how to save data obtained as a result of the script. The possibility of using locaStorage or similar database solutions on the client side came to mind. As the scheduling code progressed along this path, it turned out that downloading files and storing them on a user's computer is not a good idea, since the user will be tied to one computer.
Therefore, it is necessary to save user scripts, data obtained as a result of server-side scripts execution. Uploading user files should also occur through the server side. The user must register, reset the password, etc. directly through the web extension interface, because registering through a separate page would not allow you to comfortably interact with the web extension.
Thus, the use of XHR with the RESTful API was chosen. This option also allows you to use the same code in the future to build a pipeline data delivery system. For example, the retrieval of web resource positions in search engine results can be scheduled in a daily cycle. After that, the third-party application requests the data obtained as a result of the script's work using the API and builds a graph showing the tendency of the position to change in the search results.
Many frameworks in different programming languages have libraries, modules and packages for the rapid development of RESTful server applications. For about 12 years I have been working with different PHP-based frameworks, and for the last 3 years I have also been working with Meteor.js. This framework runs on top of node.js, so I chose it for the server part because of its high performance (112 milliseconds on average, to write one line of data received from the script) and a rich set of ready-made packages for building the application.
As a RESTful library, I chose the github.com/kahmali/meteor-restivus package , which, in other matters, needed to be patched, since it worked incorrectly with hook authorization and protection against the endless selection of passwords.
Restivus has good functionality for describing the RESTful API and supports operation in an authorized or anonymous way. To confirm the authority to execute a method-specific user using a RESTful address, token and userId are used. These parameters can be stored in localStorage web permissions and used when calling XHR.
For the administrative zone, I chose an extensible and highly configurable package github.com/yogiben/meteor-admin , which allowed me to reduce the time to create many of the same type of pages for the needs of the security team.
Interface Design for Web Extensions
The interface plays a crucial role in the choice of the user of a product. Convenience and minimalism were chosen as the main criteria for the web extension interface. Later on, the aesthetic side was added to make the whole application complete.
Semantic-ui was chosen as the css / html framework. At the same time, I refused additional npm packages and webpack-type assembly mechanisms to minimize dependencies and the size of the web extension. The code is written as transparently as possible, using only three js libraries and third-party files.
To start the process a list of required pages was compiled. This list has been grouped pages to work with a user account.
- Login page
- Registration page
- Password reset page
At this stage there was a problem with a one-time password reset link. In a normal application, the user can follow the link, reset the password, and then enter the application. A web extension can “intercept” links using navigator.registerProtocolHandler and authorize a user in a web extension, but not all browsers support this. Therefore, it was decided to reset the password by reference, and then the user can enter the web extension with the help of a new password.
Below is a form for registering a new user in a web extension.
Here you can see elements for social marketing, such as the Invite code and links for distribution on social networks next to the support element.
After registration, or after authorization, the user can edit his profile directly in the web extension. This qualitatively improves the current solution, because it removes the need to work with a separate website. The user can perform all actions in one place.
All screens are divided into tabs, which allows you to quickly move between screens and plays the role of a kind of menu. In some cases, for a more visual presentation, the minimization of elements for the current iteration is deliberately denied. For example, the signatures of the buttons are placed on the buttons themselves, although in the future we plan to replace the vertical list of buttons with a horizontal one and take out signatures into tooltips. This is done to maximize the simplicity of the web extension presentation for new users. So made the main screen with a list of scripts for the current iteration.
Each user script must be created by someone. As mentioned earlier in the web extension can be as public scripts, as well as private. To add scripts use the following simple form.
Further, the user can set a sign of the publicity of the script, which will enable other users to use the experience of the web extension community.
Public scripts have a number of restrictions. For example, they cannot be used for autoloading on the page, in the task scheduler and have hotkeys. In addition, each public script has the ability to “edit”, during which the user can remove the sign of publicity and save it in a private state as its own script. These additional steps are necessary to protect against unscrupulous developers. When “editing” a public script, the user will be required to review the code and decide on its use for their needs.
Downloading files in the web extension is implemented by reading the contents of the file into the buffer, encoding and then sending it via the XHR channel to the server. On the POST server, the request is processed and the file is sent to the cloud storage. In scripts, users can read downloaded files via GC.loadFile ('filename.ext'); , function from the internal library, about which a separate article will be written.
Each script can record the data from the calculations by calling the function of the internal library. Each call will write one row to a cell with the same name for all rows. This way you can write csv files. The screen below shows the interface for working with the output data. The user can download data directly from the web extension, send the generated file to his email, get a link to the API for use in a third-party application, or delete the data.
All actions that change the state of the script, such as a delete action, require confirmation. In the web extension there is a restriction on the use of pop-up or confirm windows, since it itself has an interface running in pop-up. Therefore, a confirmation word input mechanism is used.
In the next article I will talk about "pitfalls in the implementation of the interaction of web expansion and server-side . "