Push message server
In any modern Internet service, you can select only two main functions:
The first point, I think, does not need an explanation.
The second point is client server technology, but vice versa. The client does not periodically make a request to the server - are there any new messages. The server, when a certain event occurs, immediately sends a message to the client.
For better understanding, the service is not spherical in vacuum. Service can be represented as:
The list can be expanded to infinity, given, as an example, only the most well-known use cases.
Almost all of the examples of services can be represented as a "messenger". Some of the examples were described exactly like this, I saw articles on how to connect a camera and send pictures to one well-known messenger.
Not so long ago there was an article that the service of cameras in the door eyes, instead of artificial intelligence was observed by strangers. I will not focus on free services from large "good" corporations. As they say in the famous proverb “Free cheese is in a mousetrap” and guided by another proverb “Your shirt is closer to the body”, your “service” is better.
The server script code is open and free.
To implement, I chose socket.io postgreSQL node.js, implemented the first version more than two years ago, and the service was closely related to 1c https://infostart.ru/public/545047/ , they had never announced an interaction server anywhere. But it was a long time ago, at the moment, the server part is not connected in any way with the mentioned program, the client part can be integrated in the form of external processing or extension.
The name has chosen meaningful - "push0k". The first part is taken from the phrase “push messages”, the second well-known English is okay abbreviated, but is written in zero for greater significance.
In this case, this is not a “free cheese in a mousetrap”, since it requires a windows computer, linux, macOS (server) to work through the Internet, an external ip address is needed, possibly forward. Preferably the domain name associated with the external ip address. Also, it is desirable, but not necessary, not a self-signed certificate associated with the domain name.
There are no requirements for hardware as such; it can work on a home nas where there is a docker. Docker in nas means x86-64 architecture, and postgreSQL in such nas without docker cannot be installed. The exact understanding of how many clients such a server can support depends on the tasks — the logic of the service and the traffic that is transmitted between clients.
The server uses additional modules:
1. Initially, you must install node.js nodejs.org/en/download
2. PostgreSQL must also be installed on the computer on the local network or on the same postgrespro.ru/products/download
On the postgreSQL server, you must execute the file “push0kStructure.sql” or almost queries from this file to create a database with the necessary tables.
3. Download the first five server files, except push0kStructure.sql, to any computer directory.
4. Edit the “starter_cfg.js” file in any text editor. It is important to specify the parameters for connecting to the postgreSQL server and set the port to connect to the administrative part of the server push0k.
5. Start the terminal (console) and use the "cd / path / of your / directory /" command to go to the server files directory. The path specified in the terminal command must be from point 3.
6. Execute the command in the “npm install” terminal to install additional modules.
Run the command in the node nodeterter.js terminal to start the administration service.
7. Download and install the program push0k admin. In "push0k admin", the basic websocket parameters, the server port, the number of processes and many others are configured. You can control the start and stop of server processes, create and manage users and their rooms (groups).
The application is made on electron using vue.js. Inside there is a similarity of the window system, small modal dialogs can be dragged as windows, windows headers for windows are made similar to windows 10, for Mac OS as in recent versions, but without taking into account the dark theme. For linux I will compile later, there is a bit more difficult with the dialog headers, I think it will be like in ubuntu and, if not ubuntu, then win 10 style. Memory consumption in win 10 and Mac OS more than 150 megabytes did not see.
In previous versions it was not possible to add animation, did a little without fanaticism. The previously described dialogs - windows appear from the center of the application window, gradually increasing, when closing they fly off to the center, decreasing. Pressing buttons and required fields are also animated.
When implementing, there was a great desire to make a dark theme, but since there was still no light theme, I had to be content with just the button panel on the right. In Mac OS, this single dark part has the effect of “vibrancy”, that is, it is partially transparent, similar to most windows of the standard interface. Unfortunately, in Windows, a similar effect called fluent design could not be made because there is no electron-like functionality for Windows.
As previously mentioned, the http and ws (websocket) protocols are used for control. It is possible to use secure connections https and wss. With secure connections, you can see the data of the certificates used. Similarly, browsers use the “Lock” icon - the connection is safe, “Open lock” - the connection is unsafe. And without icons, respectively - a secure connection is not used.
I've never seen this kind of application. When developing, I was guided by the fact that I myself would be interested in monitoring and what parameters to see in statistics.
For example, in the connection table for each connection there is data “connection time”, “authorization time”, “synchronization time”. First, it allows you to understand how quickly the “ws: //” connection or the secure “wss: //” connection is established. The second is that after the connection, a message with the authorization data is sent separately; during authorization, the user is prompted to verify the user and verify the password hash unique for each connection. The third is the time for receiving new messages and new or updated reference data. Also, versions of node.js, socket.io, and other described data as a whole allow to understand how the update of a node.js or a separate module influenced the speed, or some server refinement could affect.
In the example above, only a small portion of such data is described. Also, in a separate table, time, size and speed of MB / s are saved. Download sync data. The same table stores the data of downloading attachment files to the server and downloading this data from the server.
The time for calculating the speed is calculated on the client or server, depending on the situation. It is impossible to use time and server and client as it cannot be perfectly synchronized. This imposes a small lag of about the size of the ping (delivery confirmation) between the client and the server. When uploading or downloading large files, the described lag is irrelevant.
In the connection table, you can see the closed connection with the data "transferred bytes", "received bytes" and "size" of the synchronization data more than "transferred bytes". When synchronizing data, the socket.io automatic compression is used; when downloading or downloading attachments, automatic compression is disabled, as often compressing an already compressed negatively affects the speed.
As written in the beginning of the article, it was never the goal to make one common service and connect as many users as possible to it. But it was always interesting how many users there could theoretically be and how much the server could handle.
For this, I made a simple test: Online clients of a certain room (group), a message is sent with test parameters (there are only one main parameter in the parameters - the number of messages), which each client will send to other clients online. After the test distribution, the total time of all messages, notifications, records in postgresql tables, the number of all messages, notifications and records is calculated, and accordingly, the speed is calculated, including the total of all operations on the server.
Example from the screenshot:
sometimes surprising, only 10,000 messages and 390,000 gestures,
sending 50,000 messages: Server processes: 4 Users: 3
Total messages received 10,000 * 3 = 30,000
Forwarding to other server processes 30,000 * Server processes: 4 - 1 = 90,000
Отправлено адресатам 30 000 * Пользователи: 3 — 1 = 60 000
Записано сообщений в таблицу postgreSQL 30 000
Получено уведомлений доставки 90 000
Записано в таблицу postgreSQL уведомлений доставки 90 000
Все операции 390 000
With each connection, all reference data is obtained: users, rooms, devices, databases, as well as the last 300 "log-statistical" entries: Connections, Data Synchronization, Forwarding Attachments, Messages, Log.
Messages practically repeat the postgresql table and do not have filters, settings and a form where you can see the message completely. The logic of the table is debugging, it is possible to quickly look at the development whether the message reached the table or not.
The "Users", "Rooms (groups)", "Connections" and "Log" tables are updated automatically. For the rest of the data does not make sense in the online update.
The push0k admin application can be downloaded here:
Windows: push0kadmin Setup 19.1.11.exe
Mac OS: push0kadmin-19.1.11.dmg
It is free, but unlike the server, I don’t plan to open the source code.
In the next article I will describe the client part of the push0k desctop client, as well as a small example, the code and how to connect, log in and get data from the server.
For the previously mentioned earlier versions, there was also an android client, immortally living in the background on 7th and 8th versions of android. But he still, I think not long behind. Later, I think, there will be a third article with the android client, and then you look close to iOS.
- The first is user authorization.
- The second is the instant dispatch of a certain event from the server to the client.
The first point, I think, does not need an explanation.
The second point is client server technology, but vice versa. The client does not periodically make a request to the server - are there any new messages. The server, when a certain event occurs, immediately sends a message to the client.
For better understanding, the service is not spherical in vacuum. Service can be represented as:
- Folder with files in the cloud. Information about the change, addition and deletion is sent to other users or the current user, but to other devices.
- A computer program for reading server logs, when “error” records appear, sending the contents of a record to a user on a mobile phone.
- Video peephole (camera), taking pictures when moving near the door of the apartment.
- Service receiving telemetry from the android-auto application.
- Similar to the previous item service, allowing you to know whether the child reached the school or came home from school.
The list can be expanded to infinity, given, as an example, only the most well-known use cases.
Almost all of the examples of services can be represented as a "messenger". Some of the examples were described exactly like this, I saw articles on how to connect a camera and send pictures to one well-known messenger.
Not so long ago there was an article that the service of cameras in the door eyes, instead of artificial intelligence was observed by strangers. I will not focus on free services from large "good" corporations. As they say in the famous proverb “Free cheese is in a mousetrap” and guided by another proverb “Your shirt is closer to the body”, your “service” is better.
The server script code is open and free.
To implement, I chose socket.io postgreSQL node.js, implemented the first version more than two years ago, and the service was closely related to 1c https://infostart.ru/public/545047/ , they had never announced an interaction server anywhere. But it was a long time ago, at the moment, the server part is not connected in any way with the mentioned program, the client part can be integrated in the form of external processing or extension.
The name has chosen meaningful - "push0k". The first part is taken from the phrase “push messages”, the second well-known English is okay abbreviated, but is written in zero for greater significance.
In this case, this is not a “free cheese in a mousetrap”, since it requires a windows computer, linux, macOS (server) to work through the Internet, an external ip address is needed, possibly forward. Preferably the domain name associated with the external ip address. Also, it is desirable, but not necessary, not a self-signed certificate associated with the domain name.
There are no requirements for hardware as such; it can work on a home nas where there is a docker. Docker in nas means x86-64 architecture, and postgreSQL in such nas without docker cannot be installed. The exact understanding of how many clients such a server can support depends on the tasks — the logic of the service and the traffic that is transmitted between clients.
Server Description:
The server uses additional modules:
- socket.io - the module adds a websocket protocol with additional mass. opportunities.
- node-postgres is a module for communicating with the postgreSQL database server.
- pm2 - module for running multiple server processes with a load balancer.
Server files:
- starter.js - http (s) service, you can tell the server admin panel. Through this service, the settings are changed and the processes of the main websocket server are started.
- push0k.js is the main websocket server.
- starter_cfg.js - server administrative script settings. Connect to postgreSQL and files for https connection. Files for https connections, as well as the domain name, may differ from the main server, for greater security. Manually changed when first started.
- config.js - the basic settings of the entire server.
- package.js - file description version author needed modules for work.
- push0kStructure.sql - file description of the postgreSQL database for initial initialization, empty database.
Installation:
1. Initially, you must install node.js nodejs.org/en/download
2. PostgreSQL must also be installed on the computer on the local network or on the same postgrespro.ru/products/download
On the postgreSQL server, you must execute the file “push0kStructure.sql” or almost queries from this file to create a database with the necessary tables.
3. Download the first five server files, except push0kStructure.sql, to any computer directory.
4. Edit the “starter_cfg.js” file in any text editor. It is important to specify the parameters for connecting to the postgreSQL server and set the port to connect to the administrative part of the server push0k.
5. Start the terminal (console) and use the "cd / path / of your / directory /" command to go to the server files directory. The path specified in the terminal command must be from point 3.
6. Execute the command in the “npm install” terminal to install additional modules.
Run the command in the node nodeterter.js terminal to start the administration service.
7. Download and install the program push0k admin. In "push0k admin", the basic websocket parameters, the server port, the number of processes and many others are configured. You can control the start and stop of server processes, create and manage users and their rooms (groups).
Push0k admin application
The application is made on electron using vue.js. Inside there is a similarity of the window system, small modal dialogs can be dragged as windows, windows headers for windows are made similar to windows 10, for Mac OS as in recent versions, but without taking into account the dark theme. For linux I will compile later, there is a bit more difficult with the dialog headers, I think it will be like in ubuntu and, if not ubuntu, then win 10 style. Memory consumption in win 10 and Mac OS more than 150 megabytes did not see.
In previous versions it was not possible to add animation, did a little without fanaticism. The previously described dialogs - windows appear from the center of the application window, gradually increasing, when closing they fly off to the center, decreasing. Pressing buttons and required fields are also animated.
When implementing, there was a great desire to make a dark theme, but since there was still no light theme, I had to be content with just the button panel on the right. In Mac OS, this single dark part has the effect of “vibrancy”, that is, it is partially transparent, similar to most windows of the standard interface. Unfortunately, in Windows, a similar effect called fluent design could not be made because there is no electron-like functionality for Windows.
As previously mentioned, the http and ws (websocket) protocols are used for control. It is possible to use secure connections https and wss. With secure connections, you can see the data of the certificates used. Similarly, browsers use the “Lock” icon - the connection is safe, “Open lock” - the connection is unsafe. And without icons, respectively - a secure connection is not used.
Common logic
I've never seen this kind of application. When developing, I was guided by the fact that I myself would be interested in monitoring and what parameters to see in statistics.
For example, in the connection table for each connection there is data “connection time”, “authorization time”, “synchronization time”. First, it allows you to understand how quickly the “ws: //” connection or the secure “wss: //” connection is established. The second is that after the connection, a message with the authorization data is sent separately; during authorization, the user is prompted to verify the user and verify the password hash unique for each connection. The third is the time for receiving new messages and new or updated reference data. Also, versions of node.js, socket.io, and other described data as a whole allow to understand how the update of a node.js or a separate module influenced the speed, or some server refinement could affect.
In the example above, only a small portion of such data is described. Also, in a separate table, time, size and speed of MB / s are saved. Download sync data. The same table stores the data of downloading attachment files to the server and downloading this data from the server.
The time for calculating the speed is calculated on the client or server, depending on the situation. It is impossible to use time and server and client as it cannot be perfectly synchronized. This imposes a small lag of about the size of the ping (delivery confirmation) between the client and the server. When uploading or downloading large files, the described lag is irrelevant.
In the connection table, you can see the closed connection with the data "transferred bytes", "received bytes" and "size" of the synchronization data more than "transferred bytes". When synchronizing data, the socket.io automatic compression is used; when downloading or downloading attachments, automatic compression is disabled, as often compressing an already compressed negatively affects the speed.
As written in the beginning of the article, it was never the goal to make one common service and connect as many users as possible to it. But it was always interesting how many users there could theoretically be and how much the server could handle.
For this, I made a simple test: Online clients of a certain room (group), a message is sent with test parameters (there are only one main parameter in the parameters - the number of messages), which each client will send to other clients online. After the test distribution, the total time of all messages, notifications, records in postgresql tables, the number of all messages, notifications and records is calculated, and accordingly, the speed is calculated, including the total of all operations on the server.
Example from the screenshot:
sometimes surprising, only 10,000 messages and 390,000 gestures,
sending 50,000 messages: Server processes: 4 Users: 3
Total messages received 10,000 * 3 = 30,000
Forwarding to other server processes 30,000 * Server processes: 4 - 1 = 90,000
Отправлено адресатам 30 000 * Пользователи: 3 — 1 = 60 000
Записано сообщений в таблицу postgreSQL 30 000
Получено уведомлений доставки 90 000
Записано в таблицу postgreSQL уведомлений доставки 90 000
Все операции 390 000
Логика данных в push0k admin
With each connection, all reference data is obtained: users, rooms, devices, databases, as well as the last 300 "log-statistical" entries: Connections, Data Synchronization, Forwarding Attachments, Messages, Log.
Messages practically repeat the postgresql table and do not have filters, settings and a form where you can see the message completely. The logic of the table is debugging, it is possible to quickly look at the development whether the message reached the table or not.
The "Users", "Rooms (groups)", "Connections" and "Log" tables are updated automatically. For the rest of the data does not make sense in the online update.
The push0k admin application can be downloaded here:
Windows: push0kadmin Setup 19.1.11.exe
Mac OS: push0kadmin-19.1.11.dmg
It is free, but unlike the server, I don’t plan to open the source code.
In the next article I will describe the client part of the push0k desctop client, as well as a small example, the code and how to connect, log in and get data from the server.
For the previously mentioned earlier versions, there was also an android client, immortally living in the background on 7th and 8th versions of android. But he still, I think not long behind. Later, I think, there will be a third article with the android client, and then you look close to iOS.