Build time, network speed and routing: how we improved our mesh network and a bit about neural networks
On this resource, they regularly write about mesh networks. We want to tell how we use them in our project to create an adaptive room automation system. The technology of self-organizing networks allows us not only to quickly deploy the system, but also to achieve high indicators of its fault tolerance and speed. We successfully tested our solution on a real site with a network capacity of 120 devices.
Since the advent of the concept of “smart home”, the head unit has been considered its “heart” and “brain”. But gradually, the hubs cease to be necessary and soon completely disappear into the past. Major players like Google and Amazon also confirm this trend. Google has been offering cellular routers for several years, and Amazon recently bought Eero to strengthen its position in the smart home market.
In our products, we use mesh networks, as they solve several problems at once. Firstly, they ensure the smooth operation of the system. Usually, if the hub crashes or loses its connection to the Internet, the system actually shuts down, the devices stop responding and perform their basic functions. Mesh networks avoid this. Even if one device fails, the rest will continue to work in the same mode. The user will only receive a notification of the malfunction.
Secondly, mesh networks help route traffic. In our system, all devices are able to relay the signal so that the network coverage area becomes almost unlimited. In the event of a network configuration change, the system will independently determine the optimal route.
In a network created on the basis of our solution, an almost unlimited number of devices can work at a considerable distance from each other. Obtained by the calculation method, this figure is 10 to 10 degrees, which is currently impossible to provide in view of the limited amount of RAM in the devices.
Most of the solutions on the market today are simply a set of sensors and actuators that are limited in capabilities. Typically, this is a remote control in the form of a mobile application, a separate tablet or a smart speaker, which allows you to set the temperature, turn on / off household appliances, adjust lighting and set up primitive battery life scenarios. Let's admit that the “smart” is not enough, this is just another toy.
Therefore, we set ourselves the goal of creating an adaptive home automation system based on a mesh network. As you use it, our system analyzes user habits and behavior. Due to this, over time, she begins to adapt all scenarios and settings for a specific user, offer new scenarios and, after the approval of the owner, configure them herself.
The system architecture is designed in such a way that a head box is not required. A separately taken device can work by itself, using all the functionality embedded in it.
Another feature of our solution: before making a decision, the system collects data from several sources at once (for example, from a camera, air quality sensor, presence sensor, etc.). This has become possible due to the fact that all devices of the system are self-sufficient and are able to exchange information among themselves directly. Which allowed us to significantly reduce the likelihood of incorrect implementation of actions within the framework of a given scenario.
All information is processed inside the system (and not by the server or the central hub) in a distributed way. What is meant? If one of the devices with autonomous power supply needs to perform complex and energy-intensive calculations, the system will redirect this task to the nearest device with constant mains power. This approach provides an acceptable battery life for self-powered devices and speeds up the decision process.
I would also like to note the simplicity of the connection procedure. To do this, you do not need to configure anything, scan QR codes and more. It is enough to attach the phone with the application to the device - and that’s it: it’s already on the network, received the basic settings and is ready to go!
Initially, we wanted to use the affordable Espressif ESP-MESH solution. But after analyzing it, we realized that it does not quite suit us. Therefore, I had to invent my own. Our key advantages are construction time, automatic channel change when it is loading, network speed and routing.
Self-healing is the automatic detection and elimination of network failures. If suddenly some device fails and the signal is lost, the network from the remaining devices will rebuild on its own. The network assembly from the moment it is turned on takes up to 15 seconds, and the rebuild takes no more than 6 seconds. Support between nodes for redirection is less than 10 milliseconds. And these are real numbers obtained during testing at one of our large facilities. In general, this is an average indicator for military facilities and excessive in our segment. We can say we have a huge supply!
Another important parameter is routing, how the signal is transmitted from one device to another. Usually, all elements of similar solutions operate at the same frequencies as Wi-Fi - 2.4 Hz. Protocols use algorithms based on the state of the air channels. Our solution allows you to change the channel without affecting network performance. For example, at the facility, all devices around the perimeter operate on channel 13. But in the center of the room on this channel there was interference. In this case, the device that transmits the signal will automatically change the channel and bypass the problem area. The rest of the devices equipped with our module are organized among themselves and choose the optimal frequencies.
To achieve a high routing speed, we have optimized all the software for the hardware. For each processor and coprocessor they wrote code in Assembler. This allowed us to reduce power consumption and increase network speed by 15 times.
The theoretical maximum capacity of our network is huge. Obviously, in practice we cannot verify it. During the tests, we connected more than 300 devices to the network, and the system worked stably. That is, our solution will make it easy to cover the shopping center and you don’t have to ditch the walls, lay additional wiring, and configure the network for a long and painful time. It is enough to install our devices where necessary. Even if it is on a mobile site, they will always be online.
For the safety of user data, we use double encryption, which is based on the AES protocol, recognized as one of the most reliable. Traffic is tunneled between the server and the device or between a pair of devices. This eliminates the possibility of gaining access to the system by hacking a WIFI network.
We have applied for a patent for a utility model for applying our self-organizing network algorithm to automate rooms. She has undergone a preliminary assessment and is now under consideration.
Our team carried out a large-scale project to modernize the lighting of the Khimik ice palace in Novopolotsk. There, all the lighting equipment (in the amount of 120 units) is made with the function of remote control of light intensity and remote on / off access via Wi-Fi and Bluetooth.
It is also possible to configure stand-alone work scenarios. For example, in standby mode, the level of illumination, in order to save electricity, should not exceed 100 lux. During training - 600. And during a hockey match with broadcast this indicator should be at least 1200 lux. To install the system, it was not necessary to lay additional wiring, change the network configuration or other costs. The system architecture is completely built on the basis of our mesh-network. It was enough to install new LED lamps with control units.
After modernization, the ice arena in terms of illumination began to meet the stringent requirements of the Hockey Federation of the Republic of Belarus for direct international television broadcasts.
In our decision, we also use artificial intelligence technologies, in particular, neural networks. As noted earlier, all the data is collected by the modules installed in the devices of the room automation system and all calculations are carried out inside the system. Therefore, the more devices in the system, the better and faster they distribute complex calculations among themselves.
As a rule, developers train neural networks on large data sets. But in our case, the training is specific. Our neural network does not need to know what, conditionally, African monkeys look like. At the same time, triggers are needed to train the network. We figured out how to make the devices learn on certain events at the hardware level, the system purposefully studies user habits. At first, the network implements well-thought-out scenarios in order to understand what conditions are most comfortable for users. Then, using the accumulated experience, she selects the lighting, heating and ventilation modes that are most comfortable for a particular user. Over time, the system takes over part of the functions of managing a house, apartment or other facility.
Our task is to make users as comfortable as possible and they themselves practically do not change anything in the system settings, they are all happy.
Due to the fact that the system is completely autonomous, data is stored in the non-volatile memory of the devices themselves, the operation of the system does not depend on Internet access. If he suddenly disappeared, she would continue to work, offer scripts and self-study. In our case, the Internet is needed only for organizing remote access and communication with the server if the user wants to store certain data in the cloud (for example, recordings from video cameras).
By the end of the year, we plan to partially release a line of devices for our adaptive room automation system. It will include almost everything listed above, with the exception of video cameras.
A video camera with elements of technical vision is currently under development. We plan to transmit video data from several cameras at once. We are also trying to introduce an innovative approach. Our solution uses two neural networks to detect and track people in a room (the multitracking task is being solved) in real time using CUDA technology. To accompany, a neural network model was proposed and trained, which, according to the test results, surpasses DeepSort in four main parameters, which showed the best results in the MOT16 competition, as part of the MOTChallenge.
It is known that designing models of neural networks requires significant time resources for development, it is also necessary to conduct many experiments. Thus, the system not only knows the number of people in the room, but also provides their tracking, which works stably even in situations when people leave the frame and then come back. Our neural network already recognizes people by 128 signs and determines them according to the “friend / foe” system. To date, the implementation of works on adapting the technology of escorting people using video sequences obtained from different cameras indoors is promising. Over time, we plan to add a definition of non-standard patterns of behavior (for example, falling or fighting) and the detection of flames or smoke. Developments in these areas are already available.
Why do we need a mesh network
Since the advent of the concept of “smart home”, the head unit has been considered its “heart” and “brain”. But gradually, the hubs cease to be necessary and soon completely disappear into the past. Major players like Google and Amazon also confirm this trend. Google has been offering cellular routers for several years, and Amazon recently bought Eero to strengthen its position in the smart home market.
In our products, we use mesh networks, as they solve several problems at once. Firstly, they ensure the smooth operation of the system. Usually, if the hub crashes or loses its connection to the Internet, the system actually shuts down, the devices stop responding and perform their basic functions. Mesh networks avoid this. Even if one device fails, the rest will continue to work in the same mode. The user will only receive a notification of the malfunction.
Secondly, mesh networks help route traffic. In our system, all devices are able to relay the signal so that the network coverage area becomes almost unlimited. In the event of a network configuration change, the system will independently determine the optimal route.
In a network created on the basis of our solution, an almost unlimited number of devices can work at a considerable distance from each other. Obtained by the calculation method, this figure is 10 to 10 degrees, which is currently impossible to provide in view of the limited amount of RAM in the devices.
What is a "smart home" (in our opinion)
Most of the solutions on the market today are simply a set of sensors and actuators that are limited in capabilities. Typically, this is a remote control in the form of a mobile application, a separate tablet or a smart speaker, which allows you to set the temperature, turn on / off household appliances, adjust lighting and set up primitive battery life scenarios. Let's admit that the “smart” is not enough, this is just another toy.
Therefore, we set ourselves the goal of creating an adaptive home automation system based on a mesh network. As you use it, our system analyzes user habits and behavior. Due to this, over time, she begins to adapt all scenarios and settings for a specific user, offer new scenarios and, after the approval of the owner, configure them herself.
The system architecture is designed in such a way that a head box is not required. A separately taken device can work by itself, using all the functionality embedded in it.
Another feature of our solution: before making a decision, the system collects data from several sources at once (for example, from a camera, air quality sensor, presence sensor, etc.). This has become possible due to the fact that all devices of the system are self-sufficient and are able to exchange information among themselves directly. Which allowed us to significantly reduce the likelihood of incorrect implementation of actions within the framework of a given scenario.
All information is processed inside the system (and not by the server or the central hub) in a distributed way. What is meant? If one of the devices with autonomous power supply needs to perform complex and energy-intensive calculations, the system will redirect this task to the nearest device with constant mains power. This approach provides an acceptable battery life for self-powered devices and speeds up the decision process.
I would also like to note the simplicity of the connection procedure. To do this, you do not need to configure anything, scan QR codes and more. It is enough to attach the phone with the application to the device - and that’s it: it’s already on the network, received the basic settings and is ready to go!
Key features of our mesh network
Initially, we wanted to use the affordable Espressif ESP-MESH solution. But after analyzing it, we realized that it does not quite suit us. Therefore, I had to invent my own. Our key advantages are construction time, automatic channel change when it is loading, network speed and routing.
Self-healing is the automatic detection and elimination of network failures. If suddenly some device fails and the signal is lost, the network from the remaining devices will rebuild on its own. The network assembly from the moment it is turned on takes up to 15 seconds, and the rebuild takes no more than 6 seconds. Support between nodes for redirection is less than 10 milliseconds. And these are real numbers obtained during testing at one of our large facilities. In general, this is an average indicator for military facilities and excessive in our segment. We can say we have a huge supply!
Another important parameter is routing, how the signal is transmitted from one device to another. Usually, all elements of similar solutions operate at the same frequencies as Wi-Fi - 2.4 Hz. Protocols use algorithms based on the state of the air channels. Our solution allows you to change the channel without affecting network performance. For example, at the facility, all devices around the perimeter operate on channel 13. But in the center of the room on this channel there was interference. In this case, the device that transmits the signal will automatically change the channel and bypass the problem area. The rest of the devices equipped with our module are organized among themselves and choose the optimal frequencies.
To achieve a high routing speed, we have optimized all the software for the hardware. For each processor and coprocessor they wrote code in Assembler. This allowed us to reduce power consumption and increase network speed by 15 times.
The theoretical maximum capacity of our network is huge. Obviously, in practice we cannot verify it. During the tests, we connected more than 300 devices to the network, and the system worked stably. That is, our solution will make it easy to cover the shopping center and you don’t have to ditch the walls, lay additional wiring, and configure the network for a long and painful time. It is enough to install our devices where necessary. Even if it is on a mobile site, they will always be online.
For the safety of user data, we use double encryption, which is based on the AES protocol, recognized as one of the most reliable. Traffic is tunneled between the server and the device or between a pair of devices. This eliminates the possibility of gaining access to the system by hacking a WIFI network.
We have applied for a patent for a utility model for applying our self-organizing network algorithm to automate rooms. She has undergone a preliminary assessment and is now under consideration.
An example of the implementation of our solution
Our team carried out a large-scale project to modernize the lighting of the Khimik ice palace in Novopolotsk. There, all the lighting equipment (in the amount of 120 units) is made with the function of remote control of light intensity and remote on / off access via Wi-Fi and Bluetooth.
It is also possible to configure stand-alone work scenarios. For example, in standby mode, the level of illumination, in order to save electricity, should not exceed 100 lux. During training - 600. And during a hockey match with broadcast this indicator should be at least 1200 lux. To install the system, it was not necessary to lay additional wiring, change the network configuration or other costs. The system architecture is completely built on the basis of our mesh-network. It was enough to install new LED lamps with control units.
After modernization, the ice arena in terms of illumination began to meet the stringent requirements of the Hockey Federation of the Republic of Belarus for direct international television broadcasts.
Why do we need neural networks and how do we train them
In our decision, we also use artificial intelligence technologies, in particular, neural networks. As noted earlier, all the data is collected by the modules installed in the devices of the room automation system and all calculations are carried out inside the system. Therefore, the more devices in the system, the better and faster they distribute complex calculations among themselves.
As a rule, developers train neural networks on large data sets. But in our case, the training is specific. Our neural network does not need to know what, conditionally, African monkeys look like. At the same time, triggers are needed to train the network. We figured out how to make the devices learn on certain events at the hardware level, the system purposefully studies user habits. At first, the network implements well-thought-out scenarios in order to understand what conditions are most comfortable for users. Then, using the accumulated experience, she selects the lighting, heating and ventilation modes that are most comfortable for a particular user. Over time, the system takes over part of the functions of managing a house, apartment or other facility.
Our task is to make users as comfortable as possible and they themselves practically do not change anything in the system settings, they are all happy.
Due to the fact that the system is completely autonomous, data is stored in the non-volatile memory of the devices themselves, the operation of the system does not depend on Internet access. If he suddenly disappeared, she would continue to work, offer scripts and self-study. In our case, the Internet is needed only for organizing remote access and communication with the server if the user wants to store certain data in the cloud (for example, recordings from video cameras).
Vision Implementation Plans
By the end of the year, we plan to partially release a line of devices for our adaptive room automation system. It will include almost everything listed above, with the exception of video cameras.
A video camera with elements of technical vision is currently under development. We plan to transmit video data from several cameras at once. We are also trying to introduce an innovative approach. Our solution uses two neural networks to detect and track people in a room (the multitracking task is being solved) in real time using CUDA technology. To accompany, a neural network model was proposed and trained, which, according to the test results, surpasses DeepSort in four main parameters, which showed the best results in the MOT16 competition, as part of the MOTChallenge.
It is known that designing models of neural networks requires significant time resources for development, it is also necessary to conduct many experiments. Thus, the system not only knows the number of people in the room, but also provides their tracking, which works stably even in situations when people leave the frame and then come back. Our neural network already recognizes people by 128 signs and determines them according to the “friend / foe” system. To date, the implementation of works on adapting the technology of escorting people using video sequences obtained from different cameras indoors is promising. Over time, we plan to add a definition of non-standard patterns of behavior (for example, falling or fighting) and the detection of flames or smoke. Developments in these areas are already available.