Real-Time IoT Solution Architecture

    We continue to talk about solution companies (ISVs). In this issue, the technical director of INPROSISTEM AlexandrSurkov talks about the experience in developing the architecture of the CECMIC security IoT system .



    Many people believe that the concept of the “Internet of things” is inextricably linked with the network that we use every day. You can imagine a picture where many devices, combined into a single whole through a global network, exchange data between themselves and servers and create a digital picture of the world. In this article I will talk about how we made a system that combines hundreds of sensors.


    The concept of “Internet” should be considered much broader. In this case, it is a network common to devices. It can contain 10 devices, and maybe 10,000. It can be wired, or maybe wireless. It can be located in one room, and can cover several countries. It all depends on the tasks that are set for the system.

    Moreover, the creation of even a small network of devices is accompanied by many difficulties.

    Formulation of the problem



    We were tasked with developing a perimeter security system. A perimeter is a fence surrounding an object. Its length is unlimited.

    The system was created from scratch. By the time the design began, there was a prototype sensor that was able to collect perimeter vibrations, analyzing which, it was possible to clearly determine the fact of overcoming or breaking the fence. Empirically, we determined that sensors should be installed approximately every 10 meters.

    In addition to sensors, control devices with relays and controlled devices with a “dry contact” were also planned. The system should work in outdoor conditions with a wide range of temperatures and weather conditions.

    So there is:
    • 3 types of devices;
    • At least 100 devices per kilometer;
    • The number of kilometers is not limited;
    • The system must have street performance.


    You can immediately highlight the main questions on architecture:
    • Organization of data and power transmission;
    • Distribution of information flows: where and how to analyze data;
    • Security solutions: which protocols to use;
    • How to manage so many devices.


    General solution scheme



    Clemens Vasters wrote an excellent article about what the developer of the Internet of Things system is facing. We had to solve the same problems.

    Many decisions were dictated by GOSTs and other requirements for such systems. But a lot had to be invented by ourselves.

    First, it was necessary to understand how to distribute information flows in the system. For the analysis of oscillations, both the time and frequency components were used. Frequency range from 0 to 500 hertz. Means, measurements need to be done 1000 times per second. Each measurement is done in 3 axes of 10 bits each.
    Total 3 * 10 * 1000 = 30 kilobits per second from only one sensor. For a kilometer (100 sensors) this is 3 megabits.

    You can transfer this data, but how to process it? A perimeter of 10 km would already give 30 megabits of data per second. It turns out that the server load increases with the number of devices.

    It was decided that each sensor will process the data itself and give out only alarm facts. This significantly reduced the amount of data and distributed the computing load.

    Networking was chosen wired due to the fact that wireless requires a battery. Since we have a real-time system, the battery life will be very short. In addition, negative temperatures greatly reduce the capacity of the supply elements. And damping wireless communications is relatively easy, and this is already a very significant drawback for a security system.

    GOSTs prohibit laying 220V along the fence, so our devices will be “powered” by constant voltage.

    We have chosen a bus as the network architecture. This allowed us not to pull the wire to each sensor.

    However, the bus imposes a limitation on the network length and the number of devices. Therefore, a new device was introduced: a gateway. It has 2 buses, is engaged in device management and data transmission to and from the server. He is also involved in monitoring nutrition and environmental parameters.
    Thus, another big load is removed from the server: sensor management.

    In addition, this modular approach with distributed computing load allows a very large number of sensors to be integrated into a common system without significantly increasing server requirements.

    The remaining actuators, as well as sensors, are connected to the bus.

    The result is such a scheme:



    Choice of protocols and methods of interaction

    Now it was necessary to determine how the data will be transmitted. Each device in the system participates in 2 types of exchange:
    • command from the server - response to the server;
    • Asynchronous event from device to server.


    Commands are used to change the state of a device and configure it. Events are generated if the device displays information for the server (for example, an alarm from the sensor).

    On the server side, the gateway is considered to be the same executive device as all others, since it, in addition to managing devices on the bus, also monitors power parameters and temperature.

    As a bus, the choice was mainly between RS485 and CAN . Only these two standards made it possible to connect many devices at sufficiently large distances.

    As a result of the analysis, we preferred CAN. The network settings were selected as follows:
    • 500 meters maximum;
    • Maximum 100 devices;
    • 50 kbps speed.


    In our opinion, this is the optimal balance of speed, device density and network length for our system.

    The gateway uses a microcontroller with two CAN drivers on board, which made it possible to make 2 flanks and close 1 km with one gateway.

    CAN also has other advantages: noise immunity, resolution of collisions and guaranteed delivery of packets to the addressee. In addition, it has a fairly strong model for diagnosing network errors.

    However, CAN represents only the link layer of the network. For direct work with it, there are many protocols of a higher level. The most famous of them is CANOpen .

    Having studied the various options, we came to the conclusion that none of the protocols suits us. Some are too complicated for our task, some require deductions for use, and some do not support what we would like to implement.

    And I wanted to implement the system according to the PlugAndPlay principle :
    • Connect and disconnect devices without turning off the power;
    • Automatic detection of system configuration changes;
    • The system should start working immediately after assembly
    .

    In the end, we managed to do what was planned by writing our own simple but powerful enough protocol. Since we are talking about a small bus bandwidth and a small computing power of microcontrollers, the protocol type was chosen byte. Due to the optimization of bandwidth, the protocol turned out to be quite strongly connected with CAN, but we managed to make it theoretically portable to other standards.

    The protocol allows you to:
    • Detect connected devices on the fly
    • Detect device disconnection
    • work in request-response mode;
    • transmit asynchronous events;
    • Transfer streaming data from the device.


    Having established the exchange between devices and the gateway, it remains to deal with the server.

    The gateway has an Ethernet output . This is the most versatile data transfer technology. A network can be organized as you like: optical channels, wireless channels, a regular twisted pair cable - at the same time, we can always connect to this network using optical converters and access points. This allows the customer to design the network infrastructure of any complexity and length.

    Data transfer was organized using Berkeley Socketsbased on TCP / IP. This solution allows the server to be guaranteed to receive information from any sensor and not depend on software platforms. We developed our own protocol over TCP / IP. It is also byte, to optimize the work on the side of the microcontroller. Byte protocols have a big minus: complexity with the subsequent modification. However, the text protocol for the microcontroller device is too redundant.

    The most difficult in terms of software development was the server. We implemented an asynchronous multi-threaded interaction model, which allowed us to get a “live” system that instantly responds to any changes. Connecting a new device, loss of connection with the gateway, alarm from the sensor, opening the cover on the device - any event in the system is instantly recorded, even if they occur simultaneously.

    As a result, we got a flexible modular system managed through a single center - a server. It also has its own protocol, allowing you to connect to it and receive events in the system. This allows us to use our system as an integral part of a large complex and scale it almost to infinity.



    Security questions



    With system security, everything turned out to be quite simple. The fact is that all networks that are located on protected sites are themselves protected sites. Thus, all networks with which the system works become “trusted”.

    In addition, the “price” of hacking an information security system is much higher than other methods of overcoming it. In other words, an experienced intruder will find an easier way to overcome the obstacle, while a less experienced one will simply not be able to crack the system.

    Therefore, we did not use any special methods of information protection, limiting ourselves only to basic principles.

    What's next?



    Despite the fact that the system has already been formed, we continue to actively develop it and look for new ways to use it.

    One of the directions in the development of the system is machine learning. Using these algorithms, you can filter out regular noise such as noise from truck trains and planes. In experiments for this direction, Azure Machine Learning helps us a lot. It contains many ready-made solutions for machine learning, which allows you to quickly get results.
    Analysis of fencing fluctuations is far from the only way to use the technologies embedded in our system. The vibration control of high-rise buildings, pipelines and gas pipelines, fragile goods, vibration diagnostics of turbines and moving parts of various designs is not a complete list of possibilities.

    The number of sensors in such systems will only increase, and here we will hardly miss the transition to cloud systems at sites for which the use of the Internet is not prohibited.

    The new Microsoft IoT technologies seem very promising. A single Windows platform can theoretically save a lot of time, since you can write code that is common for different hardware platforms.

    And for data processing use Azure IoT Suite. According to the developers, it contains tools that allow not only to combine and manage many IoT devices, but also to process large amounts of data from them. This is what we are going to check in the near future.

    Conclusion



    When we started developing the system, the concept of the “Internet of Things” had not yet gained such popularity. There was little experience, with many things we came across for the first time. Now that many people write and talk about this concept, it has become clear that the right path has been chosen.

    The work was difficult and long. Creating the first commercial version of the system took about 3 years. The first went to the development of engineering samples of individual devices. Another year was spent on developing the system as a whole. The third year was debugging and debugging.

    During this time, we have gained vast experience in solving a variety of engineering problems. Moreover, the selection of cases, cable products, organization of production and logistics took no less effort than the development of the system itself.

    Now the system is mounted and works at many facilities in Russia and abroad. The largest of them consists of several perimeters with a total length of more than 15 km. In the design are also larger objects.

    More detailed information is available on the system website .

    Also popular now: