Tibbo AggreGate Concept - IoT Platforms

    The development of the Internet quickly reaches a level where it is "just there." Increasingly, we don’t have to think about how exactly we are connected to the network, who our communication operator is, and, moreover, how the connection is organized technically. The widespread penetration of wireless networks and the gradual spread of IPv6 allows thousands of simple devices and sensors to continuously communicate with each other and send their data “to the cloud”. The rapid complexity of the infrastructure has led to the replacement of the term “Machine-to-Machine” (Machine-to-Machine, M2M) with the more current term “Internet of things” (Internet of things, IoT).



    Forming something like distributed intelligence, Internet of Things devices nevertheless need centralized management, a system or service to configure, store and interpret the collected data. Being the central “brain” of the device cloud infrastructure, the control system also replenishes the machine knowledge bases of connected devices and updates their software.



    Operators study data aggregated by device groups and time periods, visualizing them in various sections. The same data is transferred for more detailed analysis to various systems of the Business Intelligence class. It is curious that even when it comes to personal devices (for example, fitness trackers), few cloud service operators deny themselves the possibility of anonymous analysis of statistics on the use and use of collected data to further improve devices and services.

    The development of devices for the Internet of things is constantly being simplified and becoming cheaper, allowing small companies to enter this market. Recognizing the need to create a management system, many companies underestimate the complexity of its development, do not take into account the need to use industrial server technologies (such as clustering for fault tolerance or multi-server distributed architecture). Often, development begins on its own, in house. The success of IoT devices on the market leads to a rapid increase in the number of users, creating quickly unsolvable problems with scaling the service and ensuring its performance.

    Fearing further problems and not being able to quickly create a server software development department, IoT operators often outsource the development of a central system, concentrating on the devices themselves. But this does not solve the problem - third-party developers begin to create a system from scratch in the same way and do not have sufficient resources and time to introduce serious technologies into its foundation.

    Our AggreGate Platformwas born in the year 2002. In those years, we were actively engaged in the production of Serial-over-IP converters, and we needed a central server transferring data between the converters hidden behind firewalls or NAT and not having the ability to communicate directly. The first version of the product, called LinkServer, was written in C ++ and is only available as a service. LinkServer simply pumped data streams through itself without any processing.

    A little later, our converters turned into freely programmable controllers. They began to “understand” the data passing through them and a logical desire arose to teach the central server the same thing. Around the same time, we realized that 90 percent of the work on creating a monitoring and device management system for any industry is “the invention of the bicycle,” and only a small amount of effort is spent on tasks unique to a particular business.

    In 2004, the system was ported to Java. From that moment, it developed as a "device management framework." For several years we moved almost blindly, not fully understanding what we want to get as a result. Fortunately, we managed to avoid mono-customers, not to leave “with a head” in any particular industry and to preserve the universality of the system.

    Now the platform is used in various fields, such as remote monitoring and maintenance, IT infrastructure management and network monitoring, SCADA and process automation, physical access control, building automation, fleet management, vending and payment machine management, sensor network monitoring, counting visitors and cars, centralized management of events and incidents, management of advertising and information panels and mobile device management.

    The main objectives of the platform



    Figuratively speaking, AggreGate is a Lego constructor for quickly creating interfaces to the device cloud. Allowing architects of IoT solutions to concentrate more on equipment and business logic, it solves infrastructure problems:

    • Maintaining communications between servers and devices when working through unreliable cellular and satellite channels
    • Unification of work with these devices regardless of their physical meaning
    • Storage of large volumes of collected events and historical data in various types of databases (relational, ring, NoSQL)
    • Visual construction of complex chains of source data analysis and event correlation
    • Modeling the processes of combining data from multiple devices and calculating the KPI of the entire infrastructure
    • Fast construction of interfaces of operators and system engineers from ready-made cubes, without programming
    • Implementation of integration scripts using ready-made universal connectors (SQL, HTTP / HTTPS, SOAP, CORBA, SNMP, etc.)


    System integration



    Being universal, the platform helps to integrate various monitoring and control systems. This avoids unnecessary integration points and reduces the number of integration scenarios. For example, the integrated monitoring system has a single integration point with the Service Desk / ITSM / Maintenance Management system for the delivery of incidents (alarms), as well as a single integration with the Inventory / Asset Management system to obtain information about the available physical assets and the degree of their impact on business services .

    The organization of role-based access allows in such cases to provide different departments with individual scenarios for using the system and the uniqueness of the operator interface.

    Platform architecture



    The main components of the platform are:

    • Server is a Java application that provides communication with devices, data storage and their automated processing. Servers can be clustered to provide high availability and maintain peer-to-peer relationships with each other in distributed installations. The AggreGate server manages an embedded web server that provides web interface support.
    • A single console is a cross-platform desktop client software that provides simultaneous work with one or several servers in administrator, system engineer or operator mode.
    • Agent - a library that can be embedded in the device’s IoT firmware in order to provide communications with servers, unify device settings, perform operations with it, and send events asynchronously. There are many library implementations (Java, .NET, C / C ++, Android Java, etc.) Agent implementation is not required if communication with the server is carried out using a standard or proprietary protocol, in the latter case a separate device driver is developed for the server. An agent can also be configured as a separate hardware device (gateway).
    • An open source API to extend the functionality of all other components and implement complex integration scenarios




    The server manages the reading of data from devices and the recording of changes, this process is called two-way synchronization . On the server side, a so-called device snapshot is created containing copies of the latest device metrics and changes made by operators and system modules and not written to the device due to lack of communication. Configuration changes are delivered to devices on the basis of the principle of first opportunity, allowing you to make group changes to the configuration of devices without waiting for their simultaneous appearance online.

    The server also provides reception and maintenance of incoming connections of devices that do not have static "white" IP addresses. This feature is important for the IoT platform.

    Device data and events become partsingle data model . Within this model, each device is represented by a so-called context , which is part of the hierarchical structure of contexts. Each context includes formalized data elements of three types: variables (properties, settings, attributes), functions (methods, operations), and events (alerts). The context also contains metadata describing all available elements. Thus, all data and context metadata are completely concentrated within itself. This technology is called device normalization . A normalized presentation of various types of devices is created by device drivers and agents.

    You can draw parallels with object-oriented programming, where objects also provide properties, methods, and events. Properties are internal variables of a device, methods are operations that it can perform, and events are a way for a device to notify the server about changes in internal data or the external environment.

    Almost every device can be described by a set of properties, methods, and events. For example, a remotely controlled water tank may have the property “water level”, as well as the methods “open the valve” and “close the valve” to control the flow of liquid. The smart tank can also generate alerts, such as “almost empty”, “almost full”, and “overflow”. We have developed more than a hundred communication protocol drivers in Java, and the idea of ​​normalization has proven its worth. Moreover, many modern “universal” protocols (such as OPC UA, JMX or WMI) use very similar data models.

    All server contexts are arranged in a hierarchical structure called a context tree. Despite the fact that contexts correspond to different objects (devices, users, reports, alarms, etc.), they all have a common interface and can interact within the server’s context tree, providing a high level of flexibility. The same principle allows different servers to interact with each other in a distributed installation.

    Each connected device allows operators to perform direct configuration (reading and editing the device configuration), direct control (manual forcing the device to perform various operations) and direct monitoring (viewing events received from the device in the log in pseudo-real-time mode).



    Events and changes in device metrics are stored in server storage. Depending on the task of the system, this can be a storage of various types. For example, in the case of a micro server running inside Raspberry Pi, the simplest file storage is used, and the central server of a large distributed installation can use a cluster of Apache Cassandra NoSQL databases, ensuring the storage of tens of thousands of events per second from a thinned original stream of several hundred thousand events per second.

    However, in most cases, the regular relational database is used as storage. Using the ORM layer (Hibernate) provides compatibility with MySQL, Oracle, Microsoft SQL Server, PostgreSQL and other DBMSs.

    Data and events received from devices affect the life cycle of “active” server objects, allowing it to respond to changes in the external environment. Active objects include:

    • Alarms that convert a certain complex state of an object or a certain chain of events into a new type of event - Incident
    • Models that enable business rules to convert source events and values ​​into user-defined event types and values
    • Scheduler that guarantees scheduled tasks and their execution even in case of temporary server shutdown
    • Sensors and some other types of objects


    Active objects can add new types of variables, functions, and events to a single data model, provide for sending changes to user variables and events to the repository, and also trigger automated operations of devices and other objects.

    There are widgets for creating data entry forms, tables, dynamic maps, graphs and mnemonic diagrams. They can be combined into dashboards, both global, showing the state of the entire infrastructure and based on aggregated KPIs, and “object-based”, showing the status of one device or component of the infrastructure.



    Widgets and report templates are edited in specialized visual editors, which are part of the platform and are closely integrated into its ecosystem. The widget editor allows you to build complex interfaces consisting of many nested container panels with visual components located in them. In addition to the absolute positioning typical for the editors used in SCADA systems, you can use a grid layout that is familiar to anyone who has encountered editing tables inside HTML pages. The grid layout allows you to create data entry forms and tables that are neatly scalable to fit any screen size.

    As a result, the operator interface of the first or second line, developed visually using tools for visualizing data, consists of dashboards with widgets, forms, tables, diagrams, reports, mnemonic diagrams and navigation between them.



    The interface editor allows you to use dozens of ready-made components, such as labels, text fields, buttons, checkboxes, sliders and spinners, lists, date / time selectors, scales and pointers. There are more complex components, such as trees, video windows, dynamic vector SVG images, and geographic maps based on Google Maps / Bing / Yandex / OpenStreetMap. The list of supported chart types includes both classic chart types and various statistical charts, Gantt charts, and charts in polar coordinates.

    All widgets drawn in the editor work in the web version of the interface, including browsers without Java support, i.e. on mobile devices. Only HTML5 and JavaScript support is required.

    The properties of server objects (devices, models, alarms) and user interface components are connected to each other using bindings that determine when and where to get data, how to process it, and where to put the result. When processing data, bindings use an expression language and a query language.

    An expression-based binding is similar to a formula in Microsoft Excel. The formula takes data from several cells, applies mathematical operations or various data processing functions to them, and puts the result in the current cell. An expression is also a formula that describes where you need to get data from and what transformations you need to apply to them.

    The query language is very similar to regular SQL. It also allows you to combine data from various tables into one, applying filtering, sorting, grouping, etc. to them. The difference between the built-in query language and classical SQL is that instead of ordinary tables, the data sources are virtual tables formed on the fly from various data of a single model. Any query also automatically takes into account the access rights of the operator or system object executing it, which compares favorably with direct SQL query to the server database.

    To solve the most complex data processing tasks, you can write a Java script or even a separate plug-in. However, each script for data processing written by one of the partners is an alarming bell for us - why do we need a platform if you still need classic development, and even outside the usual environment (such as Eclipse or Idea)?

    I would also like to talk about the technical device of distributed architecture. Within the framework of this concept, peer-to-peer relationships between servers are set up in such a way that one server (“provider”) attaches part of its single data model to another server (“consumer”). This allows consumer server objects to interact with provider server objects on a par with their own. The number of connections per server is unlimited, and it can simultaneously act as a supplier and a consumer in relation to its neighbors.



    Distributed architecture allows you to solve many problems of large systems:

    • Horizontal system scaling
    • Distributed monitoring with the installation of local monitoring servers and intermediate data storage at remote sites
    • Vertical scaling, distribution of functions between servers at several logical levels


    This post is our debut on Habré. In the following articles, we plan to talk more about the internal structure of the platform, our own solutions created on its basis, as well as our understanding of how “heavy PC software” will get into embedded systems and devices of the Internet of things.

    PS If you are developing server software for the Internet of Things, please answer the question below.

    Only registered users can participate in the survey. Please come in.

    What do you develop server software for data collection and device management on?

    • 33.3% Java 20
    • 20% .NET 12
    • 15% PHP 9
    • 21.6% C / C ++ 13
    • 10% Other (write in the comments) 6

    Also popular now: