How the Internet of Things is pushing analytics to the edge
- Transfer
In the era of the Internet of Things (IoT), the tasks of processing, filtering and analyzing data as close to their source as possible are becoming more relevant, since this allows you to act remotely instead of sending this data to data centers (DPC) or the cloud for filtering and analysis.
Another reason for deploying analytics tools on the periphery of the network is that today there are ever new applications for the Internet of things, and in many cases the amount of data generated on the periphery places such high demands on bandwidth and processing power that the available resources are not able to satisfy them.
That is, data centers designed to solve more traditional corporate problems may not cope with data streams from smart devices, sensors, etc.
For example, if the readings of the temperature sensor of the wind turbine engine are within the operating range, it is not necessary to save them with an interval of one second, because a huge amount of data will accumulate very quickly. It is better to adhere to an approach where readings that fall outside the operating range or indicate certain trends (for example, an inevitable malfunction of the component) will generate a warning and, possibly, remain centralized only after identifying this first anomaly for subsequent analysis.
There are too many suppliers in this market to make an exhaustive list. However, it is worth mentioning here that last year the company, formerly known as JustOne Database, conducted a comprehensive rebranding covering all branches. Not only products were renamed, but the company itself. Now it is called Edge Intelligence. A company representative told me that their database, which can run on relatively compact servers on the periphery of the network, in the data center or in the cloud, became so popular that the company decided to rebrand after six years of operation.
So, what do you need to know about peripheral analytics if you are trying to decentralize at least part of the analytic tasks?
Some standards in this area are likely to disappear, however, by focusing on technologies that support standards, integration is likely to be simplified in the future. But there are a lot of specialized standards and APIs. Standards and protocols include POSIX and HDFS APIs for file access, SQL for queries, Kafka APIs for event streams, HBase, and possibly the OJAI API (JSON Open API) for compatibility with NoSQL databases. Support for legacy proprietary telemetry protocols is also needed so that legacy equipment (which has often been in use for decades) can be incorporated into more modern IoT infrastructures. This is especially important in an industry where the Internet of Things is of particular value because it helps improve the effectiveness of preventative maintenance.
In a sense, this is the basis of peripheral analytics, since it implies the provision of high-speed local data processing, which is especially important if there are restrictions on the geographical basis or in terms of information confidentiality (for example, when working with personal data). Aggregation can also be used to consolidate data from peripheral IoT devices.
This applies to technologies that regulate network bandwidth between peripheral devices and the cloud and / or data center, even if sensors or devices are connected from time to time.
Making operational decisions and real-time data analysis on the periphery.
Comprehensive IoT security features provide authentication, authorization, and access control both at the edge and at the central cluster level. Under certain circumstances, it is desirable to provide encryption in the data channel between objects on the periphery and the main data center. Identity control is also a complex problem: tools are needed to manage things, including authentication, authorization, and privileges within or outside the boundaries of the system and the enterprise itself.
A reliable computing environment is being formed that can cope with numerous equipment failures that can occur in remote isolated systems.
If not today, then in the future it may be necessary to provide reliable integration between the peripheral analytics node and the cloud. This means that warnings and even “basic” data points can be stored in the cloud, and not in the company's own data center. Therefore, integration with your cloud computing service provider (if any) will be a very forward-thinking solution. If you practically do not use the clouds for data processing and storage, in the future you may still be interested in Amazon Web Services and Google Cloud Platform or Microsoft Azure cloud platforms; In addition, it will be useful for you to find out about the availability of support for the OpenStack infrastructure with open source code in the framework of the concept of "infrastructure as a service" (Infrastructure as a Service, IaaS).
In the past few years, peripheral analytics is becoming more widespread as new uses for the Internet of things appear. It is worth at least analyzing the possible role of peripheral computing in projects in the field of the Internet of things that you plan to implement.
Another reason for deploying analytics tools on the periphery of the network is that today there are ever new applications for the Internet of things, and in many cases the amount of data generated on the periphery places such high demands on bandwidth and processing power that the available resources are not able to satisfy them.
That is, data centers designed to solve more traditional corporate problems may not cope with data streams from smart devices, sensors, etc. For example, if the readings of the temperature sensor of the wind turbine engine are within the operating range, it is not necessary to save them with an interval of one second, because a huge amount of data will accumulate very quickly. It is better to adhere to an approach where readings that fall outside the operating range or indicate certain trends (for example, an inevitable malfunction of the component) will generate a warning and, possibly, remain centralized only after identifying this first anomaly for subsequent analysis.
There are too many suppliers in this market to make an exhaustive list. However, it is worth mentioning here that last year the company, formerly known as JustOne Database, conducted a comprehensive rebranding covering all branches. Not only products were renamed, but the company itself. Now it is called Edge Intelligence. A company representative told me that their database, which can run on relatively compact servers on the periphery of the network, in the data center or in the cloud, became so popular that the company decided to rebrand after six years of operation.
So, what do you need to know about peripheral analytics if you are trying to decentralize at least part of the analytic tasks?
Standards and protocol translation
Some standards in this area are likely to disappear, however, by focusing on technologies that support standards, integration is likely to be simplified in the future. But there are a lot of specialized standards and APIs. Standards and protocols include POSIX and HDFS APIs for file access, SQL for queries, Kafka APIs for event streams, HBase, and possibly the OJAI API (JSON Open API) for compatibility with NoSQL databases. Support for legacy proprietary telemetry protocols is also needed so that legacy equipment (which has often been in use for decades) can be incorporated into more modern IoT infrastructures. This is especially important in an industry where the Internet of Things is of particular value because it helps improve the effectiveness of preventative maintenance.
Distributed Data Aggregation
In a sense, this is the basis of peripheral analytics, since it implies the provision of high-speed local data processing, which is especially important if there are restrictions on the geographical basis or in terms of information confidentiality (for example, when working with personal data). Aggregation can also be used to consolidate data from peripheral IoT devices.
Bandwidth analysis
This applies to technologies that regulate network bandwidth between peripheral devices and the cloud and / or data center, even if sensors or devices are connected from time to time.
Converged Analytics
Making operational decisions and real-time data analysis on the periphery.
Security & Identity Management
Comprehensive IoT security features provide authentication, authorization, and access control both at the edge and at the central cluster level. Under certain circumstances, it is desirable to provide encryption in the data channel between objects on the periphery and the main data center. Identity control is also a complex problem: tools are needed to manage things, including authentication, authorization, and privileges within or outside the boundaries of the system and the enterprise itself.
Enterprise Reliability
A reliable computing environment is being formed that can cope with numerous equipment failures that can occur in remote isolated systems.
Cloud integration
If not today, then in the future it may be necessary to provide reliable integration between the peripheral analytics node and the cloud. This means that warnings and even “basic” data points can be stored in the cloud, and not in the company's own data center. Therefore, integration with your cloud computing service provider (if any) will be a very forward-thinking solution. If you practically do not use the clouds for data processing and storage, in the future you may still be interested in Amazon Web Services and Google Cloud Platform or Microsoft Azure cloud platforms; In addition, it will be useful for you to find out about the availability of support for the OpenStack infrastructure with open source code in the framework of the concept of "infrastructure as a service" (Infrastructure as a Service, IaaS).
In the past few years, peripheral analytics is becoming more widespread as new uses for the Internet of things appear. It is worth at least analyzing the possible role of peripheral computing in projects in the field of the Internet of things that you plan to implement.