Notes of the IoT provider. Pitfalls of the survey of utilities meters
Hello dear Internet of Things lovers. In this article, I would again like to talk about housing and communal services and a survey of metering devices.
Periodically, another major telecom player tells how soon he will enter this market and crush everyone. Each time with such stories, I think: “guys, good luck!”
You have no idea where you are climbing.
In order for you to understand the scale of the problem, I will briefly describe a small part of our experience in developing the Smart City platform. The part of it that is responsible for scheduling.
If we are not talking about individual metering devices, but those that are in basements, boiler houses and at enterprises, then most of them are now equipped with a telemetric output. Less often pulsed, more often - RS-485/232 or Ethernet. As a rule, the most "bread" metering devices are those that consider heat. It is they who are ready to pay for their dispatching in the first place.
I have already dwelt in detail in my article on the features of RS-485. In short, it's just a data transfer interface. In essence - the requirements for electrical pulses and communication lines. The description of the packages goes up to the data transfer standard, which runs on top of RS-485. And what kind of standard will be there - it is left to the manufacturer. Often Modbus, but not required. Even if Modbus, it may still be slightly modified.
In fact, for each meter, you need your own survey script, which knows how to "talk" with it and interrogate it. This means that the dispatch system is a set of scripts for each individual counter. A database where all this is stored. And a certain user interface in which he can generate the report he needs.
It looks easy. The devil, as always, is in the details.
Let's start with the first part.
How to write them? Well, obviously, buy an meter, pick it up, learn how to communicate with it and integrate it into a common platform.
Unfortunately, this solution will cover only part of our needs. As a rule, a popular counter has several generations, and the script for each generation may differ. Sometimes a little, sometimes significantly. Buying something, you get the latest generation. The subscriber is very likely to be something more ancient. It is no longer sold in stores. And the subscriber will not change the metering node.
Hence the first problem. Writing such scripts is a tough bunch of software developers and engineers "on the ground." We bought the last generation, wrote some initial template and then modified it on real devices. To do this in the laboratory is unrealistic, only in the course of working with live subscribers.
It took us a lot of time to create such a bundle. Now the algorithm is worked out. Initial templates were constantly adjusted and supplemented, depending on what we met in our practice. Of course, the subscriber was warned if suddenly his counter was a little “not like that”. When such a device appears, it is connected according to the standard scheme and the polling script is modified along the way. At the time of integration, the subscriber works for free. He is notified that he still lives in test mode. The integration process itself is a rather unpredictable thing. It happens that you need to make a minimum of corrections. There is a complex process with a field trip, shoveling literature and consecutively overcoming a rake.
The task is not simple, but solvable. The result is a working script. The larger the library of scripts, the easier it is to live.
The second problem.
To make you aware of the complexity of this work, I will give an example. Take the extremely popular VKT-7 heat meter.
The name itself does not mean anything to us. VKT-7 has several iron solutions. What kind of interface is it inside?
There are different options. There may be an output in the standard DB-9 block (this is RS-232). It could just be a terminal block with RS-485 contacts. Maybe even a network card with RJ-45 (in this case, ModBus is packed in Ethernet).
Or maybe nothing at all. Just a bare meter. You can install an interface output into it, it is sold separately by the manufacturer and costs money. The main trouble - to install it, you need to open the counter and break the seals. That is, a resource supplying organization is included in this process. She is notified that the seals will be broken, a day is appointed and our engineer, in the presence of a representative of the resource, carries out the necessary improvements, after which the meter is sealed again.
Depending on the installed interface, further refinement is performed. For example, we decided to connect a meter for the wire. This is the easiest option, if our switch is within 100 meters of accessibility, then it is redundant to be wiser with LoRa. Easier cable to our network, in an isolated VLAN.
RS-485/232 needs an Ethernet converter. Many will immediately recall MOX, but it is expensive. For our solutions, we have chosen a cheaper Chinese solution.
If the output is Ethernet immediately, then a converter is not needed.
Question. Suppose we set the interface output ourselves. Can make his life easier and immediately install Ethernet everywhere?
This is not always possible. We must look at the performance of the case. He may not have the right hole for the interface to stand up as it should. And the counter, I recall, is in our basement. Or in the boiler room. There is high humidity, tightness cannot be broken. Finishing the file with a file is a bad idea. Better to put something that initially does not require large alterations. Often - RS-485 is the only way out.
Further. Is the meter connected to guaranteed power? If not, it lives on battery power. In this mode, it is designed for manual polling once a month for three minutes. Constant appeal to VKT-7 will drop its battery. So, you need to pull guaranteed power and install a voltage converter.
For each meter manufacturer, the power module is different. It can be an external unit on a DIN rail or an integrated converter.
It turns out that a set of different interfaces and power modules for each meter should always be stored in our warehouse. The nomenclature there is impressive.
Of course, all this will ultimately be paid by the subscriber. But he will not wait a month until the desired device arrives. And he needs an estimate for connection here and now. So the technological stock falls on our shoulders.
Everything that I described turns into a clear technical connection map so that field engineers do not think what kind of animal they met in the next basement and what they need for its work.
The task list is adjacent to the general connection regulations. After all, it’s not enough to include the counter in our network, you still need to throw the same VLAN on the switch port, you need to carry out diagnostics, do a test poll. We strive to automate the entire process as much as possible in order to avoid mistakes and not to attract the extra forces of engineers.
Well, they wrote technical maps, regulations, automation. Established logistics.
Data is read and poured into the database.
The subscriber from these figures is neither hot nor cold. He needs a report. Preferably in the form in which he is used. Even better, if immediately in the form of a report that he understands, which he can print, put a signature and hand over. This means that you need a simple and intuitive interface that displays information on the meter and can automatically generate a report.
Here our zoo continues. The fact is that there are several report forms. At their core, they reflect the same thing (consumed heat), but in different ways.
Some of the subscribers report in absolute values (that is, values are written in the heat consumption graph, starting with the installation of the meter), some in deltas (this is when we write the consumption over a period of time without reference to the initial values). In fact, they do not use common standards, but established practice. There have been cases when subscribers see all the values that they need (the amount of heat consumed, the amount of coolant supplied and left, the temperature difference), but the columns in the report are not in the same sequence.
Hence the next step - the report should be customizable. That is, the subscriber himself chooses what sequence is coming and what resources are in his document.
Here is an interesting point. All is well if our meter is installed correctly. But it happens that the installation organization, when installing the ITP, screwed up and incorrectly set the time for the meter. We met devices that think - in the yard in 2010. In our system, it will look like zero readings for the current date, and real consumption - if you choose 2010. The deltas are very helpful here. That is, we say that over the past day so much has come.
It would seem, why such difficulties? So hard to let the clock down?
It is precisely with VKT-7 that this will lead to a complete reset of the counter and deletion of archives from it.
The subscriber will be forced to prove to the resource people that he set the ITP not yesterday, but only about five years ago.
And finally, a cherry on the cake.
We have a metering device, we have a report. Between them is our system, which generates this report. Do you believe her?
I am. But how to prove that nothing is changing inside us, that we are not distorting the meaning. This is a certification issue. The survey system must have a certificate that confirms its impartiality. All large systems, such as LERS, I Energetik and others have a similar certificate. We got it, although it is expensive and time consuming.
Of course, you can always cut the corner and buy something ready. But for this you have to pay the developer. And the developer can ask for not only an entrance fee, but also a monthly fee. That is, we will be forced to share part of our pie with him.
This is not the main problem. Developing your own system is also very expensive and much harder. However, it gives an important advantage. We clearly understand how this works. We easily scale it, we can modify it if such a need suddenly arises. The subscriber receives a more complete service, and on our part, one hundred percent control over the process.
That is why we have chosen the second path. We have invested in it a year of the life of our developers and field engineers. But now we clearly understand the work of the whole chain.
Looking back, I understand that without the knowledge gained, I simply could not correctly interpret the abnormal behavior of one or another counter.
In addition, something more can be built on the basis of the dispatch system. Excess consumption alarms, accident report. We are soon preparing to launch a mobile application.
We went even further and added to our platform (or you won’t call it anything else) added the ability to receive calls from residents, the ability to manage our “smart intercoms”, immediately control street lighting and a few more projects that I haven’t written about yet.
All this is complicated, brain-breaking and long. But the result is worth it. Subscribers receive a finished comprehensive product.
Each operator who plans to go to housing and communal services will definitely take this path. Will it pass?
There is a question. It's not even about the money. As I wrote above, here we need a bunch of work in the field and development. Not all major players are used to this. If your developers are in Moscow, and connections are made in Novosibirsk, then your time for the finished product is significantly stretched.
Time will tell who will stay in this market and who will say - well, let him go to hell! But one thing I know for sure - to come and take market share exclusively with money will not work. This process requires unconventional approaches, good engineers, digging in the regulator, communicating with resources and subscribers, constantly identifying and overcoming the rake.
PSIn this article, I deliberately focused on heat and did not mention electricity or water. I also describe cable connection. If we have a pulse output, there are some nuances, such as mandatory reconciliations after installation. It may be that the wire does not reach, then LoRaWAN is used. It is simply unrealistic to describe our entire platform and the stages of its development in one article.
Periodically, another major telecom player tells how soon he will enter this market and crush everyone. Each time with such stories, I think: “guys, good luck!”
You have no idea where you are climbing.
In order for you to understand the scale of the problem, I will briefly describe a small part of our experience in developing the Smart City platform. The part of it that is responsible for scheduling.
General idea and first difficulties
If we are not talking about individual metering devices, but those that are in basements, boiler houses and at enterprises, then most of them are now equipped with a telemetric output. Less often pulsed, more often - RS-485/232 or Ethernet. As a rule, the most "bread" metering devices are those that consider heat. It is they who are ready to pay for their dispatching in the first place.
I have already dwelt in detail in my article on the features of RS-485. In short, it's just a data transfer interface. In essence - the requirements for electrical pulses and communication lines. The description of the packages goes up to the data transfer standard, which runs on top of RS-485. And what kind of standard will be there - it is left to the manufacturer. Often Modbus, but not required. Even if Modbus, it may still be slightly modified.
In fact, for each meter, you need your own survey script, which knows how to "talk" with it and interrogate it. This means that the dispatch system is a set of scripts for each individual counter. A database where all this is stored. And a certain user interface in which he can generate the report he needs.
It looks easy. The devil, as always, is in the details.
Let's start with the first part.
Scripts
How to write them? Well, obviously, buy an meter, pick it up, learn how to communicate with it and integrate it into a common platform.
Unfortunately, this solution will cover only part of our needs. As a rule, a popular counter has several generations, and the script for each generation may differ. Sometimes a little, sometimes significantly. Buying something, you get the latest generation. The subscriber is very likely to be something more ancient. It is no longer sold in stores. And the subscriber will not change the metering node.
Hence the first problem. Writing such scripts is a tough bunch of software developers and engineers "on the ground." We bought the last generation, wrote some initial template and then modified it on real devices. To do this in the laboratory is unrealistic, only in the course of working with live subscribers.
It took us a lot of time to create such a bundle. Now the algorithm is worked out. Initial templates were constantly adjusted and supplemented, depending on what we met in our practice. Of course, the subscriber was warned if suddenly his counter was a little “not like that”. When such a device appears, it is connected according to the standard scheme and the polling script is modified along the way. At the time of integration, the subscriber works for free. He is notified that he still lives in test mode. The integration process itself is a rather unpredictable thing. It happens that you need to make a minimum of corrections. There is a complex process with a field trip, shoveling literature and consecutively overcoming a rake.
The task is not simple, but solvable. The result is a working script. The larger the library of scripts, the easier it is to live.
The second problem.
Technological cards of connection
To make you aware of the complexity of this work, I will give an example. Take the extremely popular VKT-7 heat meter.
The name itself does not mean anything to us. VKT-7 has several iron solutions. What kind of interface is it inside?
There are different options. There may be an output in the standard DB-9 block (this is RS-232). It could just be a terminal block with RS-485 contacts. Maybe even a network card with RJ-45 (in this case, ModBus is packed in Ethernet).
Or maybe nothing at all. Just a bare meter. You can install an interface output into it, it is sold separately by the manufacturer and costs money. The main trouble - to install it, you need to open the counter and break the seals. That is, a resource supplying organization is included in this process. She is notified that the seals will be broken, a day is appointed and our engineer, in the presence of a representative of the resource, carries out the necessary improvements, after which the meter is sealed again.
Depending on the installed interface, further refinement is performed. For example, we decided to connect a meter for the wire. This is the easiest option, if our switch is within 100 meters of accessibility, then it is redundant to be wiser with LoRa. Easier cable to our network, in an isolated VLAN.
RS-485/232 needs an Ethernet converter. Many will immediately recall MOX, but it is expensive. For our solutions, we have chosen a cheaper Chinese solution.
If the output is Ethernet immediately, then a converter is not needed.
Question. Suppose we set the interface output ourselves. Can make his life easier and immediately install Ethernet everywhere?
This is not always possible. We must look at the performance of the case. He may not have the right hole for the interface to stand up as it should. And the counter, I recall, is in our basement. Or in the boiler room. There is high humidity, tightness cannot be broken. Finishing the file with a file is a bad idea. Better to put something that initially does not require large alterations. Often - RS-485 is the only way out.
Further. Is the meter connected to guaranteed power? If not, it lives on battery power. In this mode, it is designed for manual polling once a month for three minutes. Constant appeal to VKT-7 will drop its battery. So, you need to pull guaranteed power and install a voltage converter.
For each meter manufacturer, the power module is different. It can be an external unit on a DIN rail or an integrated converter.
It turns out that a set of different interfaces and power modules for each meter should always be stored in our warehouse. The nomenclature there is impressive.
Of course, all this will ultimately be paid by the subscriber. But he will not wait a month until the desired device arrives. And he needs an estimate for connection here and now. So the technological stock falls on our shoulders.
Everything that I described turns into a clear technical connection map so that field engineers do not think what kind of animal they met in the next basement and what they need for its work.
The task list is adjacent to the general connection regulations. After all, it’s not enough to include the counter in our network, you still need to throw the same VLAN on the switch port, you need to carry out diagnostics, do a test poll. We strive to automate the entire process as much as possible in order to avoid mistakes and not to attract the extra forces of engineers.
Well, they wrote technical maps, regulations, automation. Established logistics.
Where else are the pitfalls lurking?
Data is read and poured into the database.
The subscriber from these figures is neither hot nor cold. He needs a report. Preferably in the form in which he is used. Even better, if immediately in the form of a report that he understands, which he can print, put a signature and hand over. This means that you need a simple and intuitive interface that displays information on the meter and can automatically generate a report.
Here our zoo continues. The fact is that there are several report forms. At their core, they reflect the same thing (consumed heat), but in different ways.
Some of the subscribers report in absolute values (that is, values are written in the heat consumption graph, starting with the installation of the meter), some in deltas (this is when we write the consumption over a period of time without reference to the initial values). In fact, they do not use common standards, but established practice. There have been cases when subscribers see all the values that they need (the amount of heat consumed, the amount of coolant supplied and left, the temperature difference), but the columns in the report are not in the same sequence.
Hence the next step - the report should be customizable. That is, the subscriber himself chooses what sequence is coming and what resources are in his document.
Here is an interesting point. All is well if our meter is installed correctly. But it happens that the installation organization, when installing the ITP, screwed up and incorrectly set the time for the meter. We met devices that think - in the yard in 2010. In our system, it will look like zero readings for the current date, and real consumption - if you choose 2010. The deltas are very helpful here. That is, we say that over the past day so much has come.
It would seem, why such difficulties? So hard to let the clock down?
It is precisely with VKT-7 that this will lead to a complete reset of the counter and deletion of archives from it.
The subscriber will be forced to prove to the resource people that he set the ITP not yesterday, but only about five years ago.
And finally, a cherry on the cake.
Certification
We have a metering device, we have a report. Between them is our system, which generates this report. Do you believe her?
I am. But how to prove that nothing is changing inside us, that we are not distorting the meaning. This is a certification issue. The survey system must have a certificate that confirms its impartiality. All large systems, such as LERS, I Energetik and others have a similar certificate. We got it, although it is expensive and time consuming.
Of course, you can always cut the corner and buy something ready. But for this you have to pay the developer. And the developer can ask for not only an entrance fee, but also a monthly fee. That is, we will be forced to share part of our pie with him.
Why is it all?
This is not the main problem. Developing your own system is also very expensive and much harder. However, it gives an important advantage. We clearly understand how this works. We easily scale it, we can modify it if such a need suddenly arises. The subscriber receives a more complete service, and on our part, one hundred percent control over the process.
That is why we have chosen the second path. We have invested in it a year of the life of our developers and field engineers. But now we clearly understand the work of the whole chain.
Looking back, I understand that without the knowledge gained, I simply could not correctly interpret the abnormal behavior of one or another counter.
In addition, something more can be built on the basis of the dispatch system. Excess consumption alarms, accident report. We are soon preparing to launch a mobile application.
We went even further and added to our platform (or you won’t call it anything else) added the ability to receive calls from residents, the ability to manage our “smart intercoms”, immediately control street lighting and a few more projects that I haven’t written about yet.
All this is complicated, brain-breaking and long. But the result is worth it. Subscribers receive a finished comprehensive product.
Each operator who plans to go to housing and communal services will definitely take this path. Will it pass?
There is a question. It's not even about the money. As I wrote above, here we need a bunch of work in the field and development. Not all major players are used to this. If your developers are in Moscow, and connections are made in Novosibirsk, then your time for the finished product is significantly stretched.
Time will tell who will stay in this market and who will say - well, let him go to hell! But one thing I know for sure - to come and take market share exclusively with money will not work. This process requires unconventional approaches, good engineers, digging in the regulator, communicating with resources and subscribers, constantly identifying and overcoming the rake.
PSIn this article, I deliberately focused on heat and did not mention electricity or water. I also describe cable connection. If we have a pulse output, there are some nuances, such as mandatory reconciliations after installation. It may be that the wire does not reach, then LoRaWAN is used. It is simply unrealistic to describe our entire platform and the stages of its development in one article.