Why do we need in the production of AR and VR
Hello! AR and VR are fashionable, now only the lazy one (or someone who just doesn’t need it) has made applications using them. From Oculus to MSQRD, from simple toys that delight children with the appearance of a dinosaur in the room, to IKEA and more, such as “Set up furniture in your dvushka” applications. There are many applications.
And there is also not the most popular compared to them, but actually a useful area - teaching a person new skills and simplifying his daily work. Here, for example, we can cite simulators for doctors, pilots, and even law enforcement agencies. We in SIBUR use these technologies as part of the digitalization of production. The main consumer is a direct production employee in gloves and a helmet, which is located at the enterprise, at high-risk facilities.
My name is Alexander Leus, I am Product Owner of Industry 4.0, and I will talk about what features arise here.
In general, in neighboring Europe it is customary to consider industry 4.0 everything that is connected with digital in an enterprise in the general sense. We have 4.0 - these are digital products that are somehow related to hardware. First of all, of course, this is the industrial Internet of things, IIoT, plus the direction associated with video analytics (the factory has a huge number of cameras, and the images from them need to be analyzed), and also a direction called XR (AR + VR).
The main objective of IIoT is to increase the level of automation in production, reduce the influence of the human factor on the process of managing non-critical technological processes, and reduce the cost of operating plants.
Video analytics in SIBUR consists of two main parts - technological observation and situational analytics. Technological observation allows you to control the production parameters themselves (as we wrote here about the extruder , for example, or the quality control of rubber briquettes according to the image of its crumbs). And situational, as the name implies, tracks the onset of certain events: one of the employees was in the area where he should not be (or where there should not be anyone at all), steam jets and the like suddenly began to burst from the pipe.
But why do we need XR.
The term was coined at the end of the year before last at the Khronos Group consortium engaged in creating standards for working with graphics. The letter “X” itself is not deciphered in any way, the thing is this:
XR includes everything that is somehow connected with interactive computer graphics, CGI, AR + VR directions, as well as the technology stack accompanying all this stuff. In our work, XR allows us to solve a number of important problems.
First, we give a person a new tool that simplifies his life (at least during working hours). We offer a whole platform based on video technologies and AR, which allows you to directly connect a production employee (operator) at the plant and a remote expert - the first one walks around the enterprise in AR glasses, broadcasting everything that happens on video (not much different from a tourist walk with GoPro, except for the surroundings ), the second on his monitor sees what is happening on behalf of the operator and can display the first prompts necessary. For example, in what order to disassemble the unit, what parameters to set and the like.
Secondly, we pump the skills of employees. There is generally a story about the constant updating of knowledge. For example, a new employee comes to us, and at the beginning of his work, his qualification has some definite significance, if he is after a technical school, he will remember almost everything that he was taught. At least it should be. After working for several years, he can either improve his skills or lose his skills a little, it all depends on what he was doing, because even a lot of useful knowledge can be pushed into a far corner by a daily routine.
For example, during its shift, an event occurs, an unplanned emergency stop. And here it is important, what is the employee’s knowledge base at this moment, will he be able to handle all the tasks necessary in an emergency right now or not. It is one thing if you work with scheduled repairs on average once every 3 years, here you can refresh your knowledge yourself (or with our help) a couple of months before the planned work, another thing is such a production surprise. But you didn’t finish the tea and you have a qualification at a lower level than is required right now.
In such cases, our AR platform also helps - we give it to an employee, and it turns out that when paired with a remote specialist they are already faster in making the right decisions on the go.
Another area of application of XR is simulators and simulators, which allow you to work out the correct reaction to possible situations in the workplace. Now we have a control simulator for working with compressors, we will soon launch another one for working with hazardous reagents.
In addition to simulators, we create detailed virtual tips. For example, our operational personnel in tasks have switching switchboards, when electricity must be supplied to different sections. A classic approach to creating such instructions is photo-instruction or applications with interactive 360-degree panoramas. And with the help of glasses, wearable video cameras and materials developed by us, we will be able to form a detailed knowledge base on maintenance and repair technologies.
By the way, such a base itself is already a full-fledged digital product of wide coverage, on the basis of which new simulators can be built, plus this knowledge can be transported through the platform, helping local people make operational decisions. The guys are already building a data lake, which can be read about here .
The AR platform here is used as an interface for visualizing tips - for example, a more experienced colleague (or AI) can tell you that you need to do more temperature in that area. That is, you just need to go to the compressor - and the glasses will appear advice.
If simplified, then the AR platform consists of a media resource with a database and a media server, to which specialists in AR glasses can connect, performing certain actions at the factory. And specialists can already connect to them from their computers, it can be both our internal experts and external experts - vendors and equipment suppliers. The process looks something like this: an employee at the plant performs a certain operation, and to make a decision, he needs information, or is being supervised by installation or commissioning. The picture from the employee’s glasses is transmitted to the specialists on the monitors, they can send him “hints” from their computers, both in the text, simply sending tips to the glasses interface and in graphics - the employee sends a photo from the glasses,
And to make it even easier, there is the possibility of creating automatic access to the database so that the employee by the label on the device’s body will immediately receive information about him and the necessary actions.
It is one thing to come up with all this and even implement it on hardware under ordinary conditions. Well, seriously, what’s complicated, I deployed the environment, connected the AR-glasses to the laptop, everything works and everything is cool.
And then you come to the factory.
By the way, many similar stories about “We have excellent industrial products” quickly end when the product falls into actual industrial conditions. We have a lot of restrictions here. Wireless network for data transfer is not secure = no wireless network. There is a wired connection through which the Internet is connected.
But (you already understood, yes?) The Internet is also unsafe = a proxy is used for protection, and most of the ports are closed.
Therefore, it’s not enough to come up with a cool solution for the industry that will help users, you should immediately think about how to push all this into the industry in the conditions of the restrictions existing in it. And the situation now is such that such an approach has not yet been implemented within the industry.
We cannot just make a server with everything necessary for the platform to work, leave it at the factory and leave with our heads held high - no one will connect to this server. Putting a dedicated laptop next to it also makes no sense, it spoils the whole idea - we do all this in order to be able to connect with each other both the site employee in Nizhnevartovsk and the person from the Pyt-Yakh factory (and we have a factory there, yes), and the German by the vendor. And so that they can together normally discuss the repair of the pump or compressor, each from their workplace (or on their own when the employee is on site). And you don’t have to fly anywhere, coordinate business trips, get visas, waste time and money.
I connected - I saw everything - I decided everything, well, or suggested a solution and went / flew to help.
Another specificity that sets an additional framework is our work with gas. And it is always a question of explosion protection and the requirements of specific rooms. When creating a device, you should always ask yourself the question - who will use it at all and in which particular conditions? Someone works with us in the repair shop, where they carry out maintenance and repairs, someone directly at the factory, someone in the server room, someone at the substation.
Ideally, for each of the tasks and each use case, you need to create your own device.
There are no problems with the availability of XR AR glasses. There are problems with their application in the industry. Take the same Google Glass when they were tested in 2014, it turned out that they work for 20 minutes on a single charge, and in the process they work well to warm the face. It is, of course, good when the site in Tobolsk is -40, and you have something warm on your face. But still not that.
One Japanese company came closer, it already had industrial designs for implementation at power facilities in the same 2014. In principle, the venture with AR equipment on the market has long and by and large little changed. For example, helmets for pilots - now everything is almost the same, just the systems have become smaller, the power lasts longer, and the resolution of microdisplays and video cameras has improved significantly.
Here it must also be taken into account that such devices are made monocular and binocular. And that makes sense. If in your work you need to read some information, consider documents and the like, then you need a binocular device to form a picture for two eyes at once. If you just need to transfer the video stream and photos, while receiving information in the format of short tips and parameters, then the capabilities of the monocular device are enough.
The monoculars even have an explosion-proof sample, RealWear HMT-1z1, manufactured at the iSafe German plant, but this is generally the only sample from serial products. Solid monocular device with explosion protection and a small monocular screen. But sometimes binoculars are also needed. For example, the energy sector, which is involved in operational switching, needs a larger screen to see the entire switching scheme. Still important are the standard characteristics of the camcorder in terms of the quality of shooting and its convenience - so that nothing obscures the viewing angle, that there is normal autofocus (twist something small with gloves or look at small chips on details in a weak minus, catching focus, it’s like that pleasure).
But for employees of repair shops, everything is a little easier, there are different requirements for explosion safety, which allows you to choose devices from a wider model range. Here, the main thing is simply quality - that the device works, does not slow down, it is made soundly in industrial design, so that it does not break under mechanical stress and so on. In general, a normal serial piece of iron, not a prototype.
And one more thing, without thinking over which you can’t drag a solution into the industrial world - infrastructure. There is such a thing as digital ready infrastructure. On the one hand, this is the same marketing hype as the Windows 7 ready mouse for the computer. On the other hand, there is a rather important meaning. You will not use a mobile phone in the absence of a base station in range? Well, OK, you can use it, read a book, see photos and so on, but there’s no longer a call.
All digital products fall on the infrastructure. If it is not, there is no working digital product. And if very often digitalization is understood simply as transferring everything from paper to digital, for example, a person had a paper pass in a company - they made a digital pass, and so on, then we have all this business based on tasks, on what exactly needs to be done.
Let's say there is a simple wish - the infrastructure for communications. And the factory area is about 600 football fields. Is it worth it to do the infrastructure? If so, in which particular areas, squares? The plots are all different, for each you need to write TK. Well and most importantly - but do the people who work here really need this infrastructure?
Digital products in production are always a phased work, and the thing is that you do not understand how and what to do with the infrastructure until you drag the product itself. You brought a product, but there is no infrastructure. I deployed wireless networks on crutches from available operators, I realized that working somehow works, but I want stability - and roll back, as in the good old Soviet system approach to design. And you begin to build the infrastructure that was not here and in the form in which users need it.
Somewhere it’s enough to put a couple of access points, somewhere there is an installation with a bunch of stairs and passages the height of a 20-story house, and here at least hang with dots and transmitters, but you won’t get such a network quality as in a room, so it makes sense to flash install and use portable access points such as miners use (explosion proof!). Each object has its own specifics, requiring its decision.
Having created the infrastructure, dragged into the industry the necessary devices and adjusted everything from a technical point of view, remember - there are still people with whom you must go through three stages in order to use the product.
In fact, you give people something that they definitely have not used before. Now, if you transplanted relatives from button clamshells to modern smartphones, the story is about the same. Show the device where the video camera is, and how to set up a microdisplay for yourself, and where to click something for communication — and so on, so on, so on.
And there is one ambush.
You come to people and bring a product, talk about it. Employees can agree, not really argue, with interest and enthusiasm to learn from you to use this new device. They can even remember everything quickly the first time. They can pass the test on the knowledge of the device brilliantly and use it as confidently as you.
And then it turns out that you didn’t specify in advance which one of their team would go with these glasses directly to the site. And it turns out that you need to re-teach completely different people.
But you will have several employees who are well versed in a product that will not be used.
And we also have a short video on how this works.
And there is also not the most popular compared to them, but actually a useful area - teaching a person new skills and simplifying his daily work. Here, for example, we can cite simulators for doctors, pilots, and even law enforcement agencies. We in SIBUR use these technologies as part of the digitalization of production. The main consumer is a direct production employee in gloves and a helmet, which is located at the enterprise, at high-risk facilities.
My name is Alexander Leus, I am Product Owner of Industry 4.0, and I will talk about what features arise here.
Industry 4.0
In general, in neighboring Europe it is customary to consider industry 4.0 everything that is connected with digital in an enterprise in the general sense. We have 4.0 - these are digital products that are somehow related to hardware. First of all, of course, this is the industrial Internet of things, IIoT, plus the direction associated with video analytics (the factory has a huge number of cameras, and the images from them need to be analyzed), and also a direction called XR (AR + VR).
The main objective of IIoT is to increase the level of automation in production, reduce the influence of the human factor on the process of managing non-critical technological processes, and reduce the cost of operating plants.
Video analytics in SIBUR consists of two main parts - technological observation and situational analytics. Technological observation allows you to control the production parameters themselves (as we wrote here about the extruder , for example, or the quality control of rubber briquettes according to the image of its crumbs). And situational, as the name implies, tracks the onset of certain events: one of the employees was in the area where he should not be (or where there should not be anyone at all), steam jets and the like suddenly began to burst from the pipe.
But why do we need XR.
The term was coined at the end of the year before last at the Khronos Group consortium engaged in creating standards for working with graphics. The letter “X” itself is not deciphered in any way, the thing is this:
XR includes everything that is somehow connected with interactive computer graphics, CGI, AR + VR directions, as well as the technology stack accompanying all this stuff. In our work, XR allows us to solve a number of important problems.
First, we give a person a new tool that simplifies his life (at least during working hours). We offer a whole platform based on video technologies and AR, which allows you to directly connect a production employee (operator) at the plant and a remote expert - the first one walks around the enterprise in AR glasses, broadcasting everything that happens on video (not much different from a tourist walk with GoPro, except for the surroundings ), the second on his monitor sees what is happening on behalf of the operator and can display the first prompts necessary. For example, in what order to disassemble the unit, what parameters to set and the like.
Secondly, we pump the skills of employees. There is generally a story about the constant updating of knowledge. For example, a new employee comes to us, and at the beginning of his work, his qualification has some definite significance, if he is after a technical school, he will remember almost everything that he was taught. At least it should be. After working for several years, he can either improve his skills or lose his skills a little, it all depends on what he was doing, because even a lot of useful knowledge can be pushed into a far corner by a daily routine.
For example, during its shift, an event occurs, an unplanned emergency stop. And here it is important, what is the employee’s knowledge base at this moment, will he be able to handle all the tasks necessary in an emergency right now or not. It is one thing if you work with scheduled repairs on average once every 3 years, here you can refresh your knowledge yourself (or with our help) a couple of months before the planned work, another thing is such a production surprise. But you didn’t finish the tea and you have a qualification at a lower level than is required right now.
In such cases, our AR platform also helps - we give it to an employee, and it turns out that when paired with a remote specialist they are already faster in making the right decisions on the go.
Another area of application of XR is simulators and simulators, which allow you to work out the correct reaction to possible situations in the workplace. Now we have a control simulator for working with compressors, we will soon launch another one for working with hazardous reagents.
In addition to simulators, we create detailed virtual tips. For example, our operational personnel in tasks have switching switchboards, when electricity must be supplied to different sections. A classic approach to creating such instructions is photo-instruction or applications with interactive 360-degree panoramas. And with the help of glasses, wearable video cameras and materials developed by us, we will be able to form a detailed knowledge base on maintenance and repair technologies.
By the way, such a base itself is already a full-fledged digital product of wide coverage, on the basis of which new simulators can be built, plus this knowledge can be transported through the platform, helping local people make operational decisions. The guys are already building a data lake, which can be read about here .
The AR platform here is used as an interface for visualizing tips - for example, a more experienced colleague (or AI) can tell you that you need to do more temperature in that area. That is, you just need to go to the compressor - and the glasses will appear advice.
If simplified, then the AR platform consists of a media resource with a database and a media server, to which specialists in AR glasses can connect, performing certain actions at the factory. And specialists can already connect to them from their computers, it can be both our internal experts and external experts - vendors and equipment suppliers. The process looks something like this: an employee at the plant performs a certain operation, and to make a decision, he needs information, or is being supervised by installation or commissioning. The picture from the employee’s glasses is transmitted to the specialists on the monitors, they can send him “hints” from their computers, both in the text, simply sending tips to the glasses interface and in graphics - the employee sends a photo from the glasses,
And to make it even easier, there is the possibility of creating automatic access to the database so that the employee by the label on the device’s body will immediately receive information about him and the necessary actions.
Implementation and Barriers
It is one thing to come up with all this and even implement it on hardware under ordinary conditions. Well, seriously, what’s complicated, I deployed the environment, connected the AR-glasses to the laptop, everything works and everything is cool.
And then you come to the factory.
By the way, many similar stories about “We have excellent industrial products” quickly end when the product falls into actual industrial conditions. We have a lot of restrictions here. Wireless network for data transfer is not secure = no wireless network. There is a wired connection through which the Internet is connected.
But (you already understood, yes?) The Internet is also unsafe = a proxy is used for protection, and most of the ports are closed.
Therefore, it’s not enough to come up with a cool solution for the industry that will help users, you should immediately think about how to push all this into the industry in the conditions of the restrictions existing in it. And the situation now is such that such an approach has not yet been implemented within the industry.
We cannot just make a server with everything necessary for the platform to work, leave it at the factory and leave with our heads held high - no one will connect to this server. Putting a dedicated laptop next to it also makes no sense, it spoils the whole idea - we do all this in order to be able to connect with each other both the site employee in Nizhnevartovsk and the person from the Pyt-Yakh factory (and we have a factory there, yes), and the German by the vendor. And so that they can together normally discuss the repair of the pump or compressor, each from their workplace (or on their own when the employee is on site). And you don’t have to fly anywhere, coordinate business trips, get visas, waste time and money.
I connected - I saw everything - I decided everything, well, or suggested a solution and went / flew to help.
Another specificity that sets an additional framework is our work with gas. And it is always a question of explosion protection and the requirements of specific rooms. When creating a device, you should always ask yourself the question - who will use it at all and in which particular conditions? Someone works with us in the repair shop, where they carry out maintenance and repairs, someone directly at the factory, someone in the server room, someone at the substation.
Ideally, for each of the tasks and each use case, you need to create your own device.
There are no problems with the availability of XR AR glasses. There are problems with their application in the industry. Take the same Google Glass when they were tested in 2014, it turned out that they work for 20 minutes on a single charge, and in the process they work well to warm the face. It is, of course, good when the site in Tobolsk is -40, and you have something warm on your face. But still not that.
One Japanese company came closer, it already had industrial designs for implementation at power facilities in the same 2014. In principle, the venture with AR equipment on the market has long and by and large little changed. For example, helmets for pilots - now everything is almost the same, just the systems have become smaller, the power lasts longer, and the resolution of microdisplays and video cameras has improved significantly.
Here it must also be taken into account that such devices are made monocular and binocular. And that makes sense. If in your work you need to read some information, consider documents and the like, then you need a binocular device to form a picture for two eyes at once. If you just need to transfer the video stream and photos, while receiving information in the format of short tips and parameters, then the capabilities of the monocular device are enough.
The monoculars even have an explosion-proof sample, RealWear HMT-1z1, manufactured at the iSafe German plant, but this is generally the only sample from serial products. Solid monocular device with explosion protection and a small monocular screen. But sometimes binoculars are also needed. For example, the energy sector, which is involved in operational switching, needs a larger screen to see the entire switching scheme. Still important are the standard characteristics of the camcorder in terms of the quality of shooting and its convenience - so that nothing obscures the viewing angle, that there is normal autofocus (twist something small with gloves or look at small chips on details in a weak minus, catching focus, it’s like that pleasure).
But for employees of repair shops, everything is a little easier, there are different requirements for explosion safety, which allows you to choose devices from a wider model range. Here, the main thing is simply quality - that the device works, does not slow down, it is made soundly in industrial design, so that it does not break under mechanical stress and so on. In general, a normal serial piece of iron, not a prototype.
Infrastructure
And one more thing, without thinking over which you can’t drag a solution into the industrial world - infrastructure. There is such a thing as digital ready infrastructure. On the one hand, this is the same marketing hype as the Windows 7 ready mouse for the computer. On the other hand, there is a rather important meaning. You will not use a mobile phone in the absence of a base station in range? Well, OK, you can use it, read a book, see photos and so on, but there’s no longer a call.
All digital products fall on the infrastructure. If it is not, there is no working digital product. And if very often digitalization is understood simply as transferring everything from paper to digital, for example, a person had a paper pass in a company - they made a digital pass, and so on, then we have all this business based on tasks, on what exactly needs to be done.
Let's say there is a simple wish - the infrastructure for communications. And the factory area is about 600 football fields. Is it worth it to do the infrastructure? If so, in which particular areas, squares? The plots are all different, for each you need to write TK. Well and most importantly - but do the people who work here really need this infrastructure?
Digital products in production are always a phased work, and the thing is that you do not understand how and what to do with the infrastructure until you drag the product itself. You brought a product, but there is no infrastructure. I deployed wireless networks on crutches from available operators, I realized that working somehow works, but I want stability - and roll back, as in the good old Soviet system approach to design. And you begin to build the infrastructure that was not here and in the form in which users need it.
Somewhere it’s enough to put a couple of access points, somewhere there is an installation with a bunch of stairs and passages the height of a 20-story house, and here at least hang with dots and transmitters, but you won’t get such a network quality as in a room, so it makes sense to flash install and use portable access points such as miners use (explosion proof!). Each object has its own specifics, requiring its decision.
People
Having created the infrastructure, dragged into the industry the necessary devices and adjusted everything from a technical point of view, remember - there are still people with whom you must go through three stages in order to use the product.
- Introduce in detail, show your own example.
- To teach to use independently, to test after that, as far as everyone understood.
- Ensure survival of the product.
In fact, you give people something that they definitely have not used before. Now, if you transplanted relatives from button clamshells to modern smartphones, the story is about the same. Show the device where the video camera is, and how to set up a microdisplay for yourself, and where to click something for communication — and so on, so on, so on.
And there is one ambush.
You come to people and bring a product, talk about it. Employees can agree, not really argue, with interest and enthusiasm to learn from you to use this new device. They can even remember everything quickly the first time. They can pass the test on the knowledge of the device brilliantly and use it as confidently as you.
And then it turns out that you didn’t specify in advance which one of their team would go with these glasses directly to the site. And it turns out that you need to re-teach completely different people.
But you will have several employees who are well versed in a product that will not be used.
And we also have a short video on how this works.