Tank recognition in a video stream using machine learning methods (+2 videos on Elbrus and Baikal platforms) / Smart Engines Blog

In the process of our activities, we daily face the problem of determining development priorities. Given the high dynamics of the development of the IT industry, the ever-increasing demand on the part of business and the state for new technologies, each time, determining the development vector and investing our own resources and resources in the scientific potential of our company, we make sure that all our research and projects are fundamental and interdisciplinary.

Therefore, developing our main technology - the HIEROGLYPH data recognition framework, we take care both of improving the quality of document recognition (our main business line) and of the possibility of using the technology to solve related recognition problems. In today's article, we will tell how, based on our recognition engine (documents), we made the recognition of larger, strategically important objects in the video stream.

Formulation of the problem

Using existing experience, to build a tank recognition system that allows to classify an object, as well as determine basic geometric parameters (orientation and distance) in poorly controlled conditions without the use of specialized equipment.

Decision

As the main algorithm for solving the problem, we chose the approach of statistical machine learning. But one of the key problems of machine learning is the need for a sufficient amount of learning data. Obviously, in-kind images obtained from real scenes containing the objects we need are inaccessible to us. Therefore, it was decided to resort to generating the necessary data for training, the benefit of experience in this place is great. And yet, it seemed unnatural to completely synthesize the data for this task, so a special layout was prepared for modeling real scenes. Various objects modeling the countryside are installed on the layout: a characteristic landscape cover, bushes, trees, barriers, etc. Images were captured using a digital small format camera. In the process of capturing images, the background of the scene changed significantly to ensure greater stability of the algorithms to changes in the background.

Four battle tank models were used as targets: T-90 (Russia), M1A2 Abrams (USA), T-14 (Russia), Merkava III (Israel). The objects were located at different positions of the polygon, thereby expanding the list of acceptable visible angles of the object. A significant role was played by engineering barriers, trees, bushes and other landscape elements.

Thus, in a couple of days we have assembled a sufficient set for training and subsequent evaluation of the quality of the algorithm (several tens of thousands of images).

They decided to divide the recognition directly into two parts: object localization and object classification. Localization was carried out using the trained classifier Viola and Jones (after all, a tank is a normal rigid object, no worse than a face, therefore the “blind with details” Viola and Jones method quickly localizes the target object). But we entrusted the classification and definition of the angle with the convolutional neural network - in this task it is important for us that the detector successfully identifies those features that, say, distinguish the T-90 from Merkava. As a result, it was possible to build an effective composition of algorithms that successfully solves the problem of localization and classification of objects of the same type.

Next, we launched the resulting program on all the platforms we have (Intel, ARM, Elbrus, Baikal, COMDIV), optimized computationally difficult algorithms to improve performance (we have already written about this in our articles, for example, here https: // habr .com / ru / company / smartengines / blog / 438948 / or https://habr.com/en/company/smartengines/blog/351134/ ) and have achieved stable operation of the program on the device in real time.

As a result of all the described actions, we have obtained a full-fledged software product that has significant tactical and technical characteristics.

Smart tank reader

So, we present to you our new development - a program for recognizing tank images in the Smart Tank Reader video stream , which:

• It solves the “friend or foe” problem for a given set of objects in real time;
• Defines geometric indicators (distance to the object, preferred orientation of the object);
• It works in uncontrolled weather conditions, as well as in the case of partial overlapping of the object by foreign objects;
• Fully autonomous operation on the target device, including in the absence of radio communications;
• List of supported processor architectures: Elbrus, Baikal, COMDIV, as well as x86, x86_64, ARM;
• The list of supported operating systems: Elbrus OS, AstraLinux OS, Atlix OS, as well as MS Windows, macOS, various Linux distributions supporting gcc 4.8, Android, iOS;
• Completely domestic development.

Usually, in conclusion to our articles on Habré, we give a link to the marketplace, where everyone who wants using their mobile phone can download a demo version of the application in order to actually evaluate the performance of the technology. This time, taking into account the specifics of the resulting application, we wish all our readers never to face the problem of quickly determining whether a tank belongs to a certain side.