Lisa Alert: life-saving volunteers

The search and rescue squad " Liza Alert " has been around for 8 years. This is a volunteer association, a community of caring people who are looking for missing people, effectively interacting with the Ministry of Emergency Situations and the Ministry of Internal Affairs. Lisa Alert collects applications for missing people, conducts various training events, organizing searches and the search operations themselves. The detachment does not conduct any commercial activity, does not have a settlement account and does not accept monetary donations.

Recently, Mail.Ru Cloud Solutions provided Lise Alert for free with a platform for hosting services that are actively used in search and rescue operations. We decided to talk with Sergey Chumak, senior IT director at Lisa Alert, about how volunteer rescuers work and how high technologies help them.

How people are searched

- Take a typical situation - a man went into the woods and disappeared. It is autumn now, which means grandparents are actively going for mushrooms. And there are lost. Relatives begin to look for them, call the hotline "Liza Alert".

- By launching the search, we create a separate topic on the detachment's forum, which indicates the main signs of the missing person (“orientation”) and information about the time and place for gathering volunteers who are ready to take part.

While the departure is being prepared, the cartographers promptly make a set of maps of the area — these maps with gridded squares will be loaded into the navigators issued by the search groups.

At the entrance to the forest headquarters unfolds, the radio tower rises. Arriving volunteers are divided into small groups ("foxes"). Each group search coordinator defines the task. When the “foxes” come back, the tracks of their movement from the navigators are copied to the staff laptop and the coordinator analyzes the incoming information. For example, the group transmits some landmarks missing on the map.

All information collected during the search is saved. If in these parts someone is lost again, the prepared sets of maps will be immediately available to the search engines, as well as data on the terrain.

Cards

Without detailed maps it is extremely difficult to search for a person in the forest. We use satellite images, topographic maps, including Soviet, as well as modern, like OpenStreetMaps.

Cartographic search kits are made in two formats: for Garmin navigators, with which groups go to the forest, and for the operation of the operational headquarters. Such cards are large and require processing, they need to be stored somewhere. Previously, this used the old, low-powered private server of one of the volunteer participants. But the number of active searches is increasing every year, the activity of the detachment is geographically expanding - new regional offices are being created, so there is an urgent need for a qualitatively and quantitatively superior resource. Then we turned for help to the cloud platform from Mail.Ru Group.

The company provided “Liza Alert” with cloud resources as part of the Cloud Servers project , and we deployed the main and backup servers to which our cartographic repository has moved since April 2018. This is a very significant support, because now we have much more space available for current activities. In addition, the speed of Cloud Servers is much higher, not to mention the convenience of administering and working with the storage. With the help of a special tool, those responsible for the storage themselves regulate the rights within the group: each new cartographer quickly gains access to the server.

In addition to replication, we now regularly create backups for which Mail.Ru Group has allocated object storage.. Before they could not do, and if someone deleted something, the data disappeared forever.

For all our products that are already helping in the search, and for the ideas of the products that our volunteers will have to develop, we have been allocated Cloud Servers and Cloud Storage services. These resources are enough to perform capacious operations.

Heavenly eye

One of our active search tools is quadrocopters. They automatically fly along pre-loaded routes over a person’s missing area and photograph the terrain.

What is it done for? A lost person may go to an open place - to a clearing, a clearing, into a field. Or to be in the windfall, that is, in a part of the forest cluttered with felled trees, which is clearly visible from the air. The flying quadcopter will "notice" the lost one, thereby significantly saving the resources of hiking groups.

Until recently, quadcopter pilots had to independently review all the pictures taken. Hell of a job: with each departure they get about 1000, and you need to check carefully - to peer, to increase incomprehensible fragments. According to the experience, it is impossible to view more than one hundred images at a time: then you disconnect from the details, the attention is dissipated, the eye is “blurred”. Therefore, we have created a page for collective viewing of photos.

Crowdsourcing to the rescue

Now a group of volunteers is studying with us, which will deal with photogrammetry “professionally”. The fact is that objects familiar to everyone - stump, pillar, garbage bag - sometimes look unexpected or incomprehensible from a height. And we need some experience to immediately determine what you see.

In the viewing system developed for “observers”, the frames are issued in such a way that in minimum time the group can check in total as many pictures as possible. At first, each user watches his selection, then they begin to look at each other’s pictures, and if there is time, they review what they have already seen. Cross and repeated scans reduce the likelihood of missing important details.

Although to recognize the person lying down can often and completely untrained spectator. That is why we have an open service at the same time, a kind of crowdsourcing tool, where everyone who wants to view the results of aerial photography can help the detachment - watcher.lizaalert.ru.

When a message appears on the forum and in the VKontakte group stating that pictures from the next departure have been uploaded, up to 150 people simultaneously begin viewing them.

Here the pictures are strictly depersonalized. And for the convenience of research, each photo is divided into 32 small fragments, which are placed on the screen without scaling. Network users need to answer “yes” or “no” to just one question - is there a person in the picture?



The system remembers how many times for each fragment the participants answered positively or negatively, calculates the average score and ranks the fragments by it, and reports the results to the observers of the professional group - they check the most “promising” pictures in detail.





The team of trained detachment photogrammetrists, while viewing, has detailed information about the missing person: how he looks, what he was wearing, what he had with him. Therefore, not only abstract people in the picture are looked at here, but also artifacts - a person could have thrown or lost something from his things (basket, shoes, headdress). Notifications of such findings immediately come to the search coordinator, who decides whether to send one of the search teams for review. Having found a characteristic item, we can assume the route of movement of the one we are looking for.

Now the maximum viewing speed on the site is up to 10 shots per second. As a result, it is possible to process the entire footage of a quadrocopter quite quickly - in some hours. While search groups are walking in the forest, observers from computers or mobile devices “explore” open areas of the terrain.

- Surely you had the idea to try to solve the problem of viewing images using neural networks?

- Yes, they considered this possibility, although the opinions of experts about the applicability of machine learning in this case differ. In any case, to train a neural network, we need to collect a data set - a large set of pictures, pictures from the drone, for which there are people and for which there are no people. The larger the set will be able to assemble, the more stable the model should be. In any case, experiments with the neural network and the creation of a training sample will begin no earlier than winter - now the search season is active, the squad members simply “exhaust themselves” on constant trips. But we will definitely return to this. I hope that the neural network will at least help many times to speed up the "sifting out" of images, which are guaranteed there are no people, and prioritizes the photos that need to be viewed first.

From plane to volume

Finally, our last idea - the generation of three-dimensional maps based on photographs. To this end, photographs should be taken from a greater height and overlap each other more. The IT-direction of Liza Alert already has experience in creating 3D-cards in expensive paid software. But there is also an open source OpenDroneMap application, and a web version is also available. If the results of their work are comparable, we will begin to actively use OpenDroneMap in searches.

Three-dimensional map gives a more accurate picture of the area in the search area. Often, those who are lost go to the forest with a telephone and stay in touch for a while. While the battery is not sitting down, you can find out from the person details, ask about what he sees around. For example, he is sitting on the edge of a felling or has fallen into a “natural trap” - a windbreak, windfall. Such objects are usually not marked on ordinary maps, they are difficult to recognize in single shots. But they "manifest" on a three-dimensional map, and you can immediately send search groups to check.



To shoot with the desired overlapping area, for example, 2 x 0.7 km from the quadcopter will take 30-40 minutes. It will take too long to generate a 3D map from a personal computer from these images, but on a cloud server in a powerful configuration, the process will take a reasonable time for responsiveness. That is, in the first hours of the search, it will be possible to get very important information about the area and adjust the tasks for the search teams. Of course, this information is important on the second day, but the sooner we get it, the more chances to quickly find and save the lost.