June 26, 2019 at 18:16Engineers save people lost in the forest, but the forest has not yet surrendered
Every year, rescuers look for tens of thousands of people lost in the wild. From cities, our technological power seems so huge that it can handle any task. It seems, take a dozen drones, hang on each one on the camera and the thermal imager, screw on the neural network and everything will find anyone in 15 minutes. But this is not at all true.
Until now, technology rests against a lot of restrictions, and rescue teams comb through vast areas with hundreds of volunteers.
Last year, the Sistema charity foundation launched the Odyssey project to find new technologies for finding people. Hundreds of engineers and designers took part in it. But even savvy technically and experienced people sometimes did not suspect how impenetrable the forest was for technology.
In 2013, in the village of Sinsk in Yakutia, two little girls disappeared - Alina Ivanova and Ayana Vinokurova. Enormous forces were thrown into their searches: equipped a hundred volunteers, rescue teams, divers, drones with thermal imagers. Helicopter shots were shared so that everything could be viewed on the Internet. But there wasn’t enough strength. What happened to the girls is still unknown.
Yakutia is huge. If it were a state, it would have entered the top ten largest in terms of area. But in a gigantic territory less than a million people live. In such an endless deserted taiga, Nikolai Nakhodkin worked for 12 years in the Rescue Service of Yakutia, 9 of which - as a leader. When the conditions are worse, and there are few resources, one has to invent new ways of finding people. And as Nikolai says, ideas do not come from a good life.
Nikolai Nakhodkin
Since 2010, the Rescue Service of Yakutia has been using drones. This is a separate organization from the Ministry of Emergencies of the Russian Federation, financed by the republic itself. There are no such strict regulations for equipment, so in the Ministry of Emergency Situations drones began to be used much later. Also at the service there is a scientific group where enthusiastic engineers are engaged in the development of applied technologies for lifeguards.
“The existing search methods that the Ministry of Emergencies has, the rescue services, in all kinds of power structures - have not changed since the 30s. A tracker follows the trail, the dog helps not to stray, ”says Alexander Aitov, who was the head of the scientific group. “If a person is not found, an entire village rises in Yakutia, two, three. All come together and comb the forests. To search for a living person every hour is important, and time is running out quickly and quickly. Its always small. When the tragedy occurred in Sinsk, a lot of people and equipment were involved, but without result. Similar situations are found in deserted taiga. To somehow fix this, the idea came up not to perceive the missing as a passive link, but to use his own craving to be saved and an active thirst for life. ”
Rescue engineers decided to assemble rescue light and sound beacons - rather large, but lightweight devices that make a loud sound and glow for a long time, attracting attention to themselves day and night. A lost person, coming to them, will find water, biscuits and matches - and at the same time instructions to sit still and wait for the rescuers.
Such lighthouses stand at a distance of three kilometers from each other and encircle an approximate search area for the missing person. They make a low sound, as if a car rumbles - because high frequencies are much worse distributed in the forest. Often, those rescued thought that they were taking the sound of the road or a group of tourists about to leave.
The lighthouses were incredibly simple. This is not the first time that a scientific team has implemented elementary, but ingenious solutions.
“There, for example, they developed a floating suit for lifeguards. Pants and a jacket look like ordinary workwear, but in water they keep a person afloat. Speaking quite utilitarian, the suit is two-layer. Polyurethane foam granules are sewn inside. There is a development for diving a diver at low temperature. With the expansion of compressed air in the cold, the valves are covered with hoarfrost, and a person suffocates. Several institutes could not think of what to do with it - they developed special materials, did electrical heating, and introduced all sorts of modern approaches.
Our guys solved the problem for 500 rubles. They passed the cold air that comes from the tank (and goes to -57 under the water) through a coil, passed through a Chinese thermos. The air heats up, people go under water and can work there. ”
But the beacons are too simple, they lacked many useful functions. During search operations, the rescuer regularly had to travel huge distances to check every lighthouse. If there are ten lighthouses, it means the rescuer must go 30 km through the taiga every 3-4 hours.
In 2018, the Sistema charity foundation launched the Odyssey project, a competition for teams that, using new technologies, will try to find the latest ways to save missing people in the wild. Nikolai Nakhodkin and Alexander Aitov with friends decided to take part - they called the team “Nakhodka” and brought their simplest device to improve it in competition with others.
According to the Ministry of Internal Affairs, in 2017 almost 84 thousand people disappeared in Russia, and half of them were not found. Each missing person on average was searched for by a hundred people. Therefore, the mission of the Odyssey competition was to “create technologies that will help find missing people in the forest without a source of communication. It can be instruments, sensors, drones, new means of communication and everything your imagination is capable of. ”
“Of the non-obvious solutions — or fantasy ones — we can name an airship equipped with a bio-radar system. But the team did not have a prototype, and they limited themselves only to presenting their ideas, ”says Maxim Chizhov, an expert on the competition.
Another team decided to use a seismic sensor - a device that, among the vibrations on the ground, can recognize human steps and show the direction from where they come. Using the prototype, they even managed to find an extras that depicted a “loss” (as the participants affectionately called them), but the team did not go far in the competition.
By June 2019, after several training tests in the forests of the Leningrad, Moscow and Kaluga regions, the 19 best teams reached the semifinals. They were given the task of finding two extras in less than 2 hours on a plot of 4 square kilometers. One traveled through the forest, the other lay in one place. Each team had two attempts to find a person.
“Among the semi-finalists, one team wanted to create a swarm of drones, which was supposed to fly under the crowns of trees, controlled by artificial intelligence, determining the direction of movement, flying around the trunks, dodging branches and twigs. Using AI, he would analyze the environment and identify the person.
But this solution is still quite far from the implementation in a functioning form. I think it will take about a year for it to work even in test conditions, ”says Maxim Chizhov.
Close to success was the ALB Search team. They had a loudspeaker on board that connected to the walkie-talkie, a microphone that could listen to the surrounding space, a camera and a computer with AI and a trained neural network that processed real-time images from the camera, where a person could light up.
“The operator could analyze not thousands of images, which is physically impossible, but tens or even units, and then decide whether to change the drone’s route, whether an additional drone is needed for reconnaissance, or immediately send a search group.”
But most teams faced similar problems - the technology was not adapted to the conditions of a real forest.
Computer vision, which many relied on, worked when tested in parks and woodlands - but proved useless in a dense forest.
The thermal imagers, which were hoped by about a third of the teams, were also ineffective. In the summer - and at this time most people disappear - the foliage heats up so that it turns into a continuous hot spot. It’s easier to search in a small interval at night, but there are still a lot of heat spots - heated stumps, animals and much more. A camera could help to verify suspicious places, but it’s of little use at night.
On top of that, it was difficult to get thermal imagers. “Unfortunately, due to the restrictions placed on us by the EU and other countries, good thermal imagers are not available in Russia,” said Alexey Grishaev from the Top team, which relied on this technology.
“Thermal imagers available on the market have a digital output with a frequency of 5-6 frames per second and an additional analog video output with a high frame rate but low image quality. As a result, we found a very good Chinese thermal imager. We can say that we were lucky - in Moscow there was only one. But he gave out a picture on a small monitor, where nothing was visible.
Most teams used just the video output. Our team was able to finalize the model and get a high-quality digital image from it with a frequency of 30 frames per second. The result was a very serious thermal imager. Better, probably, only military models. "
But even these problems are only the beginning. In the short time that the UAV flew over the search area, cameras and thermal imagers captured tens of thousands of images. It was impossible to transfer them to a point on the fly - there was no Internet or cellular communication over the forest. Therefore, the drone returned to the point, records were downloaded from its carriers, spending at least half an hour on it, and in the end they received such an amount of material that it was physically impossible to watch for hours. In this situation, the “Top” team used a special algorithm that extracted images where thermal anomalies were detected. This reduced data processing time.
“We saw that not all the teams that came to the qualification tests understood what forest was. That a radio signal spreads differently in a forest and is lost quite quickly, ”Maxim Chizhov announced at a press conference. “We saw the surprise of the teams when the connection disappeared already at a distance of one and a half kilometers from the start point. For some it was a surprise the lack of Internet over the forest. But it is a reality. This is the forest where people are lost. ”
The technology based on light and sound beacons has shown itself well. Four teams reached the final, three of which relied precisely on this decision. Among them is Nakhodka from Yakutia.
“When we saw this jungle near Moscow, we immediately realized that there is nothing to do with drones. Each tool is needed for its task, and they are good for exploring large open spaces, ”says Alexander Aitov.
To the semi-finals in the team there were only three people who went through the woods on foot and placed beacons in the search area. And while many were solving engineering problems, Nakhodka worked like rescuers. “It’s not necessary to use agrarian psychology when you simply cover the area. We must behave as a lifeguard, put ourselves in the place of the missing person, look at the approximate direction where he can go, what paths. ”
But even the lighthouses of Nakhodka by this moment were not as simple as they were several years ago in Yakutia. Using grants from Sistema, the team’s engineers developed radio technology. Now, finding a lighthouse, a person presses a button, rescuers immediately receive a signal and know for sure which lighthouse the lost will be waiting for them. An UAV is needed not for search, but in order to raise the radio signal repeater into the air and increase the radius of transmission of the activation signal from the beacons.
Two more teams developed entire search engines based on sound beacons. For example, the MMS Rescue team created a network of portable beacons, where each beacon is a repeater, which allows you to transmit a signal about its activation even in the absence of direct radio communication with the search headquarters.
“We have a group of enthusiasts who have taken up this task for the first time,” they say. “We were engaged in other industries - technology, IT, we have specialists from the space field. We got together, stormed and decided to make such a decision. The main criteria were low cost and ease of use. So that people without training can take and apply. ”
Another team - "Stratonauts" - using a similar solution was able to find an extras faster than anyone. They developed a special application that tracked the position of the drone, and the location of the lighthouses, and the position of all rescuers. The drone that delivered the beacons also acted as a relay for the entire system so that the signal from the beacons would not be lost in the forest.
“It was not easy. One day we got wet well. Two of our people went into the forest through the windbreak, and they realized that this was far from a picnic trip. But tired and contented, we returned - all the same, we found the person in both attempts in just 45 minutes, ”says Stanislav Yurchenko from Stratonavtov.
“We brought lighthouses with drones to the center of the zone to provide the greatest coverage. A drone can carry one beacon in one flight. This is a long time - but faster than a person. We used the small compact drones DJI Mavick - one beacon the size of it. This is the maximum that he can carry, but it turns out budget. Of course, I would like to find a completely autonomous solution. With AI, so that the drone scans the forest and determines the drop points. We now have an operator, and after a kilometer, if you do not use additional devices, the connection ends. Therefore, in the next stage we will come up with something. ”
But not a single team found an immobilized person, and most importantly, they never figured out how to do this. Theoretically, only the “Top” team had chances to find it, which, despite all the difficulties, was able to find a person and go to the finals with the help of a thermal imager and a camera.
“Initially, we had the idea of using two aircraft-type drones,” says Aleksey Grishaev from “Top”, “We developed them to determine the composition of the atmosphere, and we still have the task of making an all-weather UAV. We decided to try them in this contest. The speed of each is from 90 to 260 km / h. The high speed and unique aerodynamic characteristics of the UAV provide the ability to search in any weather conditions and allow you to quickly scan a given area. ”
The advantage of such devices is that they do not fall when the engine is turned off, but continue to plan and sit on a parachute. Less - they are not as maneuverable as quadrocopters.
A thermal imager and a high-resolution camera modified by the team are installed on the Verkhnyaya main drone, and only a camera on the second. On board the main UAV is a microcomputer, which, using the software developed by the team, independently detects thermal anomalies and sends their coordinates with a detailed image from both cameras. “Thus, we do not have to watch all the material live, and this, for understanding, is about 12,000 images per hour of flight.”
But the team created the technology of the aircraft recently, and there were still many problems with it - with the launch system, with a parachute, with an autopilot. “We were afraid to take him to the test - he could just fall. I wanted to avoid technical problems. Therefore, they took the classic decision - DJI Matrice 600 Pro. "
Despite all the difficulties, due to which many abandoned cameras and thermal imagers, Vertex was able to find an extras. This required a lot of work, firstly with a thermal imager, and secondly with the search methods themselves.
For three months, the team tested the technology, which allowed the thermal imager to view the ground between the crowns. “There was a bit of luck, because the route of extras ran through such forests that not a single thermal imager would see anything. And if a person is tired and sat somewhere under the tree - it will be unrealistic to find him.
From the very beginning, we refused to completely comb the forest with our UAVs. Instead, we decided to look for a man flying over clearings, clearings and open areas. "I arrived at the place in advance to explore the area, and using all the maps available online, I drew routes for UAVs only over those places where a person could theoretically be visible."
According to Alexei, using several drones at once in a bunch is very expensive (one carrier with a technical solution for searching on board costs more than 2 million rubles), but in the final it will be necessary. He believes that this gives a chance to detect a stationary extras. “We initially wanted to search for a bedridden person. It seemed to us that we could find a moving one anyway. And the teams with the lighthouses were only looking for the moving one. ”
I asked Alexander Aitov from the Nakhodka team - do not they think that everyone had already buried a static person in advance? After all, beacons are useless to him.
He thought for a moment. It seemed to me that all the other teams spoke with smiles and a twinkle in their eyes about solving engineering problems. The guys from MMS Rescue joked that a discarded beacon could fall directly on a lying person. The "Stratonauts" recognized that this is a very difficult task for which there are no ideas yet. And the rescuer from Nakhodka spoke, as it seemed to me, with a mixture of sadness and hope:
- We have lost a girl in the taiga, in three and a half years. She spent twelve days there, and ten days were searched by a large number of people. When they found her, she lay in the grass, from above she was practically invisible. Found only by combing.
If the lighthouses were placed ... at three and a half years the child is already quite conscious. And perhaps she would come up to him and press a button. I think part of their lives would be saved.
- Did you save her?
- Her yes.
In autumn, the four remaining teams will go to the Vologda Oblast, and the task will be much more difficult for them - to find a person in a zone with a radius of 10 kilometers. That is, on an area of more than 300 square kilometers. In conditions where the drone has half an hour of flight, vision breaks down on the crowns, and communication disappears after a kilometer. According to Maxim Chizhov, so far no prototype is ready for such conditions, although he believes that everyone has a chance. Grigory Sergeev, chairman of the search and rescue squad “Lisa Alert” adds:
“Today we are ready to use a couple of technologies from those that we saw, and it will be effective. And I urge all participants and non-participants - guys, test the technology! Come to us in search! And then it will not be a secret to anyone that the forest is opaque to the radio signal, and the thermal imager does not see through the crowns. My main dream is to find more people with less power. ”