Communication restoration after the flood in Krymsk
We learned about the floods late at night from an engineer monitoring the monitoring. The base stations, one after another, immediately showed a cascade of accidents (almost all the sensors were triggered), and then completely left the network. A little later, news about the flood itself began to arrive at the technical service, two engineers from Novorossiysk left for the place, as they were the closest, and one of them had parents living there.
The city at that time was closed for entry: they promised to let them in only in the evening. One of our engineers made his way to the city for dinner (he hurried to his parents), and after making sure that his family was okay - they actually had an apartment flooded to the ceiling, but there were no casualties - he began to go around the base stations and report what happened to them. They managed to approach only a few, there was still water around the rest and it was only possible to understand that they were completely flooded.
This is what the city looked like when we arrived - and the further we went, the more destruction we saw.
At different points in time, several base stations did not work at once: several sites simply flooded the roof, some of the power was cut off, and they switched to batteries, and then turned off. Three sites were completely out of order and, what is most unpleasant, two of them were transit sites, that is, those on which the operation of other base stations depended. The remaining part of the network was under huge overload due to the exhaustion of the channel capacity - there was a huge peak in attempts to reach the city and from the city.
By the time we were allowed into the city, the water had already left. Judging by the footprints on the buildings, it rose in places up to 3 meters. Half of the city was flooded so that it was simply impossible to drive. There was no longer chaos as such, many who sat at home, guarded them. The bodies have already been taken out. There was no light in the city, due to station accidents, the power supply was cut off. There were a lot of cars on the road.
At the time of commencement of work, we had 4 all-terrain vehicles, each with a ready-made and fully-loaded generator, plus a 12-hour supply of fuel, tools and equipment for work. In each car - two people. The equipment was completed in advance according to information from monitoring and the engineer located at the site, complete assembly and loading took several hours.
Our main task was to restore communication. From evening to six in the morning we “picked up” all the base stations except one, and restored it to dinner. When the water allowed us to approach the building with the equipment, we began the restoration. Where the problem was only in the food - generators were brought (or dragged on their hands). They worked for 4 hours, cooled down for an hour, and the station used batteries at that time.
The first one raised the transit BS, which returned two more unfilled stations to the network to which the generators had already brought up, then they took up the rest.
At three flooded stations, it was necessary to fully deploy new equipment. We turned off the old blocks, installed new ones brought in advance (they are unified for the entire network) and skipped the temporary wiring to the transmitting equipment (antennas). All wiring and all flooded equipment were immediately written off. Where the water only touched - it was later taken to the technical site. Only a few units of iron were repaired.
We worked in the dark with lanterns; there was a lot of dirt and silt in the rooms.
Given that the premises of the base stations were completely or partially damaged, it was necessary to protect equipment (cases of vandalism were noticed in the city). At least one person was on duty at each station (let me remind you, radiation from the equipment under the antenna itself and through the material allows you to stay in the BS room for at least a year - there are details here ).
When it dawned, it was necessary to get to the second transit hub, but the water there in the evening was through the headlights of the Ford Ranger (the highest car of ours). In the morning the level was asleep, and in a roundabout way we managed to get to the BS container. There was water to the station itself, I had to swim. There is a blockage at the entrance, they raked it, they went in, assessed the damage, dragged iron on their hands (fortunately, the containers are mounted on a high foundation, so half of the equipment remained whole). While they threw new wiring and started the generator, at the same time they threw silt and somehow washed the floors.
Dragging equipment in the morning.
All restored stations were immediately expanded to the maximum configuration to enable communication to the maximum number of people. About a day later the channel capacity was full, after which the connection was restored in a relatively normal mode.
In the territory of the Kuban there was technical roaming - our network picked up all the calls of those people who could not get access to the network of their operator (this scheme is used when calling an emergency service outside the network, for example). Later, another team helped raise 8 offices and recharge mobile phones of all those who came. Free recharge cards were handed out around the city (in our offices and in the Red Cross branches), and there were also phones in the offices where you could call for free anywhere in Russia and the CIS.
This is what the temporary points for free calls looked like - right on the street, took the existing promotional stands instead of tables and threw the cable from the generator to charge the phones.
In normal mode, everything worked only after a week: the premises were restored, new cooling systems were installed.
In general, this is not the first accident that we are leaving for. We had cases of flooding of base stations, and much more - the restoration work itself was done clearly and very professionally, despite the scale. I cannot but mention the people with whom we worked. These are Yuri Plaksa, Alexander Kostin, Nikolay Krot, Vyacheslav Kuralesov, Sergey Smirnov, Alexander Bakholdin, Artem Blinnik, Evgeny Vozhakov, Kirill Abyzov, Anton Chertogov, Evgeny Yuryev, Andrey Nesterov, Igor Tucha, Dmitry Karpenko, Semyon Zuev, Gennady Chernykh, V Tikhonov, Igor Gorbenko, Alexey Borisov, Vladislav Kozhin, Vladimir Koltovskoy, Inessa Skobeleva, Natalya Stishenko, Dmitry Mikhalin, Sergey Suslov - our team, those who traveled to the city and those who worked on recovery with us in the group remotely. The other team was involved in the shares in the city described above,
The city at that time was closed for entry: they promised to let them in only in the evening. One of our engineers made his way to the city for dinner (he hurried to his parents), and after making sure that his family was okay - they actually had an apartment flooded to the ceiling, but there were no casualties - he began to go around the base stations and report what happened to them. They managed to approach only a few, there was still water around the rest and it was only possible to understand that they were completely flooded.
This is what the city looked like when we arrived - and the further we went, the more destruction we saw.
At different points in time, several base stations did not work at once: several sites simply flooded the roof, some of the power was cut off, and they switched to batteries, and then turned off. Three sites were completely out of order and, what is most unpleasant, two of them were transit sites, that is, those on which the operation of other base stations depended. The remaining part of the network was under huge overload due to the exhaustion of the channel capacity - there was a huge peak in attempts to reach the city and from the city.
By the time we were allowed into the city, the water had already left. Judging by the footprints on the buildings, it rose in places up to 3 meters. Half of the city was flooded so that it was simply impossible to drive. There was no longer chaos as such, many who sat at home, guarded them. The bodies have already been taken out. There was no light in the city, due to station accidents, the power supply was cut off. There were a lot of cars on the road.
At the time of commencement of work, we had 4 all-terrain vehicles, each with a ready-made and fully-loaded generator, plus a 12-hour supply of fuel, tools and equipment for work. In each car - two people. The equipment was completed in advance according to information from monitoring and the engineer located at the site, complete assembly and loading took several hours.
Our main task was to restore communication. From evening to six in the morning we “picked up” all the base stations except one, and restored it to dinner. When the water allowed us to approach the building with the equipment, we began the restoration. Where the problem was only in the food - generators were brought (or dragged on their hands). They worked for 4 hours, cooled down for an hour, and the station used batteries at that time.
The first one raised the transit BS, which returned two more unfilled stations to the network to which the generators had already brought up, then they took up the rest.
At three flooded stations, it was necessary to fully deploy new equipment. We turned off the old blocks, installed new ones brought in advance (they are unified for the entire network) and skipped the temporary wiring to the transmitting equipment (antennas). All wiring and all flooded equipment were immediately written off. Where the water only touched - it was later taken to the technical site. Only a few units of iron were repaired.
We worked in the dark with lanterns; there was a lot of dirt and silt in the rooms.
Given that the premises of the base stations were completely or partially damaged, it was necessary to protect equipment (cases of vandalism were noticed in the city). At least one person was on duty at each station (let me remind you, radiation from the equipment under the antenna itself and through the material allows you to stay in the BS room for at least a year - there are details here ).
When it dawned, it was necessary to get to the second transit hub, but the water there in the evening was through the headlights of the Ford Ranger (the highest car of ours). In the morning the level was asleep, and in a roundabout way we managed to get to the BS container. There was water to the station itself, I had to swim. There is a blockage at the entrance, they raked it, they went in, assessed the damage, dragged iron on their hands (fortunately, the containers are mounted on a high foundation, so half of the equipment remained whole). While they threw new wiring and started the generator, at the same time they threw silt and somehow washed the floors.
Dragging equipment in the morning.
All restored stations were immediately expanded to the maximum configuration to enable communication to the maximum number of people. About a day later the channel capacity was full, after which the connection was restored in a relatively normal mode.
In the territory of the Kuban there was technical roaming - our network picked up all the calls of those people who could not get access to the network of their operator (this scheme is used when calling an emergency service outside the network, for example). Later, another team helped raise 8 offices and recharge mobile phones of all those who came. Free recharge cards were handed out around the city (in our offices and in the Red Cross branches), and there were also phones in the offices where you could call for free anywhere in Russia and the CIS.
This is what the temporary points for free calls looked like - right on the street, took the existing promotional stands instead of tables and threw the cable from the generator to charge the phones.
In normal mode, everything worked only after a week: the premises were restored, new cooling systems were installed.
In general, this is not the first accident that we are leaving for. We had cases of flooding of base stations, and much more - the restoration work itself was done clearly and very professionally, despite the scale. I cannot but mention the people with whom we worked. These are Yuri Plaksa, Alexander Kostin, Nikolay Krot, Vyacheslav Kuralesov, Sergey Smirnov, Alexander Bakholdin, Artem Blinnik, Evgeny Vozhakov, Kirill Abyzov, Anton Chertogov, Evgeny Yuryev, Andrey Nesterov, Igor Tucha, Dmitry Karpenko, Semyon Zuev, Gennady Chernykh, V Tikhonov, Igor Gorbenko, Alexey Borisov, Vladislav Kozhin, Vladimir Koltovskoy, Inessa Skobeleva, Natalya Stishenko, Dmitry Mikhalin, Sergey Suslov - our team, those who traveled to the city and those who worked on recovery with us in the group remotely. The other team was involved in the shares in the city described above,