Stories from the life of an IT specialist on a vessel that passed around Antarctica

Carles Pina-Estani can not be called a typical explorer of the Arctic. This native of sunny Barcelona works as a programmer for Mendeley . This technology company is owned by the Elsevier science publisher and is based in London. Until this year, he had never slept on a ship. But when he was invited to participate in a three-month expedition around the Antarctic, he gladly jumped at the chance.

It all happened pretty quickly. Pina-Estani’s partner, Jen Thomas, who previously worked with the British Antarctic Service , worked as a data expert on a research trip led by the newly created Swiss Polar Institute [ Swiss Polar Institute]. SPI brings together researchers working in the Arctic Circle or in other extreme conditions, conveys information to the general public about their existence, and facilitates researchers' access to these areas. The tour was paid for by the Swedish billionaire and adventurer Frederick Paulsen Jr. - and he even went along with everyone . This is not for you in the office to engage in technical support.



An IT specialist needed to be on board (in addition to two electronics and maintenance engineers). Fortunately, Thomas was familiar. And suddenly Pina-Estani discovered that he was aboard the Akademik Treshnikov , a Russian research vessel, embarking on an ambitious route known as the Antarctic Loop , through Cape Town in South Africa, Hobart in Tasmania and Punta Arenas in Chile. The route "Antarctic loop"




During the brief stay of our hero in the role of a marine IT specialist, “Akademik Treshnikov” struggled with bad weather, food below average, and telecommunications equipment that did not want to work properly. And although in any such trip we should expect difficulties, Pina-Estani faced a set of problems from the very beginning because of the too little time allotted for the preparation of the equipment. As he himself commented optimistically, “I had more opportunities to solve all kinds of tasks.”



“Now I’m 35, and I’ve been working with computers for over 20 years - in the afternoon, in the evenings, on weekends, and in the form of personal projects,” he told us. - And in this campaign I used everything I know, and even the rare knowledge that I considered useless was useful to me. In the Antarctic, everything becomes useful. ”

When Pina-Estani's unusual work time came to an end this summer, and he returned to London at the end of July, he found time to talk with our editorial board and to tune our inner network salesman with stories about the life of IT in the high seas. At least from his story, technical support in the most extreme parts of the world is not too different from technical support in more familiar conditions - except for the lack of reliable communication and the ability to buy additional parts from Amazon or a local store. But the need for help is constantly present, emotions can go off-scale, and requirements vary from simple work with an email server to such things that no programming course can teach you.

Winch

One evening, about a month after the start of the campaign, one researcher cheerfully turned to Pina-Estani with what he called the "new ordeal." Before that, he mostly worked with the usual equipment that did not take him out of the comfort zone: computers, routers, hard drives and Raspberry Pi.

But this application for support was in a completely different area. A massive winch began to fail, a mechanism consisting of a cable and a crank arm.

Until then, Pina-Estani did not even know what a winch was. And this winch was responsible for the descent of the only on-board CTD-profilograph , which collected and analyzed water. Typically, CTD was lowered to a depth of 1500 m, and this device was critical for the majority of 22 teams of researchers present on board.



It turned out that the problem was in software. The error interfered with the winch function by smoothly lowering the long cable into the water. Pina-Estani tried to debug, but the equipment manufacturer told him that it was impossible to remotely enter new parameters into the computer of the CTD winch. And since “Akademik Trёshnikov” was in the ocean, this caused minor difficulties.

The decision required the use of hacking skills and a willingness to challenge the elements:

“The temperature was from zero to -2 degrees, ocean spray was everywhere, the boat rocked, and my hands were frozen. People asked me what I was doing there with the computer - the CTD was connected via a very short network cable. So you had to work outside, ”he tells me. “I got access to the winch computer that was running Windows CE from my Linux machine.” I saw the IP address on the boot screen using nmap, I discovered that it has a remote desktop server. I was very happy when after pressing Enter it turned out that I was able to change the parameters! ”



Pina-Estani wasn’t happy for long - it didn’t solve the problem. After the manufacturer reluctantly allowed him to reinstall the program, he had to wait until they moored in Hobart for restocking to download the firmware via WiFi in the hotel.

“We reinstalled everything, and the computer still did not boot,” he says. - It was one of the worst moments of the whole expedition from the point of view of IT. The problem was that CTD was one of the most necessary items among scientific equipment - but this is a big winch, and I did not have any knowledge in this area. ”

Fortunately, this story has a happy ending. After reinstalling the update in another way, the winch ceased to be capricious. Soon, all scientists successfully collected their water samples, and Pina-Estani was finally able to carry his computer back to heat.

Network hack

Another memorable application for support came from a scientist about the apparatus, which she used to measure the reflectivity of the sea. She needed to remove data from the device, but she could do it only through a connection with a remote access point.

“Well, I say - yes, there are no problems, but where is the router? - he recalls. “He was not on the ship — it turned out that he was standing somewhere in Australia at all.”



The most distant points of this loop, which ran from Cape Town to Punta Arenas, can be 10,000 km from Australia, and at that moment, according to Pina-Estani, they were about 3,000 km from South Africa. But this did not stop him, he used his laptop as a remote access point, connecting to the equipment FTP server to access the data. “But it was not a very convenient solution, because every time they needed data, they needed to get to me and my computer.”

Instead, Pina-Estani used a simple but preferred device for network hackers - a smartphone. “I figured out how to hack a network with an Android phone,” he explains. - It can raise the access point, and even without an external signal, the device can connect their laptop to the equipment through the phone. In this way, scientists could get their data when they wanted, without my participation. ”

Mail system

At the time of dial-up, you could spend a lot of time watching the download process of a single email or page. What was annoying at the time, today's people who are used to broadband access, are just crazy. But when a lot of people try to use an unreliable Internet connection via satellite to send and receive mail, don't wait. This can quickly raise social tensions.

“I've never seen so many people hitting their computers,” Pina-Estani recalls. - It was painful to see it. I can not stand the spectacle of a scientist standing in front of his computer in anticipation of the next step. My heart just bleeds. ”

With the exception of general disappointment, the accumulating queue of emails meant that people could not access some important permits and other documents related to the expedition. At some point, about 100 MB of letters got stuck in the system, so Ping-e-Estani started extracting them.



“It would be easier to return to the port and get them than to get them in the middle of the sea through Outlook,” he explains.

Of course, such a route change was impossible. Instead, he used the access to the remote server to download and archive all mail, and then send these files to the ship using the rsync program, which copes well with unstable connections. He also wrote a script that controls the resumption of the interrupted download from the same place where it was interrupted.



“So I left this program running for 8-9 hours, and then I opened this big file in Thunderbird,” he said. “After that I was able to get all the necessary letters, including the permissions.”

But such a solution to the problem on the knee did not solve the main one - the ship received letters, but the expedition still did not have a reliable communication system.

“I knew how to fix it, but for this I needed several hours of reliable Internet,” said Pina-Estani. - During our three-day break after the first transition, Jen and I went to the hotel, where I [via the Internet] set up a new domain for the website and a webmail server, and then created each user. He became very popular. Everywhere, wherever I was on a ship, people used my webmail. Watching this was amazing. ”



In order for everything to continue to work without failures, Pina-i-Estoni needed to limit the size of sent emails to 200 Kb, which means another solution was needed to transfer larger files.

“I developed a system in which letters swung in pieces,” he recalls. - It was, in fact, a hack, because the mail protocols do not allow to separate the letters - they either download them or not. I bypassed it, and made it so that if the download was interrupted by 20%, it resumed anew from the same place. ”



He also configured the queue system to work with a slow connection. Thanks to her, even if it took someone 5-10 minutes to receive mail, users immediately received notifications and knew that everything was working.

“I was very worried about the fairness of the system, so the letters were downloaded in the order received,” he said.

Ferry box

The last story from Pina-Estany happened in an area that any IT-specialist one way or another faces: data management. The task of managing the huge amount of real-time data collected by the expedition was one of the most difficult and ongoing tasks to which he applied his computer skills. The best example is Ferry Box, a Linux-based machine that is constantly used to measure salinity and temperature on the surface of the water.



Since access to each subsequent data set appeared every few days, it was very difficult for scientists to plan ahead. For quite a long time they could remain in the dark about important events, for example, the transition from one water to another, with excellent characteristics.



“To solve this problem, I figured out how to put the data on a website, to which scientists had real-time access,” says Pina-Estani. “In this way, they could make decisions about where to stop and, for example, take samples.”



This functionality allowed researchers to move from reactive to proactive data collection activities. It is easy to see how this affects science itself. And after several such stories from Pina-Estani, it’s easy to see how even the most extreme scientific research depends on modest IT workers.