Walks on data centers and telecoms of St. Petersburg

    The first days of summer are great for a development trip to St. Petersburg.
    We will visit Miran, linxdatacenter, RETN and Metrotek.

    5 am, Moskovsky Station, KFC, Moika Embankment, Plate, pigeons from the roofs, Isaky, Mars Field, Yandex.Travel Capture, and here it is - Miran.


    For several years now, Miran has been living in our lab with Eva, translation server, virtual Mikrotik Routheros, linkmeup hosting with IPv6 accessibility and now podcast hosting.

    At the entrance to Miran, we were first met by the impressive size of DGU.

    Most Diesel Generator Sets are on wheels for two reasons:

    • Mobility - in a couple of hundred meters there are several more modules of Miran. They have their own generators - but you never know - have to roll.
    • Mobility - they do not need to be registered as stationary power plants and, accordingly, project documentation is not needed.

    DGU Mirana in winter and summer starts in a few seconds. During this time, the entire data center is powered by batteries.

    And the iron HP in the Miran stand and the tour became possible thanks to Fedor Rusakov.

    The whole tour was held in a cozy home atmosphere. Coffee with cookies, a conversation for telecoms, for the origins of the company, about the transport of two hundred servers from Moscow to St. Petersburg at night. Without haste, we walked around the two buildings of the data center. Both - BrownField - remade from the factory floor. In the first one, which belongs to Miran, from which we started and where on the second floor we charged after a four-hour walk around Peter, at one time there was a diesel engine plant, a truly gigantic diesel engine. Actually there was no second floor - there were huge gates to round-trip. Everything was taken apart, the second floor was done, active cooling, redundant power supply, gas fire extinguishing system.

    Classic hot corridor, from where air flows into the air conditioners and under the raised floor into a closed cold corridor.

    Also hidden under the raised floor are the power cables.

    Miran was not born as a platform for one anchor client, but rather systematically developed supply in a market where there is demand.

    At one time, there was a large bank with all the relying gratings and ACS, but he had already moved out.

    Demand continued to grow, Miran too, a second building appeared - already for rent - also BrownField - two hundred meters from the first. The road lies along the embankment of Bolshaya Nevka past the Nobel’s house.

    The new building was built on a modular basis with the ability to transport the entire module in case of problems with the landlord. The first two boxes were bought from the manufacturer, the third one was already assembled taking into account old mistakes.

    The modular architecture turned out to be cons - they are relatively narrow - a hot corridor - 60 cm - two people can not separate. And most importantly - the cost of moving, as it turned out at the first calculation, is comparable to the construction of a new one.

    But they look both outside and inside fresh and fashionable.

    NOC for the guys on Orthodox Linux, a stable debian.

    By the way, the company got its name from the Meridian Integrated Recorded Announcement (MIRAN) system board in the Meridian telephone switchboard. And after that, the data center was so named.

    They decided to build the data center in the difficult year of 2008.

    Since then, the guys provide Dedik and the bell. Operator equipment is installed in special rooms.

    And this imposes certain restrictions and features on the work. For example, in racks there is a maximum bit of equipment, both in terms of machines and in terms of network. In Miran, I didn’t see Aristou and Mellanox. All equipment has different requirements for temperature in a cold corridor, so you have to maintain it at a very low level - about 20 ° C, that is, you can’t talk about low PUEs - it is 1.5 for Miran.
    Another caveat - with snake weddings in the racks. As soon as you let external comrades into their premises with their commutations, write “beauty has disappeared”. No one has an exception.

    Alas, introducing draconian measures here may be to their own detriment. Recent photos.

    And then merch from our table to yours and from yours to ours. Loaded the whole crowd in a dachshund - and in a link.


    These are serious guys who even wrote “Security Matters” on the temporary pass. This will be the leitmotif of the entire tour.

    After the gingerbread Miran in linxdatacenter, you realize that the luxury center came. On their day off, 4 people came together frankly not in the city center to tell other five people about themselves who are unlikely to become their customers :)

    We were met by Maria Svistunova and led to a large negotiation table. A little idle talk over Guatemalan coffee and go.

    Excursions are put on stream. Maxim and Oleg know what they are doing: they tell by the best thumb: clearly and with enthusiasm. Where necessary, they emphasize, and where not, they bypass it. With a sense of humor, too, complete order, so two and a half hours passed quickly.

    The pride of the data center is an uptime of 100% since its launch in 2011.

    Confidence is so high that for tenths there is not even a place on the board.

    Linx is an example of DC in GreenField format. The building was specially built to accommodate the DC. Therefore, the height of the ceilings, the strength of the floors, power supply inputs, its redundancy, SCS - were laid down in the project and taken into account at the construction stage.

    So, by default, the floor withstands a stand of 800 kg, but the mass can still be increased after additional strengthening. They say oracle racks worth in Falcon9 require a special attitude.

    The DC itself is powered by a gas-piston installation, and urban electricity is only a reserve. According to Maxim, this was a very right decision, because during the famous St. Petersburg blackout, the office and the nearby business center were damaged, and the data center continued to work.

    The diesel generator here is no longer a trailer on wheels, but a completely stationary room with chimneys rising high above the landscape.

    Start-up time is no more than two minutes, and on batteries the entire data center can work for about an hour. Diesel can work for 3 days without refueling.

    Here everything reigns order and thoughtfulness.

    Here it is, for example, Klyuchinitsa, where the customers of the data center can take the keys to their racks by fingerprint.

    And this is a room where customers can carry out any engineering work they need, for which there is a trench tool hanging on the wall.

    These are shelves for local operation with everything vital in everyday work:

    It is always charged and ready to go.

    The novice hiphop baseball cap in the photo above saved more than one person's head. It’s a solid hard hat - the best thing is to climb under a 1.5-meter raised floor, where there is no, no, but some sharp piece of iron sticks out.

    Actually, the employees from operation:

    Here is this unremarkable device - a differential pressure gauge.

    It is designed to solve a simple problem - to maintain the pressure difference between the corridor and the machine room. For example, so that it was inside a little higher and when the door was opened, the dust did not fly in, but instead flew away from the door.

    The fire extinguishing system is also gas - in case of trouble, the gas displaces oxygen and stops combustion.

    Only serious and very serious guys get up in the link, who have reasons for paranoia the size of Ararat.

    Lattices and separate from data center ACS - this is only the beginning.

    That's when two people come to solve a problem or make new commutation - one with a material key, the other with an electronic key - this is more interesting. And when an audience with glasses comes from a client, goes down under the raised floor, spreads two sheets of gratings and says that the child’s head will creep in here, you understand that a person saw some interesting things in his life.

    There were funny situations when, when designing the cage, everything that was possible was taken into account, all security systems, all sizes that allow you to import a rack and put equipment in it, but somehow did not think about the dimensions of the engineers who would work with it. And they are neither in a face nor in a profile placed between a cage and a rack.

    If I were the FSB, I would put my hardware in linx :)

    After the tour, we sat for a rather long time, mentally chatting with pizza.
    From this realm of security and order, we moved to RETN.


    We have known these guys for a long time and close. Therefore, it is not surprising that we finally gathered on their roof to fry kebabs and pickle bikes.

    We also visited them, although, it seems, we did not tell. Last year, plans to capture the St. Petersburg roofs failed due to Fifa.

    RETN is actually the largest fiber backbone operator worldwide. Any spider will envy their web.

    And DC on the bypass is such a home project for the soul, which has found its niche in the growing market. This is where those who need a thick connection with the world and access to other local operators get up.

    So, for example, Netflix’s CDN is in RETN.

    I was surprised to learn that each of their nodes is individually handwritten by a phrase.
    And such holiday boxes are threshed in different cities at RETN POPs (and, obviously, not only him), delivering to you as quickly as possible “Black Mirror” and “Sharp peaks”.

    There is also Google’s CDN and a bunch of operators, both local and federal.

    At RETN, next to the building, there are two tiny (compared to Miran) diesel generators and a car with canisters to go to a gas station. It sounds almost like a family business.

    Well, the evening talks on the roof brought us to the point where RETN will give us a virtualochka for a backup site for hosting podcasts and pictures.
    World guys, a superlamp data center, an amazing network.

    For some, this day ended in a duke nukem tournament, for someone, a long-awaited sleep in a hotel.

    The next day, we had not many plans: Metrotek and the podcast alive - linkmeup IRL .


    Just a month ago, Misha Sokolov and Denis Gabidullin from STC Metrotek Slash Plumspace came to visit a telecom podcast , and we looked at what Smart SFP is. And now we were able to touch them alive.

    Denis, who conducted a tour of the St. Petersburg office of Metrotek (which, by the way, has not changed since our last trip to them), actually from the threshold gave one of the samples.
    At first I was surprised how clumsy it was made - like a jigsaw and a file

    turned out of a 32 key. It turned out that the prototype case was printed on a 3D printer made of metal powder, which can be seen on a rough surface. Others - those who are ready for the series - in perfect order.

    But in general, looking at them, one wonders how they nevertheless managed to hide so many brains in an almost standard building.

    I advise you to listen to the podcast about Rosa, Tosa and their sister Bosa . It turned out interesting for my taste.

    The design of the office is as ordinary as possible. The relatively small rectangular cabinets, elongated to the window, are quite dark, which is a little crushed. But the atmosphere, despite the day off, reigned working.

    Design for two years has not changed. Pleased with some posters and boards.

    We went through several rooms: R&D, system engineers, FPGA speakers, held various types of SFPs in our hands, programmers, saw the first model in which Smart was removed from SFP on a wiring - one of the first experiments - rather, Proof of Concept, that it generally works .

    They say that it’s even convenient for some customers to have such a tail sticking out of iron.
    The very first goal two years ago was to assemble a conventional optical SFP that would at least transmit a signal.

    Then they separately assembled the Smart part from someone else's SFP (just on the cord) to make sure that they are able to assemble the chips.

    Metrotek has done a great job. Just think that in 2 years they have assembled a fully working optical module that can encapsulate, telemetry, an exact time server, and an interface converter.

    And at the same time debug to the ideal (although this remains to be verified with time) the basic functionality for signal transmission. There should be no attenuation, there should not be even rare drops due to SFP. And yet it should not be heated, despite a pack of ASICs, FPGAs and CPUs inside.

    Full-fledged computing system:

    • CPU: NXP i.MX6 ARM up to 900 MHz
    • FPGA: Lattice ECP5 to 85K LUTs
    • RAM: DDR3 up to 1 GB

    Denis noted that the company practices and encourages the exchange of knowledge and experience. And then I thought: “But is it otherwise? 0_o. " - Yes, people often come to them who, at the same place of work, cooked in their own juice, did not communicate, did double and triple work, and generally avoided communication attempts.

    And then I remembered that, in general, no more than two years ago, I worked in a multinational company, where conferences were banned and knowledge exchange was absent as a class. And it was I who was one of the evangelists of weekly meetings to exchange experiences that seemed to save single engineers in remote regions.

    In general, Metrotek regularly conducts internal trainings and lectures, why they are well done.

    And then we were taken to an amazing installation workshop. No matter how many times I have been to them, it seems that my childish delight does not quench.

    Metrotek has equipment for printing on boards right in the office.

    A stencil for applying solder pellets, a printer for SMD components (I just can't name it otherwise), a small oven for baking boards and nearby workstations for manual installation.

    To my surprise, I learned that microchips with hundreds of legs are also soldered by hand. Denis says that their guys straighten up the smallest details with amazing accuracy and speed and sometimes it makes no sense to start the shaitan-machine - a person will do it faster.

    We called each other the Metrotek Eltex-Verilight production line, in the wake of a relatively recent excursion to them.

    In addition, the office has a 3D printer and a milling machine for domestic use, on which, however, medals for last year's Chaos Construction were already engraved .

    From there we went to Krang Pizza, which was stunning in its ideology, where linkmeup already swallowed the mutant pizza burger in size and sense and moved to the IQ option office, where they recorded the first release of linkmeup IRL .

    The next day, we looked into the Mammoth Laser Center, where we experienced technocratic ecstasy and a little inflammation of greed, but this will be a different story.

    In conclusion, I want to say that all the excursions and meetings turned out to be very different, both in scale of action and in direction.

    Miran, who hospitably hosted our hosting, is a fast growing, but still friendly, simple one.

    Link is a truly cool data center, with security at the forefront and experienced marketers.
    RETN- young funny guys "from the garages" who make a giant network and as yet a small data center and would like to make a bonfire on the roof.

    Metrotek is a relatively small team of Russian developers of hardware and software, which makes a lot of measuring more than competitive equipment almost “with these hands”, and now it is entering a new market for itself with an ambitious task.

    And the Laser Center, which goes beyond the interests of linkmeup, but is the object of our personal interests. He left a very pleasant impression and the engraving of the microphone on the recorder.

    Also popular now: