How we built the TIER-III level fault tolerant data center

    In 2009, there was a warehouse of unique compressors that had not been used for a long time (in particular, for the Energy-Buran system), where a train had once called in to unload.



    We immediately liked the building . We bought it for the construction of our own data center of increased responsibility. A few years later, a modern data center appeared on the site of the workshop, which became the third in Russia certified as a facility by the Tier III Uptime Institute.

    Recently, I found quite a few photos from the construction. I think it will be interesting for you to see how all this looks in the process, usually such things do not show.


    And this is 2012, the engine room is just starting to fill

    Caution, under the cut a lot of photos

    Introductory


    This was already our third data center and by no means the first or even the twentieth of those that we built or helped build across the country for various customers. Accordingly, at this point we already had an excellent team of engineers and trusted suppliers.

    Basic parameters of the Compressor data center
    • Fault tolerance: Tier III, verified by the Uptime Institute,
    • Cooling: N + 1, an average of 6 kW / rack (there are 30 kW of rack, they are placed next to less powerful in the hall),
    • Power supply: 2N, UPS - 15 minutes, DGU - 24 hours without refueling,
    • Capacity - 800 racks
    • Premises of the warehouse and customer’s staff,
    • Protected area
    • Building owned
    • Connected by a fiber optic ring to the CROC data center network,
    • 6000 kW of total refrigeration capacity, of which 1500 kW are reserve,
    • End-to-end reservation of the cooling system N + 1 (3 + 1),
    • The average annual estimated PUE is not worse than 1.45,
    • Cold reserve 15 minutes when the power is turned off,
    • The temperature range is -36 ... + 37 degrees (this is the absolute minimum and maximum for the last 10 years in Moscow).


    We started the construction of the Compressor in 2009 . I wanted to do everything wisely, since the specifics of some of our customers are such that the data centers should work no matter what. The TIER-III level was chosen as the most appropriate for the tasks (such data centers continue to work stably without changes when any node of the system fails and have an availability indicator of 99.98%). We passed the certification of the Uptime Institute for the project and for the facility - before us in Russia there were only two data centers of this level.

    So, at the beginning of work, we had a building. In general, it was in good condition. We added overlap between the first and second floor, plus we built up the third floor for domestic needs (for example, there is now a dining room).

    First, we took out everything we found in the workshop:


    This is how the place looked in 2010, a top view. This is the installation of the ceilings of the second floor:


    The same moment, the view from the inside of the building. Here we are installing the supporting pillars of the second floor:


    2011, the installation of ceilings of the second floor is completed, we are installing internal partitions:


    Here you can see the height of the raised floor, this is January 2011:


    In the same January 2011, the technological rooms were ready for decoration:


    And in February and In March we mounted a double floor in an uninterruptible power supply room:


    Double floor in a battery room:


    Fire protection of load-bearing metal structures is visible here, also February:


    Photo of March 2011, construction of a route between a distribution substation diesel generator sets and data center building:


    And here you can clearly see the openings where the air flows under the raised floor. The height is about 1.2 meters, this is due to the high power.


    April 2011, we are building pipelines. In the photo - pipelines for chilled water to fan coil units. The main pipes have a diameter of 400 millimeters, and the valves were per ton, which created a number of difficulties in installation.


    Completion of the cooling circuit of the machine rooms


    Installation of pipelines for piping chillers and cooling towers, May 2011:


    June 2011, we install the cooling system, this is the installation of chillers and drycoolers on a special construction on top of the building.




    Their use was due to the fact that it was necessary to fit into the existing dimensions of the building, so the chillers were put on the roof.




    But the installation of the pumping station in July 2011 has already been completed:


    Busbars also appeared in July 2011:




    And here there are switchboards:


    July 2011, installation of suspended ceilings and trays in engine rooms is in full swing


    In August 2011, indoor units of air conditioners were installed:


    This is how they look when installed:


    Installation of a cooling system for a distribution transformer substation :


    Distribution substation for a diesel generator set at the cable connection facility:


    In August 2011, we also began to mount the building facade. Here the insulation is already almost installed, an entrance gate is being installed:


    The same building facade on the other hand:


    And this is the installation of cable trays of a diesel power station. Several tons of cable will pass here (photo - also August):


    The refueling site in August was already completely ready:


    The final painting of the pipelines in the premises of the ground floor refrigeration center:


    This is what the distribution substation of the diesel power station looked like in August, inside view


    August 2011, the compressor will be launched very soon, but CROC employees can’t wait to see it (on the roof with chillers):


    Here are the diesel generator sets, during installation. In total, 7 of them were installed. Start of installation of the cable system. The photo was taken in September 2011:


    And these are gas fire extinguishing cylinders of cars, also the September photo:


    By September 2011, the installation of the ventilated facade of the building was almost completed:


    The last thing they did was decorate and reserve non-critical equipment like microwaves.


    That's all. After the facade of the building was completely lined, we tidied up the territory, put up a fence for protection and planted a lawn.

    References


    Here there is a photo tour of this data center, here - more details , and here - about the features of certification .

    PS If it’s interesting, I’ll write a post about equipment, engineering systems and suppliers (by the way, some domestic suppliers have proven themselves very well).

    Also popular now: