February 16, 2016 at 16:50Data Center with Warranty
To use the equipment, we use several data centers and decided to create a series of materials about the life support system of one of them.
DataPro on Aviamotornaya Street is one of the most reliable Tier III data centers, one of the few commercial data centers in Russia that have received Uptime Design and Facility certification. Today we will talk a little about the project and consider in detail the technical implementation and features of the energy supply system.
The data center at Aviamotornaya 69 is the second data center built by DataPro, which specializes in providing services to commercial data centers.
International certification of Uptime Design and Facility means that the data center is designed and built in accordance with all applicable standards for the Tier III reliability category. The next level in 2016 will be the certification of Uptime Operations, confirming the compliance of all procedures and procedures implemented in the data center with the established requirements. The maximum downtime for a data center of this level is 1.6 hours per year.
In 2015, DataPro Data Center at Aviamotornaya was awarded the Best Project of the Year Award. In September, the first professional awards ceremony in the field of data center construction took place in Russia. The competition program was attended by 19 data centers, claiming prizes in five categories. The international jury recognized DataPro's data center as the largest and most innovative and awarded DataPro the main prize of the Russian Data Center Awards 2015 .
To solve most problems - this is the only option in which you do not need to look for some compromises.
Uptime Institute certificates cannot be purchased. All technologies and solutions used are certified, this guarantees maximum data center resiliency to ensure the continuity of customers' business. All this leads to guaranteed availability of resources and is insurance against unexpected situations.
Ultimately, the cost of placing equipment in a data center of this level may be slightly higher. What does the user pay for?
The cost is determined for a reason, the user pays not for “status”, but for confidence in the availability of resources, confirmed by professionals.
In the subsequent materials we will tell in detail about all life support systems of this data center. And today, we will briefly consider the main elements of the infrastructure and talk in detail about power supply.
The average size of the machine room in the DataPro data center is 180 square meters. m, it is designed for about 100 racks or can be divided into several partitions. There are eight such halls in total. The building provides space for technical personnel, equipment storage, loading and unloading, catering. Clients can not only rent hard-seats, but also create a backup office, where the client’s critical staff will continue to work in case of emergency.
For cooling server rooms and auxiliary technological rooms of the data center, 8 Ecobreeze installations are now usedmanufactured by Schneider Electric. The temperature regime in the halls is maintained at + 25 ° C with a humidity of 40-60%. The use of this modular cooling system in combination with other engineering solutions allowed us to achieve a high PUE, the total average annual value of which does not exceed 1.3. In the next article we will talk about Ecobreeze in more detail.
Fiber optic lines were laid to the data center via two independent routes, connecting it with all the largest telecommunication communication centers and traffic exchange points in Moscow. The technology platform’s policy is neutral with respect to telecom operators; therefore, more than 10 telecom operators are present at the site (Garstelecom, Mastertel, RETN, Macomnet, MTS, Beeline, Orange Business Services, etc.).
Considerable attention is paid to security: security is responsible for the security of the data center itself and the surrounding area. The security system includes more than 350 network cameras. The data center also provides advanced methods for preventing fire: the Vesda early smoke detection system and the automatic gas fire extinguishing system based on HFC 227.
Power supply is the basis of reliable data center operation. It must be uninterrupted, for which uninterruptible power supplies (UPS) are traditionally used, and guaranteed, for which diesel generator sets (DGU) are used, which support the operation of the data center during a prolonged accident in the power supply network.
In the DataPro data center, two independent 10 kV inputs from the Mosenergo substation, and the cables are laid in different collectors, provide the necessary electrical power to the facility. Consumer power is provided through a group of step-down transformers. These dry transformers are constantly in operation when there is power from the city substation.
Now the authorized and connected capacity of the data center is 6 MW. Another 8 MW of power is being processed. The power supply of the data center is actually reserved according to the 2N scheme. For example, even at 100% load in the event of an accident, the power board, transformer, DRUPS, the entire machine building will be powered by a second input / shield. Customers are supposed to use equipment with redundant power supplies or rack-mountable ABPs.
Here are the main components of the data center power system. On the right are transformers, on the left are diesel-rotor UPSs (DRUPS).
Dry power transformers with a capacity of 1600 kVA each are used to power the data center. They convert 10 kV at the input to 0.4 kV at the output.
Part of a data center power system diagram.
Traditionally, when the voltage of the city network disappears, the batteries of the UPS must maintain power to the equipment until the DGU is fully launched, which can provide power to the entire data center. This process usually takes from 20-30 seconds to 2 minutes, for which the UPS power and battery capacity are calculated.
The DataPro data center, at its scale, uses a different, more economical approach. The main components of the DataPro data center infrastructure are power solutions for diesel-rotor UPSs manufactured by Hitec Power Protection (currently four units) and Euro-Diesel (two machines). A service contract with two different vendors allows you to optimize prices and compare service.
In fact, the diesel-rotor or dynamic UPS (DRUPS) of these manufacturers is a diesel engine, an electromagnetic clutch, a synchronous electric machine and a dynamic flywheel assembled coaxially on the same shaft. A decoupling choke connects the input of the mains voltage to the output of the “clean” voltage of the UPS system. It provides a high degree of isolation between input and output, blocking the passage of harmonics and transients.
In the presence of external power, the synchronous electric machine of the UPS operates in electric motor mode by rotating the flywheel rotor. When the network parameters go beyond the set limits, it goes into generator mode, maintaining a continuous sinusoidal voltage at the load. The source of energy during the transition process is the flywheel, due to which a stable rotational speed of the rotor shaft of a synchronous electric machine is maintained. The flywheel holds revolutions up to one minute at full electric load without a diesel engine.
After a power failure, the DRUPS control system sends a signal to start the diesel engine. The kinetic energy of the flywheel, in addition to maintaining the output voltage, is used to output the diesel engine crankshaft to the operating frequency. 50 ms after a power failure, the diesel engine starts to work, which within a few seconds goes to normal operation - the flywheel is already spinning.
Autonomous power supply of the facility in case of power failure from the substation is provided by Hitec Power Protection 1670 kVA diesel-rotor UPSs with a 4 / 3N redundancy circuit. And no batteries to power the IT load.
Power cables and ventilation duct.
Once a month, diesel engines are launched to check their performance. The estimated stock of diesel fuel in the storage is 17 hours, taking into account the incomplete load of the halls this time is much longer - up to 60 hours. Further fuel will be supplied from Moscow and, if necessary, from Ryazan. DataPro has concluded relevant contracts with its suppliers.
For storage of fuel, two containers of 60 tons are used, each of them is half full. It is also made for redundancy: in case of an accident or repair of fuel lines and pumps.
The Dutch company Hitec Power Protection is one of the leading manufacturers of uninterruptible power solutions based on diesel-rotary UPSs. It was she who first patented the idea of a UPS with a flywheel. Hitec Power Protection's dynamic UPS is currently protected by a fleet of equipment with a total capacity of more than 1.6 million kVA.
In Russian data centers, DRUPS are still rare, however, in the world, systems based on diesel-rotor UPSs, due to their simplicity, reliability and energy efficiency, are considered a good solution for powering data centers of high power (from 1 MW). A relatively small number of components and simplicity of design significantly increase the average uptime of this equipment. So at HitecThe total service life of dynamic UPSs is at least 25 years. At the same time, depending on the amount of available fuel, diesel-rotary UPSs provide backup power for any period of time.
A Hitec installation includes: (1) a diesel engine, (2) an overrunning clutch, (3) a generator, (4) an energy storage device, (5) a flywheel. DRUPS Hitec are optimized for energy efficiency, and their claimed input-output efficiency is 97%.
High energy efficiency of dynamic UPSs is achieved due to the absence of the need for multiple voltage conversion - rectifiers, inverters, etc. There is no need for expensive and dangerous batteries, and therefore, in a special room for their placement, in the organization of its conditioning and, most importantly, in their regular replacement every 3-5 years.
Diesel-rotary UPSs are considered the most economical and energy-efficient solution to ensure uninterrupted and guaranteed power supply to large data centers. Until recently, according to the research company Frost & Sullivan, rotary UPSs occupied no more than 6% of the global UPS market. However, according to forecasts, their share should increase significantly as the size of the data center increases.
Distinctive features of DRUPS.
For a diesel-rotary UPS, an average of 40-60% less area is required than traditional static UPSs with similar characteristics, taking into account DGS. A single DRUPS unit is used instead of a system of backup DGUs, static UPSs, battery cabinets, switchboards, air conditioning systems, etc. The data center area is used in an optimal way.
Thanks to DRUPS, virtually all power quality problems are controlled. The installation provides filtering of all network interference and load, feeds capacitive and inductive loads, while the power factor is 0.98.
Electric power from the city network, passing through the inductor, feeds the synchronous motor-generator. The alternator shaft rotates at 1,500 rpm. In normal operation, the synchronous generator performs the function of a stabilizer and a voltage filter: eliminates harmonic distortion and power outages, impulse noise, eliminates surges and voltage dips. Thanks to him, the frequency and amplitude of the output voltage supplied to the load are invariably stable.
Unlike static UPSs, DRUPS is optimally used for powering air conditioners whose compressors are characterized by very high starting currents. DRUPS reduce investment in air conditioning because they do not require tight climate control. This affects the reduction in operating costs and the final cost of rent for consumers.
A dynamic diesel uninterruptible power supply has a high level of system self-diagnosis, allowing you to timely track and prevent malfunctions. When one of the six DRUPS is put out for repair, the data center will continue to function normally, with redundancy. The load is distributed between the rest of the UPS. This was checked even when simulating the full load of the data center. Given the partial load of the halls (usually it is no more than 60% of the design capacity), two DRUPS will be involved at the initial stage of entering the second stage of the data center. Later it is planned to purchase six more DRUPS for the second stage.
DataPro places projects from one rack, and the client should independently agree on cooperation with telecom operators.
Hostkeyis an official retail partner for DataPro. If you need to deploy equipment, rent dedicated servers or implement a complex converged project for high SLA in a Tier III certified DataPro data center, please contact.
Placing 1U equipment in our racks (a modern server with any power supply, not a twin) with a redundant dedicated 100Mbit / s Internet access channel costs 4000 rubles. per month, including VAT or only 40,000 rubles. in year. You can rent a whole rack, 1/2 or a quarter of a standard rack, including a seal. We will help with the installation, connection and preliminary configuration of the transferred equipment.
We offer for rent in DataPro any dedicated Supermicro serversor HP, HP / Dell disk enclosures, and network infrastructure (VLANs) from 100Mbps to 10Gbps (including security features and routers) at reasonable prices. You can create a hybrid solution consisting of your servers, leased servers and network infrastructure. HOSTKEY provides the highest SLA for servicing customers' own equipment.
Especially for readers of this article - refer to it before the end of February 2016 and get an additional 10% discount on the placement or rental of equipment in one of the best data centers in Moscow - DataPro.
DataPro on Aviamotornaya Street is one of the most reliable Tier III data centers, one of the few commercial data centers in Russia that have received Uptime Design and Facility certification. Today we will talk a little about the project and consider in detail the technical implementation and features of the energy supply system.
A bit about the DataPro project
The data center at Aviamotornaya 69 is the second data center built by DataPro, which specializes in providing services to commercial data centers.
International certification of Uptime Design and Facility means that the data center is designed and built in accordance with all applicable standards for the Tier III reliability category. The next level in 2016 will be the certification of Uptime Operations, confirming the compliance of all procedures and procedures implemented in the data center with the established requirements. The maximum downtime for a data center of this level is 1.6 hours per year.
Reliability levels in brief
The higher the reliability level of the data center, the more expensive it is during construction and operation. The cost of renting a stable place in it cannot be small. However, such costs are not significant for business-critical applications, and when choosing a data center, you need to pay attention to its certification. Only it can serve as the basis for statements that the data center really has a level of reliability of Tier III and really corresponds to it. However, for many secondary tasks, a data center with a level of reliability of Tier II or even Tier I will be sufficient
Data Processing Centers are designed to ensure the smooth operation of all critical IT processes of the company. Underestimating and reevaluating the impact of downtime negatively affects costs, because financial losses are possible in both cases - either when the critical systems are idle and the business stops, or because of unjustified overpayments for the service.
The Tier I level, according to the Uptime Institute, is a dedicated data center infrastructure without redundancy. Usually this is one or two air conditioners and a UPS without a diesel engine. If something breaks or if there is no electricity, this is an emergency. Any service is easy to provide. The issue price is about $ 10,000 in capital costs per rack of 5 kW.
Tier II provides primary redundancy, that is, power and cooling systems have redundant components and capacities. Typically, these are several air conditioners in N + 1 mode, one industrial UPS and possibly a diesel. Electricity is supplied through the ATS from 1-2 inputs. Premises - a room or a hangar, usually on lease. Maintenance is often difficult or poses a risk of accident even during routine maintenance. The issue price is about $ 20,000 of capital costs per rack 5 kW.
Tier III means redundancy of all engineering subsystems, the ability to carry out any operations, scheduled repairs and replace equipment without stopping services. This means N + 2 for air conditioners, 2 inputs per rack from different UPSs, a reserve for diesels, two inputs and a specially prepared room in the property. The issue price is about $ 30,000 of capital costs per rack 5 kW excluding real estate prices.
Finally, Tier IV means fault tolerance, the ability to withstand a single unplanned emergency event, such as a fire or explosion.
If the data center of the first level does not provide redundancy, then the second adds redundancy for the main engineering systems, but not for the “delivery routes” of the service — electricity and data transmission network channels. Reservation of the latter is implemented in the data center of the third level, which allows for routine maintenance without stopping it. The fourth level involves redundancy (N + 1 and above) of all major systems and fault tolerance in case of the worst event. The average estimated annual downtime of the data center varies from 28.8 hours for the first level to 0.8 hours for the data centers of the fourth level, and the actually observed availability factor is from 99.67 to 99.99%.
Thus, in a data center with a Tier III reliability level, it is possible to simultaneously operate and maintain a data center, up to replacing system components and equipment failure, adding systems. This makes it possible to carry out scheduled repairs and equipment replacement without stopping services - invisibly to customers.
Redundancy of all engineering subsystems (Concurrent Maintainability) significantly increases the reliability level of the data center. In order to achieve this level, it is necessary to design and build two pipelines for the cooling system, to ensure that all equipment operates with reserve capacities, taking into account failure or prevention of the power supply system. In this data center there are several ways to distribute power and cooling, but only one of them is active. In the Tier III data center, components are reserved according to the “N + 1” scheme. As a result, the failover ratio is 99.982%, which translates into a possible downtime of 1.6 hours per year.
Data Processing Centers are designed to ensure the smooth operation of all critical IT processes of the company. Underestimating and reevaluating the impact of downtime negatively affects costs, because financial losses are possible in both cases - either when the critical systems are idle and the business stops, or because of unjustified overpayments for the service.
The Tier I level, according to the Uptime Institute, is a dedicated data center infrastructure without redundancy. Usually this is one or two air conditioners and a UPS without a diesel engine. If something breaks or if there is no electricity, this is an emergency. Any service is easy to provide. The issue price is about $ 10,000 in capital costs per rack of 5 kW.
Tier II provides primary redundancy, that is, power and cooling systems have redundant components and capacities. Typically, these are several air conditioners in N + 1 mode, one industrial UPS and possibly a diesel. Electricity is supplied through the ATS from 1-2 inputs. Premises - a room or a hangar, usually on lease. Maintenance is often difficult or poses a risk of accident even during routine maintenance. The issue price is about $ 20,000 of capital costs per rack 5 kW.
Tier III means redundancy of all engineering subsystems, the ability to carry out any operations, scheduled repairs and replace equipment without stopping services. This means N + 2 for air conditioners, 2 inputs per rack from different UPSs, a reserve for diesels, two inputs and a specially prepared room in the property. The issue price is about $ 30,000 of capital costs per rack 5 kW excluding real estate prices.
Finally, Tier IV means fault tolerance, the ability to withstand a single unplanned emergency event, such as a fire or explosion.
If the data center of the first level does not provide redundancy, then the second adds redundancy for the main engineering systems, but not for the “delivery routes” of the service — electricity and data transmission network channels. Reservation of the latter is implemented in the data center of the third level, which allows for routine maintenance without stopping it. The fourth level involves redundancy (N + 1 and above) of all major systems and fault tolerance in case of the worst event. The average estimated annual downtime of the data center varies from 28.8 hours for the first level to 0.8 hours for the data centers of the fourth level, and the actually observed availability factor is from 99.67 to 99.99%.
Thus, in a data center with a Tier III reliability level, it is possible to simultaneously operate and maintain a data center, up to replacing system components and equipment failure, adding systems. This makes it possible to carry out scheduled repairs and equipment replacement without stopping services - invisibly to customers.
Redundancy of all engineering subsystems (Concurrent Maintainability) significantly increases the reliability level of the data center. In order to achieve this level, it is necessary to design and build two pipelines for the cooling system, to ensure that all equipment operates with reserve capacities, taking into account failure or prevention of the power supply system. In this data center there are several ways to distribute power and cooling, but only one of them is active. In the Tier III data center, components are reserved according to the “N + 1” scheme. As a result, the failover ratio is 99.982%, which translates into a possible downtime of 1.6 hours per year.
In 2015, DataPro Data Center at Aviamotornaya was awarded the Best Project of the Year Award. In September, the first professional awards ceremony in the field of data center construction took place in Russia. The competition program was attended by 19 data centers, claiming prizes in five categories. The international jury recognized DataPro's data center as the largest and most innovative and awarded DataPro the main prize of the Russian Data Center Awards 2015 .
More about the project and plans
DataPro Data Center on Aviamotornaya Street in Moscow, commissioned in 2014. The implementation of this ambitious project from entering the site to placing the first customer’s equipment in the machine room took only eight months. There was no need to build a building. The former building of Avtoframos enterprise and the adjacent territory were converted under the data center.
Despite the very difficult market conditions, due to the flexibility of the strategy, DataPro managed to reach a high occupancy rate in less than a year. The first phase of the DataPro data center with 850 racks (eight machine rooms) is already 80% full. Now the start of the construction of the second stage of 500 racks. The potential capacity of the object is 3000 racks. The plans include the construction of another data center in Moscow with 1000 racks.
The participation of an experienced design team made it possible to opt for modular architecture along with the use of some of the most modern and energy-efficient engineering solutions in the field of data center construction. In Russia, there are now only three or four such sites.
DataPro customers rent 1 to 100 racks. An important task for owners of commercial data centers is to learn how to work with large high-tech customers. One such DataPro anchor client is Mail.Ru Group. This company adheres to the strategy of renting commercial data centers, rather than building its own - it focuses on its own business goals, without getting into a complex and expensive infrastructure. Another well-known client is RBC, renting several dozen stays. There are also a significant number of well-known international companies of the Samsung, Chronopay, etc. level among customers. There are also state customers, but the share of such customers in the data center budget is still small.
Despite the very difficult market conditions, due to the flexibility of the strategy, DataPro managed to reach a high occupancy rate in less than a year. The first phase of the DataPro data center with 850 racks (eight machine rooms) is already 80% full. Now the start of the construction of the second stage of 500 racks. The potential capacity of the object is 3000 racks. The plans include the construction of another data center in Moscow with 1000 racks.
The participation of an experienced design team made it possible to opt for modular architecture along with the use of some of the most modern and energy-efficient engineering solutions in the field of data center construction. In Russia, there are now only three or four such sites.
DataPro customers rent 1 to 100 racks. An important task for owners of commercial data centers is to learn how to work with large high-tech customers. One such DataPro anchor client is Mail.Ru Group. This company adheres to the strategy of renting commercial data centers, rather than building its own - it focuses on its own business goals, without getting into a complex and expensive infrastructure. Another well-known client is RBC, renting several dozen stays. There are also a significant number of well-known international companies of the Samsung, Chronopay, etc. level among customers. There are also state customers, but the share of such customers in the data center budget is still small.
Why is it worth placing equipment in a certified Tier III data center?
To solve most problems - this is the only option in which you do not need to look for some compromises.
Uptime Institute certificates cannot be purchased. All technologies and solutions used are certified, this guarantees maximum data center resiliency to ensure the continuity of customers' business. All this leads to guaranteed availability of resources and is insurance against unexpected situations.
Ultimately, the cost of placing equipment in a data center of this level may be slightly higher. What does the user pay for?
The cost is determined for a reason, the user pays not for “status”, but for confidence in the availability of resources, confirmed by professionals.
Alternative options and what value is formed of
Building your own data centers is only suitable for large-scale or non-standard projects. Often, companies and organizations try to build their own server and data centers, but only in very rare cases can such projects come closer to competently designed and, moreover, certified commercial data centers in their technical level. The operation of any own site with less than 40-50 racks is usually considered economically inexpedient. The cost savings for such a project compared with the placement of equipment in a commercial data center are achieved at a scale of 100 racks, since operating costs are high for a shift on duty, security, equipment depreciation, utility bills, etc.
Accordingly, in this case, you have to spend a lot of time and money on immersion in data center construction, design and implementation. At the same time, the presence of such experience is mandatory, since in the future there may be many "pitfalls" and "rake". Making such a decision, the company risks quite a bit to get an expensive and moody toy.
Placement of equipment in an uncertified data center
This option is most often chosen in a limited budget. But, in order to choose the best option, you also have to become an expert in data center construction, as to evaluate the infrastructure of the data center and accommodation conditions will have to do it yourself.
Placing equipment in a certified Tier III DataPro data center will help ensure that life support systems are fault-tolerant, and the possibility of force majeure tends to zero.
One of the main problems of IT outsourcing in our country is the problem of trust in a service provider. When choosing a data center, you need to pay attention not only to the certification of the data center, but also to the transparency of the outsourcer’s business, his direct ownership of the building, permission for the allocated electricity capacity, long-term registration of the adjacent territory, and the possibility of further development of the site. DataPro meets these requirements. The building of the data center is owned by DataPro - customers are protected from many surprises, since no one is safe from force majeure situations, it is quite common when the owners of the building or city authorities evicted the data center on the street with a 1-2-month notice.
For the financial sector. The demand for Tier III data center services is facilitated by the availability in Russia of relevant clear regulatory rules for financial institutions. At the same time, the number of false statements by companies that they have a data center of “level” Tier III is increasing. Sometimes they talk about "compatibility" with the requirements of the certificate of international reliability level Tier III. After all, certification of a data center is becoming a competitive advantage.
What is the sum of this?
Designing
Everything is clear here, the higher the competence and experience of developers, the higher the cost of the project.
Construction itself.
It all depends on the initial project - the value of real estate, the engineering systems used, and the proposed reservation, etc.
If we take the estimated total costs for the purchase of engineering systems elements and their installation, it turns out that, without taking into account real estate, commissioning one rack of 5 kW costs:
Operation
This includes the costs of utility bills, payroll, depreciation of equipment, maintenance, etc. It is logical that the higher the reliability category of the data center, the higher the operating costs. Indeed, for example, third-party certified specialists are most often involved in carrying out routine maintenance. It is also worth noting that a significant factor affecting operating costs is the scale of the project - the larger the project, the cheaper the cost of maintenance per rack (all other things being equal).
Certification
It would seem, think, get a certificate. But in practice, this is a very long and laborious process.
The process of assessing the quality of a project by Uptime Institute begins with an analysis of the documentation, which serves as the basis for obtaining a certificate for a design project. At this stage, project documentation is submitted to Uptime Institute. It should provide answers to specific questions that relate to all engineering systems, including buildings and layouts, topologies for the location of telecommunication and engineering networks, equipment operating modes, etc.
The process of obtaining a Uptime Facility certificate is a little on the shoulder - it is expensive and takes several months of hard work. It is required to test the data center under full load, for which a "ballast" is connected - resistive load modules equivalent to the load of the required power, in units of 500 kW and above. Heat guns are used to test engine rooms. The failure of one of the cooling and power supply systems is simulated. It is checked how everything works at full load, how switching under load occurs. Such serious testing helps identify “bottlenecks” in advance, even before the data center is put into commercial operation.
Accordingly, in this case, you have to spend a lot of time and money on immersion in data center construction, design and implementation. At the same time, the presence of such experience is mandatory, since in the future there may be many "pitfalls" and "rake". Making such a decision, the company risks quite a bit to get an expensive and moody toy.
Placement of equipment in an uncertified data center
This option is most often chosen in a limited budget. But, in order to choose the best option, you also have to become an expert in data center construction, as to evaluate the infrastructure of the data center and accommodation conditions will have to do it yourself.
Placing equipment in a certified Tier III DataPro data center will help ensure that life support systems are fault-tolerant, and the possibility of force majeure tends to zero.
One of the main problems of IT outsourcing in our country is the problem of trust in a service provider. When choosing a data center, you need to pay attention not only to the certification of the data center, but also to the transparency of the outsourcer’s business, his direct ownership of the building, permission for the allocated electricity capacity, long-term registration of the adjacent territory, and the possibility of further development of the site. DataPro meets these requirements. The building of the data center is owned by DataPro - customers are protected from many surprises, since no one is safe from force majeure situations, it is quite common when the owners of the building or city authorities evicted the data center on the street with a 1-2-month notice.
For the financial sector. The demand for Tier III data center services is facilitated by the availability in Russia of relevant clear regulatory rules for financial institutions. At the same time, the number of false statements by companies that they have a data center of “level” Tier III is increasing. Sometimes they talk about "compatibility" with the requirements of the certificate of international reliability level Tier III. After all, certification of a data center is becoming a competitive advantage.
What is the sum of this?
Designing
Everything is clear here, the higher the competence and experience of developers, the higher the cost of the project.
Construction itself.
It all depends on the initial project - the value of real estate, the engineering systems used, and the proposed reservation, etc.
If we take the estimated total costs for the purchase of engineering systems elements and their installation, it turns out that, without taking into account real estate, commissioning one rack of 5 kW costs:
- For Tier I data centers - about $ 10,000
- For data center Tier II - about $ 20,000
- For Tier III data centers, around $ 30,000
Operation
This includes the costs of utility bills, payroll, depreciation of equipment, maintenance, etc. It is logical that the higher the reliability category of the data center, the higher the operating costs. Indeed, for example, third-party certified specialists are most often involved in carrying out routine maintenance. It is also worth noting that a significant factor affecting operating costs is the scale of the project - the larger the project, the cheaper the cost of maintenance per rack (all other things being equal).
Certification
It would seem, think, get a certificate. But in practice, this is a very long and laborious process.
The process of assessing the quality of a project by Uptime Institute begins with an analysis of the documentation, which serves as the basis for obtaining a certificate for a design project. At this stage, project documentation is submitted to Uptime Institute. It should provide answers to specific questions that relate to all engineering systems, including buildings and layouts, topologies for the location of telecommunication and engineering networks, equipment operating modes, etc.
The process of obtaining a Uptime Facility certificate is a little on the shoulder - it is expensive and takes several months of hard work. It is required to test the data center under full load, for which a "ballast" is connected - resistive load modules equivalent to the load of the required power, in units of 500 kW and above. Heat guns are used to test engine rooms. The failure of one of the cooling and power supply systems is simulated. It is checked how everything works at full load, how switching under load occurs. Such serious testing helps identify “bottlenecks” in advance, even before the data center is put into commercial operation.
DataPro Infrastructure
In the subsequent materials we will tell in detail about all life support systems of this data center. And today, we will briefly consider the main elements of the infrastructure and talk in detail about power supply.
The average size of the machine room in the DataPro data center is 180 square meters. m, it is designed for about 100 racks or can be divided into several partitions. There are eight such halls in total. The building provides space for technical personnel, equipment storage, loading and unloading, catering. Clients can not only rent hard-seats, but also create a backup office, where the client’s critical staff will continue to work in case of emergency.
For cooling server rooms and auxiliary technological rooms of the data center, 8 Ecobreeze installations are now usedmanufactured by Schneider Electric. The temperature regime in the halls is maintained at + 25 ° C with a humidity of 40-60%. The use of this modular cooling system in combination with other engineering solutions allowed us to achieve a high PUE, the total average annual value of which does not exceed 1.3. In the next article we will talk about Ecobreeze in more detail.
Fiber optic lines were laid to the data center via two independent routes, connecting it with all the largest telecommunication communication centers and traffic exchange points in Moscow. The technology platform’s policy is neutral with respect to telecom operators; therefore, more than 10 telecom operators are present at the site (Garstelecom, Mastertel, RETN, Macomnet, MTS, Beeline, Orange Business Services, etc.).
Considerable attention is paid to security: security is responsible for the security of the data center itself and the surrounding area. The security system includes more than 350 network cameras. The data center also provides advanced methods for preventing fire: the Vesda early smoke detection system and the automatic gas fire extinguishing system based on HFC 227.
Power Supply Data Center
Power supply is the basis of reliable data center operation. It must be uninterrupted, for which uninterruptible power supplies (UPS) are traditionally used, and guaranteed, for which diesel generator sets (DGU) are used, which support the operation of the data center during a prolonged accident in the power supply network.
In the DataPro data center, two independent 10 kV inputs from the Mosenergo substation, and the cables are laid in different collectors, provide the necessary electrical power to the facility. Consumer power is provided through a group of step-down transformers. These dry transformers are constantly in operation when there is power from the city substation.
Now the authorized and connected capacity of the data center is 6 MW. Another 8 MW of power is being processed. The power supply of the data center is actually reserved according to the 2N scheme. For example, even at 100% load in the event of an accident, the power board, transformer, DRUPS, the entire machine building will be powered by a second input / shield. Customers are supposed to use equipment with redundant power supplies or rack-mountable ABPs.
Here are the main components of the data center power system. On the right are transformers, on the left are diesel-rotor UPSs (DRUPS).
Dry power transformers with a capacity of 1600 kVA each are used to power the data center. They convert 10 kV at the input to 0.4 kV at the output.
Part of a data center power system diagram.
Diesel rotary ups
Traditionally, when the voltage of the city network disappears, the batteries of the UPS must maintain power to the equipment until the DGU is fully launched, which can provide power to the entire data center. This process usually takes from 20-30 seconds to 2 minutes, for which the UPS power and battery capacity are calculated.
The DataPro data center, at its scale, uses a different, more economical approach. The main components of the DataPro data center infrastructure are power solutions for diesel-rotor UPSs manufactured by Hitec Power Protection (currently four units) and Euro-Diesel (two machines). A service contract with two different vendors allows you to optimize prices and compare service.
In fact, the diesel-rotor or dynamic UPS (DRUPS) of these manufacturers is a diesel engine, an electromagnetic clutch, a synchronous electric machine and a dynamic flywheel assembled coaxially on the same shaft. A decoupling choke connects the input of the mains voltage to the output of the “clean” voltage of the UPS system. It provides a high degree of isolation between input and output, blocking the passage of harmonics and transients.
In the presence of external power, the synchronous electric machine of the UPS operates in electric motor mode by rotating the flywheel rotor. When the network parameters go beyond the set limits, it goes into generator mode, maintaining a continuous sinusoidal voltage at the load. The source of energy during the transition process is the flywheel, due to which a stable rotational speed of the rotor shaft of a synchronous electric machine is maintained. The flywheel holds revolutions up to one minute at full electric load without a diesel engine.
After a power failure, the DRUPS control system sends a signal to start the diesel engine. The kinetic energy of the flywheel, in addition to maintaining the output voltage, is used to output the diesel engine crankshaft to the operating frequency. 50 ms after a power failure, the diesel engine starts to work, which within a few seconds goes to normal operation - the flywheel is already spinning.
Autonomous power supply of the facility in case of power failure from the substation is provided by Hitec Power Protection 1670 kVA diesel-rotor UPSs with a 4 / 3N redundancy circuit. And no batteries to power the IT load.
Power cables and ventilation duct.
Once a month, diesel engines are launched to check their performance. The estimated stock of diesel fuel in the storage is 17 hours, taking into account the incomplete load of the halls this time is much longer - up to 60 hours. Further fuel will be supplied from Moscow and, if necessary, from Ryazan. DataPro has concluded relevant contracts with its suppliers.
For storage of fuel, two containers of 60 tons are used, each of them is half full. It is also made for redundancy: in case of an accident or repair of fuel lines and pumps.
The Dutch company Hitec Power Protection is one of the leading manufacturers of uninterruptible power solutions based on diesel-rotary UPSs. It was she who first patented the idea of a UPS with a flywheel. Hitec Power Protection's dynamic UPS is currently protected by a fleet of equipment with a total capacity of more than 1.6 million kVA.
In Russian data centers, DRUPS are still rare, however, in the world, systems based on diesel-rotor UPSs, due to their simplicity, reliability and energy efficiency, are considered a good solution for powering data centers of high power (from 1 MW). A relatively small number of components and simplicity of design significantly increase the average uptime of this equipment. So at HitecThe total service life of dynamic UPSs is at least 25 years. At the same time, depending on the amount of available fuel, diesel-rotary UPSs provide backup power for any period of time.
A Hitec installation includes: (1) a diesel engine, (2) an overrunning clutch, (3) a generator, (4) an energy storage device, (5) a flywheel. DRUPS Hitec are optimized for energy efficiency, and their claimed input-output efficiency is 97%.
High energy efficiency of dynamic UPSs is achieved due to the absence of the need for multiple voltage conversion - rectifiers, inverters, etc. There is no need for expensive and dangerous batteries, and therefore, in a special room for their placement, in the organization of its conditioning and, most importantly, in their regular replacement every 3-5 years.
Diesel-rotary UPSs are considered the most economical and energy-efficient solution to ensure uninterrupted and guaranteed power supply to large data centers. Until recently, according to the research company Frost & Sullivan, rotary UPSs occupied no more than 6% of the global UPS market. However, according to forecasts, their share should increase significantly as the size of the data center increases.
Distinctive features of DRUPS.
For a diesel-rotary UPS, an average of 40-60% less area is required than traditional static UPSs with similar characteristics, taking into account DGS. A single DRUPS unit is used instead of a system of backup DGUs, static UPSs, battery cabinets, switchboards, air conditioning systems, etc. The data center area is used in an optimal way.
Thanks to DRUPS, virtually all power quality problems are controlled. The installation provides filtering of all network interference and load, feeds capacitive and inductive loads, while the power factor is 0.98.
Electric power from the city network, passing through the inductor, feeds the synchronous motor-generator. The alternator shaft rotates at 1,500 rpm. In normal operation, the synchronous generator performs the function of a stabilizer and a voltage filter: eliminates harmonic distortion and power outages, impulse noise, eliminates surges and voltage dips. Thanks to him, the frequency and amplitude of the output voltage supplied to the load are invariably stable.
Unlike static UPSs, DRUPS is optimally used for powering air conditioners whose compressors are characterized by very high starting currents. DRUPS reduce investment in air conditioning because they do not require tight climate control. This affects the reduction in operating costs and the final cost of rent for consumers.
A dynamic diesel uninterruptible power supply has a high level of system self-diagnosis, allowing you to timely track and prevent malfunctions. When one of the six DRUPS is put out for repair, the data center will continue to function normally, with redundancy. The load is distributed between the rest of the UPS. This was checked even when simulating the full load of the data center. Given the partial load of the halls (usually it is no more than 60% of the design capacity), two DRUPS will be involved at the initial stage of entering the second stage of the data center. Later it is planned to purchase six more DRUPS for the second stage.
DataPro places projects from one rack, and the client should independently agree on cooperation with telecom operators.
Hostkeyis an official retail partner for DataPro. If you need to deploy equipment, rent dedicated servers or implement a complex converged project for high SLA in a Tier III certified DataPro data center, please contact.
Placing 1U equipment in our racks (a modern server with any power supply, not a twin) with a redundant dedicated 100Mbit / s Internet access channel costs 4000 rubles. per month, including VAT or only 40,000 rubles. in year. You can rent a whole rack, 1/2 or a quarter of a standard rack, including a seal. We will help with the installation, connection and preliminary configuration of the transferred equipment.
We offer for rent in DataPro any dedicated Supermicro serversor HP, HP / Dell disk enclosures, and network infrastructure (VLANs) from 100Mbps to 10Gbps (including security features and routers) at reasonable prices. You can create a hybrid solution consisting of your servers, leased servers and network infrastructure. HOSTKEY provides the highest SLA for servicing customers' own equipment.
Especially for readers of this article - refer to it before the end of February 2016 and get an additional 10% discount on the placement or rental of equipment in one of the best data centers in Moscow - DataPro.