Disaster tolerance of a corporate data center as a service
There are many solutions for building disaster-proof systems using technologies for consolidating and virtualizing computing resources, cluster systems, replication technologies and continuous data protection, and a customer can build a disaster-resistant data center either independently or using commercial data center sites and service providers.
Today, almost every company that actively uses IT to support its business has its own data center (DPC). Increasing data center reliability requirements is one of the market trends. Since the data center is often a key element of a company’s business, specialists have long been looking for cost-effective ways to increase its reliability. And sooner or later there is a need to ensure not only the hardware reliability of the data center, but also itsdisaster tolerance .
According to EMC research , 82% of organizations in the world are not completely sure that they will be able to restore their systems and data. Unplanned downtime and data loss cost organizations around the world annually more than $ 1.7 billion. According to Acronis research, in Russia, only 2% of the companies surveyed are absolutely sure that their IT infrastructure will withstand any test. 49% of Russian specialists expect long interruptions in its work in the event of a natural disaster or accident.
According to foreign statistics, the most common causes of accidents are equipment failures (24%), power supply systems (16%), hurricanes (16%) and floods (15%).
In any technically complex systems, accidents are inevitable, but they can be made not critical for business. To prevent such situations, catastrophic systems are created - reserve capacities are deployed at a geographically remote site. But first, let's define the terms that describe high reliability systems.
Disaster recovery (Disaster Recovery, DR) - is the ability to recover from disaster, that is, resistance to natural disasters and acts of terrorism. Fault Tolerance (FT) - the ability of a system to remain operational after a failure of one or more components. About high availability(High Availability, HA) is said when systems are able to perform the required function under given conditions at a given point in time or within a given time interval. Business continuity (Business Continuity, BC) - these are the processes, methods and equipment for non-stop performance of business-critical functions. And finally, RTO (Recovery Time Objective) - the time in which it is possible to restore the IT system, RPO (Recovery Point Objective) - how much data will be lost during disaster recovery, RCO (Recovery Capacity Objective) - what part of the load the backup system should provide .
To protect against natural, man-made disasters or terrorist attacks and ensure the continuity of business processes, backup of the main data storage and processing systems is necessary. In the event of a catastrophe, the building of the data center may suffer, therefore it is necessary to create a geographically remote site - a backup data center. When the Tier III reliability level is not enough, a geographically distributed disaster-tolerant data center infrastructure can guarantee the availability of four or even five nines.
Distribution of data centers across several sites requires the organization of redundant communication channels, data replication between storages, backup planning and system recovery. We need a data synchronization mechanism to ensure their relevance in case of failure of one of the nodes and to support the operation of those information systems that require such synchronization. A critical parameter, in addition to bandwidth, is data transfer delay.
Two main strategies for using distributed data centers are possible - “active / active”, when infrastructure applications and services are distributed between sites, and users work with the nearest data center, or “active / passive”, in which applications are centralized, and users work with the main node. In the event of a system failure, the load automatically switches to the backup data center. The ability to apply a particular strategy depends on the application.
Often at the heart of a disaster-resistant data center is a geographically distributed cluster server configuration with a connection to a common storage area network (SAN). The nodes of the spaced cluster are located on the main and reserve sites, forming a single system. This ensures uninterrupted service availability even in case of loss of the main data center. With the help of clustering, it is possible to provide automatic load switching between the sites of a distributed data center in the event of an accident. An economical modification of the solution is also possible, in which the remote data center operates in standby mode and, in the event of a failure of the main data center, supports a limited set of services.
Inter-site communications solutions may use Ethernet, MPLS, or IP, depending on the distance and architecture. Distances between data centers with synchronous replication can be up to 80-100 km - it is limited by network latency acceptable for applications. With synchronous replication, the application receives confirmation of the completion of the I / O operation after it is completed on both sides. According to FCIP technology, asynchronous interaction between data centers that are thousands of kilometers away from each other can also be arranged through separate switches, and hardware compression of traffic can be used. Compared to the Fiber Channel (FC) protocol, over 100 km away, FCIP is faster. When using FCIP, Fiber Channel packets are encapsulated in TCP / IP and then transmitted through an IP tunnel. FCIP is the main practically working way of data center communication when FC transmission through dark optics or through xWDM is impossible or impractical. It supports both direct connection of FCIP devices to each other, as well as connection via WAN.
A key element of a disaster-proof solution is a geographically distributed data storage system. Modern storage systems provide built-in tools for building disaster-proof solutions. For example, data storage systems at these sites can completely duplicate each other, and the sites themselves are connected by redundant high-speed communication channels, which allows you to implement projects with the highest requirements for data transfer reliability and availability, including synchronous data replication. Or data backup can be carried out in asynchronous mode.
Some storage systems have the ability to “stretch” volumes between sites using the disk array itself. As a result, an inexpensive disaster-resistant solution is created that does not require restructuring the data storage architecture. Another option is to use Microsoft Azure cloud infrastructure to back up data.
A typical scenario is a backup data center in another city within the region (distance - 300-400 km). For LAN communication, IP or MPLS / VPLS, DWDM is used; for communication via SAN - FCIP, DWDM. In this case, you can apply a number of "metrocluster" technologies, use asynchronous replication. Synchronous replication at such a distance requires limitations and additional tools. When the sites are spread out over thousands of kilometers, they are already talking about a “geocluster”.
Clustering methods are offered by suppliers of operating systems, virtualization environments, application developers, manufacturers of IT systems and network equipment. For example, the VMware vSphere-based metro cluster is based on duplication of data storage systems at two geographically separated sites with possible load balancing at the data center network level. If one of the data centers is unavailable, virtual machines will automatically start on the second platform. In this case, the virtual environment recovery speed (RTO) is usually several minutes.
Do not forget that the implementation of the DR strategy requires serious investment. The implementation of such a project is usually associated with high financial costs. It is very difficult to justify and decide on the construction of such a class of systems. Moreover, it is likely that you will never use the backup recovery plan. However, in the event of an emergency, a good recovery plan will save time and money and help minimize loss due to downtime. A serious accident can lead to the loss of a data center, and this is a serious problem for business. According to world statistics, 93% of companies that lost their data center for only 10 days go bankrupt within a year.
It is necessary to find a balance between the costs of maintaining disaster tolerance and the loss of business in the event of a disaster, taking into account the time of complete recovery of all business processes. The illustrations below will help to get some idea of the costs of implementing a distributed data center in a company, more accurately assess the amount of unavoidable costs and avoid possible misunderstanding of managers. In general, the dependence is as follows: the shorter the required recovery time, the more expensive the data protection methods are (according to Gartner):
If we are not just talking about backing up and restoring systems and data, but disaster tolerance, then choosing the optimal solution for any given parameters is it is also always a compromise (according to Compulink ).
Zero performance RTO / RPO has only a high availability system. Of course, this is the most expensive option (according to Cisco ).
Providing disaster tolerance always required significant costs, time and financial. You need to have two separated sites, a fast communication channel between them, a data transmission network, replication-enabled data storage systems, computing power and engineering equipment for uninterrupted power supply and data center cooling. It will require a staff of highly qualified IT professionals who can configure and maintain all this. It is necessary to pay attention to the design of systems for their implementation and testing. However, this task also has solutions without major capital investments.
With the proliferation of virtualization and cloud technology, new ways to protect against disasters have appeared:
According to EMC, which conducted a survey of Russian companies in 2014, only 6% of respondents rely on the active / active operating mode. These companies are less likely to experience data loss than those that rely on backup: 13% versus 24%.
If the company does not have the ability to create DR infrastructure, outsourcing can be a reasonable solution. Such services are quite affordable today. Interestingly, according to a recently published study by Computer Economics IT Outsourcing Statistics 2015/2016 , disaster recovery has the highest potential for cost savings among the most common types of outsourcing. It was noted by 92% of organizations surveyed.
Instead of creating its own backup site, the organization can use the provider's virtual (cloud) data center or completely abandon its own data center and switch to the cloud model. This option is suitable for many organizations. This modern approach to ensuring disaster tolerance is called “backup data center as a service” (Disaster Recovery as a Service, DRaaS). DRaaS eliminates the impact of accidents on business processes, ensures uninterrupted operation, and also removes many issues of a logistical and organizational nature from the client.
Reliability of the provider's services is provided by two (or more) geographically remote data centers, which are specialized buildings with a high level of reliability. In the case of a fully virtual data center, each of them has a copy of the client’s virtual data center — primary and backup. All changes in the primary instance in real time are reflected in the backup. Failure of any of the copies will not affect the work of the organization. When an accident occurs, instead of the main data center, a backup is instantly connected, and all employees and customers continue to work as usual. According to the OSP Data survey, more than half (54%) of Russian respondents consider it important that the service provider has several geographically remote data centers to ensure disaster tolerance.
One example of the approach described above is BDS (Backup Data Center) or “ Backup Data Center Package” of SAFEDATA. In fact, it is not limited to DRaaS. This is a whole range of design and backup site services for the main customer data center. The physical data center located on the customer’s site and the virtual IT infrastructure can act as the main data center of the customer.
SAFEDATA has its own distributed data center network in Moscow, connected by its own fiber-optic communication lines. This allows us to provide customers with not only services for the creation and placement of a backup site, but also the placement of the main data center, distributed between two remote sites.
BDC services may include the placement of equipment and virtual computing resources, the provision of fiber-optic communication lines and L2 channels, data synchronization between two sites, the provision of Internet channels with guaranteed bandwidth, protection against DDoS attacks and backup.
Choosing the SAFEDATA data center as a platform for creating a backup data center, the customer gets access to the expertise in the field of design, construction and maintenance of data centers allocated to the round-the-clock technical support service. Rent of office and warehouse premises is also possible.
Catastrophic solutions and services today are offered a lot. Please share in the comments what you use, how these solutions helped you out.
Today, almost every company that actively uses IT to support its business has its own data center (DPC). Increasing data center reliability requirements is one of the market trends. Since the data center is often a key element of a company’s business, specialists have long been looking for cost-effective ways to increase its reliability. And sooner or later there is a need to ensure not only the hardware reliability of the data center, but also itsdisaster tolerance .
According to EMC research , 82% of organizations in the world are not completely sure that they will be able to restore their systems and data. Unplanned downtime and data loss cost organizations around the world annually more than $ 1.7 billion. According to Acronis research, in Russia, only 2% of the companies surveyed are absolutely sure that their IT infrastructure will withstand any test. 49% of Russian specialists expect long interruptions in its work in the event of a natural disaster or accident.
According to foreign statistics, the most common causes of accidents are equipment failures (24%), power supply systems (16%), hurricanes (16%) and floods (15%).
In any technically complex systems, accidents are inevitable, but they can be made not critical for business. To prevent such situations, catastrophic systems are created - reserve capacities are deployed at a geographically remote site. But first, let's define the terms that describe high reliability systems.
Disaster recovery (Disaster Recovery, DR) - is the ability to recover from disaster, that is, resistance to natural disasters and acts of terrorism. Fault Tolerance (FT) - the ability of a system to remain operational after a failure of one or more components. About high availability(High Availability, HA) is said when systems are able to perform the required function under given conditions at a given point in time or within a given time interval. Business continuity (Business Continuity, BC) - these are the processes, methods and equipment for non-stop performance of business-critical functions. And finally, RTO (Recovery Time Objective) - the time in which it is possible to restore the IT system, RPO (Recovery Point Objective) - how much data will be lost during disaster recovery, RCO (Recovery Capacity Objective) - what part of the load the backup system should provide .
Catastrophic Data Center
To protect against natural, man-made disasters or terrorist attacks and ensure the continuity of business processes, backup of the main data storage and processing systems is necessary. In the event of a catastrophe, the building of the data center may suffer, therefore it is necessary to create a geographically remote site - a backup data center. When the Tier III reliability level is not enough, a geographically distributed disaster-tolerant data center infrastructure can guarantee the availability of four or even five nines.
Distribution of data centers across several sites requires the organization of redundant communication channels, data replication between storages, backup planning and system recovery. We need a data synchronization mechanism to ensure their relevance in case of failure of one of the nodes and to support the operation of those information systems that require such synchronization. A critical parameter, in addition to bandwidth, is data transfer delay.
Two main strategies for using distributed data centers are possible - “active / active”, when infrastructure applications and services are distributed between sites, and users work with the nearest data center, or “active / passive”, in which applications are centralized, and users work with the main node. In the event of a system failure, the load automatically switches to the backup data center. The ability to apply a particular strategy depends on the application.
Often at the heart of a disaster-resistant data center is a geographically distributed cluster server configuration with a connection to a common storage area network (SAN). The nodes of the spaced cluster are located on the main and reserve sites, forming a single system. This ensures uninterrupted service availability even in case of loss of the main data center. With the help of clustering, it is possible to provide automatic load switching between the sites of a distributed data center in the event of an accident. An economical modification of the solution is also possible, in which the remote data center operates in standby mode and, in the event of a failure of the main data center, supports a limited set of services.
Inter-site communications solutions may use Ethernet, MPLS, or IP, depending on the distance and architecture. Distances between data centers with synchronous replication can be up to 80-100 km - it is limited by network latency acceptable for applications. With synchronous replication, the application receives confirmation of the completion of the I / O operation after it is completed on both sides. According to FCIP technology, asynchronous interaction between data centers that are thousands of kilometers away from each other can also be arranged through separate switches, and hardware compression of traffic can be used. Compared to the Fiber Channel (FC) protocol, over 100 km away, FCIP is faster. When using FCIP, Fiber Channel packets are encapsulated in TCP / IP and then transmitted through an IP tunnel. FCIP is the main practically working way of data center communication when FC transmission through dark optics or through xWDM is impossible or impractical. It supports both direct connection of FCIP devices to each other, as well as connection via WAN.
A key element of a disaster-proof solution is a geographically distributed data storage system. Modern storage systems provide built-in tools for building disaster-proof solutions. For example, data storage systems at these sites can completely duplicate each other, and the sites themselves are connected by redundant high-speed communication channels, which allows you to implement projects with the highest requirements for data transfer reliability and availability, including synchronous data replication. Or data backup can be carried out in asynchronous mode.
Some storage systems have the ability to “stretch” volumes between sites using the disk array itself. As a result, an inexpensive disaster-resistant solution is created that does not require restructuring the data storage architecture. Another option is to use Microsoft Azure cloud infrastructure to back up data.
A typical scenario is a backup data center in another city within the region (distance - 300-400 km). For LAN communication, IP or MPLS / VPLS, DWDM is used; for communication via SAN - FCIP, DWDM. In this case, you can apply a number of "metrocluster" technologies, use asynchronous replication. Synchronous replication at such a distance requires limitations and additional tools. When the sites are spread out over thousands of kilometers, they are already talking about a “geocluster”.
Clustering methods are offered by suppliers of operating systems, virtualization environments, application developers, manufacturers of IT systems and network equipment. For example, the VMware vSphere-based metro cluster is based on duplication of data storage systems at two geographically separated sites with possible load balancing at the data center network level. If one of the data centers is unavailable, virtual machines will automatically start on the second platform. In this case, the virtual environment recovery speed (RTO) is usually several minutes.
Catastrophic Economics
Do not forget that the implementation of the DR strategy requires serious investment. The implementation of such a project is usually associated with high financial costs. It is very difficult to justify and decide on the construction of such a class of systems. Moreover, it is likely that you will never use the backup recovery plan. However, in the event of an emergency, a good recovery plan will save time and money and help minimize loss due to downtime. A serious accident can lead to the loss of a data center, and this is a serious problem for business. According to world statistics, 93% of companies that lost their data center for only 10 days go bankrupt within a year.
It is necessary to find a balance between the costs of maintaining disaster tolerance and the loss of business in the event of a disaster, taking into account the time of complete recovery of all business processes. The illustrations below will help to get some idea of the costs of implementing a distributed data center in a company, more accurately assess the amount of unavoidable costs and avoid possible misunderstanding of managers. In general, the dependence is as follows: the shorter the required recovery time, the more expensive the data protection methods are (according to Gartner):
If we are not just talking about backing up and restoring systems and data, but disaster tolerance, then choosing the optimal solution for any given parameters is it is also always a compromise (according to Compulink ).
Zero performance RTO / RPO has only a high availability system. Of course, this is the most expensive option (according to Cisco ).
Providing disaster tolerance always required significant costs, time and financial. You need to have two separated sites, a fast communication channel between them, a data transmission network, replication-enabled data storage systems, computing power and engineering equipment for uninterrupted power supply and data center cooling. It will require a staff of highly qualified IT professionals who can configure and maintain all this. It is necessary to pay attention to the design of systems for their implementation and testing. However, this task also has solutions without major capital investments.
Virtualization, clouds and disaster recovery
With the proliferation of virtualization and cloud technology, new ways to protect against disasters have appeared:
- Replication to the cloud. Private and public cloud technologies have facilitated cross-site replication. The replication process can cover all virtual machines, specific databases or data snapshots. In addition, cloud technologies help organizations choose the most suitable DR option for financial conditions - there is a flexibility in choosing the acceptable downtime. That is, you can often choose an acceptable downtime and at the same time fit into the budget.
- Virtualization as a backup / restore mechanism. Here the idea is simple: it is much easier to restore a virtual machine than a physical server. You can save VM snapshots to the backup data center or mirror virtual machines. In the latter case, the configuration is “active / active” - in the event of a failure on the main site of the VM performing critical tasks, it switches to the same VM on the backup site.
- Using software configuration technologies (Software Defined, SD). In fact, this is the development of virtualization. Software-configurable platforms (network equipment, storage systems, security device, load balancers, etc.) allow you to get a flexible fault-tolerant environment with "virtual devices" for various purposes, functioning as virtual machines on standard servers. For example, if you use DR load balancing mechanisms (Global Server Load Balancing, GSLB), you can automatically switch users to the backup site in the event of a primary failure. For users, the process will be transparent.
- IaaS (infrastructure on demand). Cloud platforms and virtualization environments enable you to quickly provision the necessary IT resources. For DR, the ability to quickly restore virtual machines and data is important. Cloud computing and virtualization are great for this. You can create very cost-effective IaaS solutions — active / active or active / passive. For example, a regular backup of the VM and data is set in the provider's data center. In the event of an accident, a new environment is deployed - VMs are started with their backup data. The process is not instant, but fast enough. In IaaS, the main thing is flexibility. When implementing the DR strategy, the provider will help the customer to get the most out of this flexibility.
According to EMC, which conducted a survey of Russian companies in 2014, only 6% of respondents rely on the active / active operating mode. These companies are less likely to experience data loss than those that rely on backup: 13% versus 24%.
If the company does not have the ability to create DR infrastructure, outsourcing can be a reasonable solution. Such services are quite affordable today. Interestingly, according to a recently published study by Computer Economics IT Outsourcing Statistics 2015/2016 , disaster recovery has the highest potential for cost savings among the most common types of outsourcing. It was noted by 92% of organizations surveyed.
Reserve data center as a service
Instead of creating its own backup site, the organization can use the provider's virtual (cloud) data center or completely abandon its own data center and switch to the cloud model. This option is suitable for many organizations. This modern approach to ensuring disaster tolerance is called “backup data center as a service” (Disaster Recovery as a Service, DRaaS). DRaaS eliminates the impact of accidents on business processes, ensures uninterrupted operation, and also removes many issues of a logistical and organizational nature from the client.Reliability of the provider's services is provided by two (or more) geographically remote data centers, which are specialized buildings with a high level of reliability. In the case of a fully virtual data center, each of them has a copy of the client’s virtual data center — primary and backup. All changes in the primary instance in real time are reflected in the backup. Failure of any of the copies will not affect the work of the organization. When an accident occurs, instead of the main data center, a backup is instantly connected, and all employees and customers continue to work as usual. According to the OSP Data survey, more than half (54%) of Russian respondents consider it important that the service provider has several geographically remote data centers to ensure disaster tolerance.
One example of the approach described above is BDS (Backup Data Center) or “ Backup Data Center Package” of SAFEDATA. In fact, it is not limited to DRaaS. This is a whole range of design and backup site services for the main customer data center. The physical data center located on the customer’s site and the virtual IT infrastructure can act as the main data center of the customer.
SAFEDATA has its own distributed data center network in Moscow, connected by its own fiber-optic communication lines. This allows us to provide customers with not only services for the creation and placement of a backup site, but also the placement of the main data center, distributed between two remote sites.
BDC services may include the placement of equipment and virtual computing resources, the provision of fiber-optic communication lines and L2 channels, data synchronization between two sites, the provision of Internet channels with guaranteed bandwidth, protection against DDoS attacks and backup.
Choosing the SAFEDATA data center as a platform for creating a backup data center, the customer gets access to the expertise in the field of design, construction and maintenance of data centers allocated to the round-the-clock technical support service. Rent of office and warehouse premises is also possible.
Catastrophic solutions and services today are offered a lot. Please share in the comments what you use, how these solutions helped you out.
