The future for flash: IBM FlashSystem
Time is a key factor in business, especially in mission-critical business applications. Those who saw the movie Apollo 13, or older people who remember this unfortunate space flight, know what happens when critical systems fail. According to the plot of the film, three American astronauts go to the moon, but on the way the spacecraft explodes and receives serious damage, and they only miraculously survive. The astronauts manage to return home only thanks to the heroic efforts and a bit of luck. Fortunately, the consequences of the failure of critical business applications are rarely so serious (compared to, say, applications that depend on the work of services and institutions, for example, in intensive care units, air traffic control services, etc.).
Applications critical for business include applications for a wide variety of tasks, for example: business intelligence (BI - business intelligence), online transaction processing (OLTP), online analytical processing (OLAP), virtual work infrastructure desks, high-performance computing systems and content delivery applications (cloud storage systems, on-demand video delivery, etc.). All these important tasks are united by one thing: high requirements for speed of work, because it depends on how quickly management, employees, customers and other key business partners get access to the necessary information.
Failure to quickly access and process sensitive data leads to significant business and financial risks. In particular, such delays cause:
- Inability to timely and correctly assess the situation - without access to critical data at the right time and in the right place, people do not see the full picture of what is happening and, as a result, make the wrong decisions, making completely unnecessary mistakes.
- A sharp decrease in the ability to predict business results - the quality of financial forecasts at the level of an individual project, a separate area of activity and the whole company depends on the ability to receive the most accurate data in a timely manner. Without access to such data, the company cannot analyze them, which negatively affects its activities.
- Growing customer dissatisfaction - in many sectors (financial services, healthcare, tourism, etc.) the quality of service directly depends on the availability of instant access to relevant information, especially since a modern client expects service 24 hours a day, 7 days in week and 365 days a year. Under these conditions, the inability to quickly provide the necessary information to customers not only means lost profits, but also often leads to loss of the client, or even market share.
- Too large volumes, variety and speed of information flow - explosive growth in data volumes leads to the inevitable decline in the performance of the main applications in many enterprises. The increase in delays is becoming a source of user dissatisfaction, and enterprises are missing out on opportunities.
Heads of IT departments looking for a solution to these problems that would have the best combination of functionality, implementation time, scalability and cost of systems, you will not envy. Ultimately, the right decision may not be obvious.
Statement of the problem (hint: processor speed is the wrong answer)
As a rule, the IT department primarily responds to system slowdowns by adding or upgrading processors and memory. However, too often, the performance gain from this upgrade is minimal, while the cost is high. The second method, often associated with the first, at least for databases, is to optimize SQL statements. Enterprises spend millions of dollars on it and even get some results. But they overlook the fact that the most ideal database will be powerless against the low speed of the data storage system, namely the latter is often the source of the problem.
Indeed, the root cause of application performance problems often lies in the storage system, rather than processor speed or lack of memory. When the processor is idle waiting for information from the storage system, valuable time and resources are wasted. And given the fact that over the past 20 years the processing power of processors has grown exponentially, this downtime is becoming more and more. Simply put, the computing resources of modern processors far exceed the speed with which hard drives can deliver data to them. This performance gap is especially acute in database servers, where the number of I / O operations is usually much larger than in other systems. As a result, ultra-fast processors and huge bandwidth are often not fully utilized, because accessing data on hard drives takes several milliseconds, which for modern processors is an eternity. And when servers wait a long time for data from storage systems, users wait a long time for a response from servers. This is where the real source of the problem with low I / O operations.
This problem is due to purely physical causes. Data movement on solid-state drives and through the network occurs at a speed comparable to the speed of the current in the electric circuit, i.e., in fact, the speed of light, while hard drives are mechanical devices that read information by rotating the plate. Thus, with all its reliability and performance, hard drives in terms of access speed are by far the weakest link in the information transfer chain.
Disadvantages of Traditional Workarounds for HDD Speed Issues
They try to solve performance problems due to the inefficiency of hard drives in several ways, in particular, by creating JBOD arrays (just a bunch of disks - just a set of disks) or RAID (redundant array of independent disks - redundant array of independent disks). With the increase in the number of disks, it becomes possible to distribute the I / O operations carried out by the database among several disks. Unfortunately, this gives a too modest increase in HDD performance.
It is also customary to move frequently used files to a separate drive. In this case, the I / O speed of one disk really grows to its maximum, but even at maximum this speed is still low. In the best case, one hard drive produces ~ 200, without the use of additional functionality, input / output operations per second (IOPS - input / output operations per second), which is several times lower than the speed that could reduce the performance gap between the processor and the HDD.
The third traditional way to increase HDD performance is to replace JBOD with RAID. The performance as a result really grows, especially if you connect the disks to the controller with a large cache, which will distribute data across different disks. But at the same time, in order to achieve the high speed of I / O operations expected by users, such a number of disks will be required that their cost, space occupied, power consumption, heat dissipation and weight will simply be unacceptable.
Answer: flash-
based storage systems. The performance gap between the processing power of servers and storage systems on hard drives, which is especially acute in mission-critical business applications, requires fundamentally different, faster storage systems.
Storage systems based on flash memory are devices with solid-state drives designed to solve the problem with insufficient speed of input-output operations; their speed of data access and input-output operations is many times better than that of hard drives. These storage systems can complement or completely replace traditional hard drive arrays for many mission-critical business applications, including ERP systems, transactional databases, and analytic applications (business intelligence, etc.). Moreover, due to the gradual decrease in prices for these new devices, it is quite possible today to build a whole storage system based on one flash memory.
In the broad sense of the word, solid state drive (SSD) refers to any data input-output device devoid of mechanical components. Recently, however, the term SSD has been used mainly to refer to solid-state drives that have a traditional hard drive form factor and are replacing it. SSDs of this form factor should not be confused with flash-based storage systems. The interfaces and controllers of such SSDs are based on traditional hard drive technologies, which were created based on the inherent HDD access delays and bandwidth limitations. Storage systems based on flash memory are initially developed based on the features of this technology and based on a fast FPGA controller,
For modern flash storage systems, I / O latency is reduced due to the fact that the access speed is 250 times faster than hard drives (0.2 ms versus 5 s). And an increase in access speed, in turn, allows flash memory to issue 2,000 times more I / O operations per second than HDDs (400,000 or more IOPS versus 200). Such an improvement can dramatically reduce delays in access to information due to the slowness of the storage system.
The introduction of flash drive technologies or the replacement of existing disks allows servers of any architecture to increase the CPU capacity in terms of the number of tasks performed per unit time. So, using the example of IBM Power Systems, the increase in server performance when working with the Oracle database reached up to 40% in open testing, which can significantly increase the effective utilization of the existing infrastructure. That is, the server without configuration changes will be able to perform 40% more requests over the same period of time.
IBM FlashSystem Family
IBM considers flash memory to be a strategically important storage technology and aims to be at the forefront of developing storage systems based only on flash memory. (One of the consequences of this policy was the purchase in October 2012 of Texas Memory Systems (TMS), a high-end manufacturer of powerful and reliable flash storage systems.)
The targeted use of IBM FlashSystem helps enterprises increase their flexibility and engage analytics more efficiently - staff can quickly receive information that is always up-to-date as it arrives in real time rather than with large delays. In addition, this system helps optimize data centers and consolidate resources, resulting in increased efficiency of business processes and critical applications. Increases the stability of systems, and without loss of performance and available capacity.
Flash-only storage systems have more capacity than all previous storage technologies. This is because they do not require additional batteries to copy the DDR cache in the event of a power outage, as well as do without a large number of expensive DDR memory modules. The amount of DDR memory they need is minimal - it serves as a write buffer for flash memory and metadata storage during operation. The power needed to copy a small cache and metadata to flash memory during a power outage is provided by small batteries. A flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size.
IBM FlashSystem products use FlashCore technology which allows using media type of the multilevel class MLC to obtain indicators of shelf life and performance no worse than SLC.
FlashSystem storage systems are capable of 1,000,000 reads per second with latencies of less than 100 microseconds and are so compact that devices with capacities up to 57 TB can be placed in one 2U bay. Another important advantage is the high level of availability and reliability, which is mandatory for enterprises: the absence of components whose failure leads to the failure of the entire system, the presence of several levels of data correction, redundant microcircuits and components with support for “hot swap”.
IBM FlashSystem products have the lowest latency and highest IOPS among competitors and are attractive in terms of total cost of ownership. It should be noted that not individual parameters are compared, but a combination of response time and the number of I / O operations. They can be used as storage systems of so-called. level 0 (Tier 0, that is, surpassing the capabilities of the traditional Tier 1 level - data storage systems needed for the enterprise’s primary tasks) in combination with SVC (IBM System Storage SAN Volume Controller). Also, IBM FlashSystem will be especially useful when the performance of mission-critical business applications is directly dependent on cooling, low power consumption and compact size. Today, two main models are presented: IBM FlashSystem 900 and IBM FlashSystem V9000. The first takes up 2 units and provides high performance while the second system takes up 6U and, in addition to high performance, has a set of High End functionality. That allows you to optimize the use of volume, more efficiently distribute the load, create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line. create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line. create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line.
Economic benefits of IBM FlashSystem
Flash-only storage systems have not only technical advantages, but also a number of economic benefits over traditional HDD-based systems. For example, the cost of licensing software for IBM FlashSystem storage systems is 50% lower than that of HDD-based systems.
In addition, the much smaller size and higher density of data storage can significantly reduce the footprint. As noted above, a flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size. Thus, in one rack you can place a system with a capacity of more than 1 petabyte.
Flash-only storage systems are also much more energy-efficient than comparable HDD-based systems — energy costs are 75% lower.
- The cost of licensing software for IBM FlashSystem is 50% lower than that of systems based on HDD.
- IBM FlashSystem takes up significantly less space and saves up to 75% of energy in comparison with HDD.
- The cost of operational support for systems based on flash memory is 35% lower compared to HDD
- The total cost of a data storage system based only on flash memory is 31% less than that of HDD
Switching to flash memory
The key to the maximum return and economic efficiency of flash memory is its strategic use. A flash-based storage system is useful for many applications, but not for everyone; First of all, it is needed by the main applications of the enterprise. The key to high application performance, satisfying users with optimal total cost of ownership lies in the ability to identify those applications that benefit from upgrading to flash-based storage systems, and those that are enough for traditional HDD systems or hybrid systems based on a combination these technologies.
Poor application performance is usually associated with a large volume of simultaneous and often highly complex database queries. If the weak point in this case is the I / O subsystem, then it is necessary to identify the database components that operate under the highest load and thus cause a delay in the I / O operations.
In some cases, entire databases are migrated to flash-based storage systems. This is usually a database with a standard high volume of simultaneous queries or regular random access to all database tables, when it is not possible to isolate a subset of the most frequently used files.
Small and medium sized databases are also suitable for transferring to flash storage systems, because such solutions are extremely attractive at a price, especially compared to RAID systems. The same is true for large databases with a large number of read requests.
Other arguments in favor of flash memory are the drawbacks of large disk arrays from the HDD, such as large footprint and high power consumption.
Applications critical for business include applications for a wide variety of tasks, for example: business intelligence (BI - business intelligence), online transaction processing (OLTP), online analytical processing (OLAP), virtual work infrastructure desks, high-performance computing systems and content delivery applications (cloud storage systems, on-demand video delivery, etc.). All these important tasks are united by one thing: high requirements for speed of work, because it depends on how quickly management, employees, customers and other key business partners get access to the necessary information.
Failure to quickly access and process sensitive data leads to significant business and financial risks. In particular, such delays cause:
- Inability to timely and correctly assess the situation - without access to critical data at the right time and in the right place, people do not see the full picture of what is happening and, as a result, make the wrong decisions, making completely unnecessary mistakes.
- A sharp decrease in the ability to predict business results - the quality of financial forecasts at the level of an individual project, a separate area of activity and the whole company depends on the ability to receive the most accurate data in a timely manner. Without access to such data, the company cannot analyze them, which negatively affects its activities.
- Growing customer dissatisfaction - in many sectors (financial services, healthcare, tourism, etc.) the quality of service directly depends on the availability of instant access to relevant information, especially since a modern client expects service 24 hours a day, 7 days in week and 365 days a year. Under these conditions, the inability to quickly provide the necessary information to customers not only means lost profits, but also often leads to loss of the client, or even market share.
- Too large volumes, variety and speed of information flow - explosive growth in data volumes leads to the inevitable decline in the performance of the main applications in many enterprises. The increase in delays is becoming a source of user dissatisfaction, and enterprises are missing out on opportunities.
Heads of IT departments looking for a solution to these problems that would have the best combination of functionality, implementation time, scalability and cost of systems, you will not envy. Ultimately, the right decision may not be obvious.
Statement of the problem (hint: processor speed is the wrong answer)
As a rule, the IT department primarily responds to system slowdowns by adding or upgrading processors and memory. However, too often, the performance gain from this upgrade is minimal, while the cost is high. The second method, often associated with the first, at least for databases, is to optimize SQL statements. Enterprises spend millions of dollars on it and even get some results. But they overlook the fact that the most ideal database will be powerless against the low speed of the data storage system, namely the latter is often the source of the problem.
Indeed, the root cause of application performance problems often lies in the storage system, rather than processor speed or lack of memory. When the processor is idle waiting for information from the storage system, valuable time and resources are wasted. And given the fact that over the past 20 years the processing power of processors has grown exponentially, this downtime is becoming more and more. Simply put, the computing resources of modern processors far exceed the speed with which hard drives can deliver data to them. This performance gap is especially acute in database servers, where the number of I / O operations is usually much larger than in other systems. As a result, ultra-fast processors and huge bandwidth are often not fully utilized, because accessing data on hard drives takes several milliseconds, which for modern processors is an eternity. And when servers wait a long time for data from storage systems, users wait a long time for a response from servers. This is where the real source of the problem with low I / O operations.
This problem is due to purely physical causes. Data movement on solid-state drives and through the network occurs at a speed comparable to the speed of the current in the electric circuit, i.e., in fact, the speed of light, while hard drives are mechanical devices that read information by rotating the plate. Thus, with all its reliability and performance, hard drives in terms of access speed are by far the weakest link in the information transfer chain.
Disadvantages of Traditional Workarounds for HDD Speed Issues
They try to solve performance problems due to the inefficiency of hard drives in several ways, in particular, by creating JBOD arrays (just a bunch of disks - just a set of disks) or RAID (redundant array of independent disks - redundant array of independent disks). With the increase in the number of disks, it becomes possible to distribute the I / O operations carried out by the database among several disks. Unfortunately, this gives a too modest increase in HDD performance.
It is also customary to move frequently used files to a separate drive. In this case, the I / O speed of one disk really grows to its maximum, but even at maximum this speed is still low. In the best case, one hard drive produces ~ 200, without the use of additional functionality, input / output operations per second (IOPS - input / output operations per second), which is several times lower than the speed that could reduce the performance gap between the processor and the HDD.
The third traditional way to increase HDD performance is to replace JBOD with RAID. The performance as a result really grows, especially if you connect the disks to the controller with a large cache, which will distribute data across different disks. But at the same time, in order to achieve the high speed of I / O operations expected by users, such a number of disks will be required that their cost, space occupied, power consumption, heat dissipation and weight will simply be unacceptable.
Answer: flash-
based storage systems. The performance gap between the processing power of servers and storage systems on hard drives, which is especially acute in mission-critical business applications, requires fundamentally different, faster storage systems.
Storage systems based on flash memory are devices with solid-state drives designed to solve the problem with insufficient speed of input-output operations; their speed of data access and input-output operations is many times better than that of hard drives. These storage systems can complement or completely replace traditional hard drive arrays for many mission-critical business applications, including ERP systems, transactional databases, and analytic applications (business intelligence, etc.). Moreover, due to the gradual decrease in prices for these new devices, it is quite possible today to build a whole storage system based on one flash memory.
In the broad sense of the word, solid state drive (SSD) refers to any data input-output device devoid of mechanical components. Recently, however, the term SSD has been used mainly to refer to solid-state drives that have a traditional hard drive form factor and are replacing it. SSDs of this form factor should not be confused with flash-based storage systems. The interfaces and controllers of such SSDs are based on traditional hard drive technologies, which were created based on the inherent HDD access delays and bandwidth limitations. Storage systems based on flash memory are initially developed based on the features of this technology and based on a fast FPGA controller,
For modern flash storage systems, I / O latency is reduced due to the fact that the access speed is 250 times faster than hard drives (0.2 ms versus 5 s). And an increase in access speed, in turn, allows flash memory to issue 2,000 times more I / O operations per second than HDDs (400,000 or more IOPS versus 200). Such an improvement can dramatically reduce delays in access to information due to the slowness of the storage system.
The introduction of flash drive technologies or the replacement of existing disks allows servers of any architecture to increase the CPU capacity in terms of the number of tasks performed per unit time. So, using the example of IBM Power Systems, the increase in server performance when working with the Oracle database reached up to 40% in open testing, which can significantly increase the effective utilization of the existing infrastructure. That is, the server without configuration changes will be able to perform 40% more requests over the same period of time.
IBM FlashSystem Family
IBM considers flash memory to be a strategically important storage technology and aims to be at the forefront of developing storage systems based only on flash memory. (One of the consequences of this policy was the purchase in October 2012 of Texas Memory Systems (TMS), a high-end manufacturer of powerful and reliable flash storage systems.)
The targeted use of IBM FlashSystem helps enterprises increase their flexibility and engage analytics more efficiently - staff can quickly receive information that is always up-to-date as it arrives in real time rather than with large delays. In addition, this system helps optimize data centers and consolidate resources, resulting in increased efficiency of business processes and critical applications. Increases the stability of systems, and without loss of performance and available capacity.
Flash-only storage systems have more capacity than all previous storage technologies. This is because they do not require additional batteries to copy the DDR cache in the event of a power outage, as well as do without a large number of expensive DDR memory modules. The amount of DDR memory they need is minimal - it serves as a write buffer for flash memory and metadata storage during operation. The power needed to copy a small cache and metadata to flash memory during a power outage is provided by small batteries. A flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size.
IBM FlashSystem products use FlashCore technology which allows using media type of the multilevel class MLC to obtain indicators of shelf life and performance no worse than SLC.
FlashSystem storage systems are capable of 1,000,000 reads per second with latencies of less than 100 microseconds and are so compact that devices with capacities up to 57 TB can be placed in one 2U bay. Another important advantage is the high level of availability and reliability, which is mandatory for enterprises: the absence of components whose failure leads to the failure of the entire system, the presence of several levels of data correction, redundant microcircuits and components with support for “hot swap”.
IBM FlashSystem products have the lowest latency and highest IOPS among competitors and are attractive in terms of total cost of ownership. It should be noted that not individual parameters are compared, but a combination of response time and the number of I / O operations. They can be used as storage systems of so-called. level 0 (Tier 0, that is, surpassing the capabilities of the traditional Tier 1 level - data storage systems needed for the enterprise’s primary tasks) in combination with SVC (IBM System Storage SAN Volume Controller). Also, IBM FlashSystem will be especially useful when the performance of mission-critical business applications is directly dependent on cooling, low power consumption and compact size. Today, two main models are presented: IBM FlashSystem 900 and IBM FlashSystem V9000. The first takes up 2 units and provides high performance while the second system takes up 6U and, in addition to high performance, has a set of High End functionality. That allows you to optimize the use of volume, more efficiently distribute the load, create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line. create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line. create storage clusters both within the data center and geographically dispersed. In the coming days, it is planned to launch the latest development that will increase the portfolio of the proposed systems and add functionality to the IBM FlashSystem line.
Economic benefits of IBM FlashSystem
Flash-only storage systems have not only technical advantages, but also a number of economic benefits over traditional HDD-based systems. For example, the cost of licensing software for IBM FlashSystem storage systems is 50% lower than that of HDD-based systems.
In addition, the much smaller size and higher density of data storage can significantly reduce the footprint. As noted above, a flash-only storage system with 57 terabytes of addressable memory and high availability is only 2U in size. Thus, in one rack you can place a system with a capacity of more than 1 petabyte.
Flash-only storage systems are also much more energy-efficient than comparable HDD-based systems — energy costs are 75% lower.
- The cost of licensing software for IBM FlashSystem is 50% lower than that of systems based on HDD.
- IBM FlashSystem takes up significantly less space and saves up to 75% of energy in comparison with HDD.
- The cost of operational support for systems based on flash memory is 35% lower compared to HDD
- The total cost of a data storage system based only on flash memory is 31% less than that of HDD
Switching to flash memory
The key to the maximum return and economic efficiency of flash memory is its strategic use. A flash-based storage system is useful for many applications, but not for everyone; First of all, it is needed by the main applications of the enterprise. The key to high application performance, satisfying users with optimal total cost of ownership lies in the ability to identify those applications that benefit from upgrading to flash-based storage systems, and those that are enough for traditional HDD systems or hybrid systems based on a combination these technologies.
Poor application performance is usually associated with a large volume of simultaneous and often highly complex database queries. If the weak point in this case is the I / O subsystem, then it is necessary to identify the database components that operate under the highest load and thus cause a delay in the I / O operations.
In some cases, entire databases are migrated to flash-based storage systems. This is usually a database with a standard high volume of simultaneous queries or regular random access to all database tables, when it is not possible to isolate a subset of the most frequently used files.
Small and medium sized databases are also suitable for transferring to flash storage systems, because such solutions are extremely attractive at a price, especially compared to RAID systems. The same is true for large databases with a large number of read requests.
Other arguments in favor of flash memory are the drawbacks of large disk arrays from the HDD, such as large footprint and high power consumption.