May 10, 2019 at 00:59What will the new storages for AI and MO systems offer?
MAX Data will be combined with Optane DC to work effectively with AI and MO systems. Photos - Hitesh Choudhary - Unsplash
According to a study by MIT Sloan Management Review and The Boston Consulting Group, 85% of the three thousand managers surveyed believe that AI systems will help their companies gain a competitive advantage in the market. However, only 39% of companies tried to implement something similar in practice.
One of the reasons for this situation is that efficient work with data and optimizing the use of capacities for machine learning tasks is not easy. IDC notes that a new technology based on Persistent Memory (PMEM) could solve the situation.
NetApp and Intel proposed this technology by combining NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory to create a product for local data storage based on non-volatile memory.
MAX Data is a server technology that improves application performance through the use of PMEM or DRAM, but does not require a redesign of the software architecture.
It implements the principles of automated tiered storage, distributing data by levels and storages depending on the frequency of use - for “cold” data, more accessible storage is used, and often used ones are “at hand” - in Persistent Memory, which minimizes the delay when working with such data .
Version 1.1 uses DRAM and NVDIMM . Both implementations have their drawbacks - relative loss of efficiency and high memory cost, respectively - compared to Optane DCPMM. A chart showing a comparative estimate of the delay is presented here.(p. 4).
The technology supports both POSIX and work with the semantics of block or file systems. Storage-level data protection and recovery is implemented using MAX Snap and MAX Recovery. These technologies use snapshots, the SnapMirror tool and other ONTAP defense mechanisms.
Schematically, the implementation is as follows:
There is no PMEM on this scheme yet, but the developers promise to add support for this type of memory in the end of the year. So far, Max Data works with DRAM and DIMM.
IDC claims that in the coming years there will be more developments like MAX Data, as corporate data volumes are constantly growing, and companies lack the capacity to process them efficiently. The technology can come in handy in a large-scale cloud environment and for working with resource-intensive tasks like training neural networks. It will find application on trading platforms, in information security systems and any other software products that require constant and quick access to large amounts of information.
There is also a degree of probability that the technology will not immediately take root in the market. As we noted above, in one form or another, only a third of companies around the world work with AI systems. From this point of view, the appearance of MAX Data may be considered by many to be premature and will focus on a more accessible infrastructure that allows solving current problems.
Our other materials about IT infrastructure:
NetApp and Intel proposed this technology by combining NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory to create a product for local data storage based on non-volatile memory.
How it works
MAX Data is a server technology that improves application performance through the use of PMEM or DRAM, but does not require a redesign of the software architecture.
It implements the principles of automated tiered storage, distributing data by levels and storages depending on the frequency of use - for “cold” data, more accessible storage is used, and often used ones are “at hand” - in Persistent Memory, which minimizes the delay when working with such data .
Version 1.1 uses DRAM and NVDIMM . Both implementations have their drawbacks - relative loss of efficiency and high memory cost, respectively - compared to Optane DCPMM. A chart showing a comparative estimate of the delay is presented here.(p. 4).
The technology supports both POSIX and work with the semantics of block or file systems. Storage-level data protection and recovery is implemented using MAX Snap and MAX Recovery. These technologies use snapshots, the SnapMirror tool and other ONTAP defense mechanisms.
Schematically, the implementation is as follows:
There is no PMEM on this scheme yet, but the developers promise to add support for this type of memory in the end of the year. So far, Max Data works with DRAM and DIMM.
Solution potential
IDC claims that in the coming years there will be more developments like MAX Data, as corporate data volumes are constantly growing, and companies lack the capacity to process them efficiently. The technology can come in handy in a large-scale cloud environment and for working with resource-intensive tasks like training neural networks. It will find application on trading platforms, in information security systems and any other software products that require constant and quick access to large amounts of information.
There is also a degree of probability that the technology will not immediately take root in the market. As we noted above, in one form or another, only a third of companies around the world work with AI systems. From this point of view, the appearance of MAX Data may be considered by many to be premature and will focus on a more accessible infrastructure that allows solving current problems.
Our other materials about IT infrastructure: