Why the future of data storage is still behind the tape
Hard disks are approaching the limits of their development, and the film only becomes better with time.
Financial institutions oblige companies to store more and more data and an increasingly long period of time. The amount of data that has to be stored increases every year by 30–40 percent compared to the previous year. The capacity of hard drives is also growing, but with a half the pace. Fortunately, all this information does not require instant access, so the tape is an excellent solution to the problem.
In general, a lot of information in the world is stored on the tape: scientific data on particle physics, astronomical data, national archives, cultural heritage, most movies , banking data, and so on. There are professionals (material specialists, engineers, physicists), whose job is to improve the methods of storing data on film.
Over the decades, the film has developed no less than hard drives or transistors. The first film for storing information in digital form - IBM model 726 - could store 1.1 MB on a reel. Today, one coil is capable of storing 15 terabytes of data, and one robotic film storage — 278 petabytes.
Of course, the film does not allow to read information as quickly as hard disks or semiconductor memory. But it has its advantages. The film is energy efficient: if the data is already recorded, the film does not require power to store them. The film is reliable: the probability of errors when writing or reading is 4-5 orders of magnitude lower than that of hard drives. The film is safe: unlike disks, which, as a rule, are permanently connected to the computer, coil cartridges can be stored without being connected to devices, which protects the data on the film from being read or modified by hackers or from human error.
In 2011, due to an error in the software on Google servers , I accidentally deleted mailin 40,000 drawers. The deletion occurred on all backups on hard drives, because the erroneous operation of the chain went through them, but the letters were restored from the tape. After this incident, it became known for the first time that Google backed up the tape, and then Microsoft confirmed that they use IBM film equipment in their Azure cloud service .
Magnetic tape was first used to record Univac computer data in 1951.
Storing data on film is 6 times cheaper than on hard drives, so it is used everywhere when it comes to large amounts of information. Since the film has practically disappeared from the consumer market, the majority do not know how fast it is developing and will continue to develop in the foreseeable future.
The film survivedbecause it is cheap and cheap with time. It can be assumed that, since the compressed recording of data on hard disks comes to naught, the same applies to the film, because it uses approximately the same technology (only the older one). It's like “Moore's Law”, but for a magnetic tape. But this is not the case: over the years, the rate of compaction of the recording on the film does not fall down, but remains at around 33% per year. That is, doubling the amount of data recorded on the tape occurs approximately every 2-3 years.
Physically, the technology of recording on hard disks and tapes is the same: the data is recorded on a magnetized surface by narrow tracks, on which polarity is switched. Information is recorded in a sequence of bits. Since the appearance of the film and hard drives in the 50s, manufacturers of both of them have been striving for greater density, speed and low cost, so the cost of storage in dollars per gigabyte has decreased by orders of magnitude. Due to the fact that production is trying to reduce the cost, the recording density per square millimeter is growing.
The more funding for research and development is obtained by companies producing magnetic media, obviously, the more these media are progressing. Now the most advanced hard drives can be recorded 100 times more information than in the same area of the film. But since this area itself is much larger on the film in the coil, it can hold up to 15 TB of data , which is more than on any existing disks on the market. At the same time, the dimensions of the cartridge with the film coil and the hard disk are approximately the same.
Outside and inside: A modern cartridge contains one coil. After installing the cartridge, the film is automatically fed to a reading or recording device.
In addition to the capacity of the film and hard drives there is another difference: the speed of access to data. In the coils are magnetic tapes with a length of several hundred meters, the average data access time is from 50 to 60 seconds. Hard disks have this time - from 5 to 10 milliseconds. However, the recording speed is twice as fast.
In recent years, the rate of consolidation of recordings on disks has decreased from 40% to 15% per year. The reason is fundamental physics. To record more data in the same area, you need to reduce the area for recording each bit. As a result, this reduces the signal strength during data reading. If the signal strength is reduced too much, it can mix with magnetic noise from neighboring magnetic beads covering the surface of the disk. You can reduce the noise by making the pellets themselves smaller. But then the granule will be already so small that it can hardly keep its state of magnetization stably. The smallest size of granules, suitable for magnetic recording, has already been achieved, in the professional field it is called the supermagnetic limit .
Until recently, the achievement of this limit remained unnoticeable for consumers, because manufacturers added additional disks and heads for writing and reading inside the container, making the hard disk of the same size, but larger. However, now more disks inside the container are already difficult to add, retaining its dimensions, so the limit becomes more noticeable.
There are alternative ways of recording on a magnetic surface that theoretically can overcome the supermagnetic limit. This is a recording, accompanied by heating of the pellets , and a microwave recording . But it is difficult in the engineering and financial aspects. Western Digital Announces Microwave-Recorded Hard Drive It Is About To Release In 2019. It is expected that such an innovation will allow maintaining the rate of consolidation of the record in the region of 15% per year.
At the same time, storage on the film is still far from reaching the supermagnetical limit, so the film can evolve for decades, not resting on its "Moore's Law" and the limitations of fundamental physics.
The film has a cunning nature. Changing cartridges with coils in the recording equipment, thin polymer material, parallel recording on 32 tracks - all this creates difficulties in the design of this storage medium.
In 2015, IBM, in collaboration with FujiFilm Corporation , found that when recording using ultra-small barric-ferritic magnetic particles arranged perpendicular to the film surfacecan achieve 12 times more density than other technologies allow. And in 2017, in collaboration with Sony, we managed to achieve a density that is 20 times higher than the most modern tape drives. In the future, film companies, for example, it will store all the material of a high-budget film on just one reel instead of a dozen.
Flood data: modern film storage contains hundreds of petabytes of data, and the model 726 from IBM, introduced in 1952, could save only a couple of megabytes.
To achieve this progress, engineers have adapted the heads for reading and writing to move along extremely narrow tracks on the film - about 100 nanometers wide. In addition, I had to make the reading heads narrower - about 50 nanometers wide. When reading, the signal-to-noise level also decreased, so we had to manipulate the size and position of the magnetized granules and the smoothness of the film surface, as well as improve the signal processing and reading errors.
In order to ensure the reliability of the recorded data for decades, engineers have developed new recording heads that produce much stronger magnetic fields than conventional ones.
By combining all these developments, IBM engineers managed to achieve a recording density of 818,000 bits per linear inch (this density measurement has developed historically). The new technology allowed 246200 recording tracks to fit one inch and provided space for 201 gigabits per square inch. The cartridge with 1140 meters of film on the coil is able to save 330 terabytes of information. This can be compared to a whole cart of hard drives.
The data storage consortium , which includes HP, IBM, Oracle, Quantum, and several research groups, in 2015 released a document on tape storage development plans. According to the consortium's forecast, by 2025 the recording density per square inch will increase to 91 gigabytes, and by 2028 - to 200 gigabytes.
The authors of the document are professionally interested in such an optimistic forecast, but it is quite realistic. The IBM laboratory confirms that 200 gigabytes per square inch is a feasible goal for the next decade.
The film is a carrier of information, which "Moore's law" will press in the last place. Therefore, the benefits of storing data on film compared to hard drives will increase in the coming years.
The author of the article, Mark Lanz, works as a manager in an IBM lab in Zurich and deals with problems with storing data on film.
The article was originally published in print under the heading “Tape Storage Mounts a Comeback,” and then published on the website of the IEEE consortium.. The translation uses photos from the original article.
Financial institutions oblige companies to store more and more data and an increasingly long period of time. The amount of data that has to be stored increases every year by 30–40 percent compared to the previous year. The capacity of hard drives is also growing, but with a half the pace. Fortunately, all this information does not require instant access, so the tape is an excellent solution to the problem.
In general, a lot of information in the world is stored on the tape: scientific data on particle physics, astronomical data, national archives, cultural heritage, most movies , banking data, and so on. There are professionals (material specialists, engineers, physicists), whose job is to improve the methods of storing data on film.
Over the decades, the film has developed no less than hard drives or transistors. The first film for storing information in digital form - IBM model 726 - could store 1.1 MB on a reel. Today, one coil is capable of storing 15 terabytes of data, and one robotic film storage — 278 petabytes.
Of course, the film does not allow to read information as quickly as hard disks or semiconductor memory. But it has its advantages. The film is energy efficient: if the data is already recorded, the film does not require power to store them. The film is reliable: the probability of errors when writing or reading is 4-5 orders of magnitude lower than that of hard drives. The film is safe: unlike disks, which, as a rule, are permanently connected to the computer, coil cartridges can be stored without being connected to devices, which protects the data on the film from being read or modified by hackers or from human error.
In 2011, due to an error in the software on Google servers , I accidentally deleted mailin 40,000 drawers. The deletion occurred on all backups on hard drives, because the erroneous operation of the chain went through them, but the letters were restored from the tape. After this incident, it became known for the first time that Google backed up the tape, and then Microsoft confirmed that they use IBM film equipment in their Azure cloud service .
Magnetic tape was first used to record Univac computer data in 1951.
Storing data on film is 6 times cheaper than on hard drives, so it is used everywhere when it comes to large amounts of information. Since the film has practically disappeared from the consumer market, the majority do not know how fast it is developing and will continue to develop in the foreseeable future.
The film survivedbecause it is cheap and cheap with time. It can be assumed that, since the compressed recording of data on hard disks comes to naught, the same applies to the film, because it uses approximately the same technology (only the older one). It's like “Moore's Law”, but for a magnetic tape. But this is not the case: over the years, the rate of compaction of the recording on the film does not fall down, but remains at around 33% per year. That is, doubling the amount of data recorded on the tape occurs approximately every 2-3 years.
Physically, the technology of recording on hard disks and tapes is the same: the data is recorded on a magnetized surface by narrow tracks, on which polarity is switched. Information is recorded in a sequence of bits. Since the appearance of the film and hard drives in the 50s, manufacturers of both of them have been striving for greater density, speed and low cost, so the cost of storage in dollars per gigabyte has decreased by orders of magnitude. Due to the fact that production is trying to reduce the cost, the recording density per square millimeter is growing.
The more funding for research and development is obtained by companies producing magnetic media, obviously, the more these media are progressing. Now the most advanced hard drives can be recorded 100 times more information than in the same area of the film. But since this area itself is much larger on the film in the coil, it can hold up to 15 TB of data , which is more than on any existing disks on the market. At the same time, the dimensions of the cartridge with the film coil and the hard disk are approximately the same.
Outside and inside: A modern cartridge contains one coil. After installing the cartridge, the film is automatically fed to a reading or recording device.
In addition to the capacity of the film and hard drives there is another difference: the speed of access to data. In the coils are magnetic tapes with a length of several hundred meters, the average data access time is from 50 to 60 seconds. Hard disks have this time - from 5 to 10 milliseconds. However, the recording speed is twice as fast.
In recent years, the rate of consolidation of recordings on disks has decreased from 40% to 15% per year. The reason is fundamental physics. To record more data in the same area, you need to reduce the area for recording each bit. As a result, this reduces the signal strength during data reading. If the signal strength is reduced too much, it can mix with magnetic noise from neighboring magnetic beads covering the surface of the disk. You can reduce the noise by making the pellets themselves smaller. But then the granule will be already so small that it can hardly keep its state of magnetization stably. The smallest size of granules, suitable for magnetic recording, has already been achieved, in the professional field it is called the supermagnetic limit .
Until recently, the achievement of this limit remained unnoticeable for consumers, because manufacturers added additional disks and heads for writing and reading inside the container, making the hard disk of the same size, but larger. However, now more disks inside the container are already difficult to add, retaining its dimensions, so the limit becomes more noticeable.
There are alternative ways of recording on a magnetic surface that theoretically can overcome the supermagnetic limit. This is a recording, accompanied by heating of the pellets , and a microwave recording . But it is difficult in the engineering and financial aspects. Western Digital Announces Microwave-Recorded Hard Drive It Is About To Release In 2019. It is expected that such an innovation will allow maintaining the rate of consolidation of the record in the region of 15% per year.
At the same time, storage on the film is still far from reaching the supermagnetical limit, so the film can evolve for decades, not resting on its "Moore's Law" and the limitations of fundamental physics.
The film has a cunning nature. Changing cartridges with coils in the recording equipment, thin polymer material, parallel recording on 32 tracks - all this creates difficulties in the design of this storage medium.
In 2015, IBM, in collaboration with FujiFilm Corporation , found that when recording using ultra-small barric-ferritic magnetic particles arranged perpendicular to the film surfacecan achieve 12 times more density than other technologies allow. And in 2017, in collaboration with Sony, we managed to achieve a density that is 20 times higher than the most modern tape drives. In the future, film companies, for example, it will store all the material of a high-budget film on just one reel instead of a dozen.
Flood data: modern film storage contains hundreds of petabytes of data, and the model 726 from IBM, introduced in 1952, could save only a couple of megabytes.
To achieve this progress, engineers have adapted the heads for reading and writing to move along extremely narrow tracks on the film - about 100 nanometers wide. In addition, I had to make the reading heads narrower - about 50 nanometers wide. When reading, the signal-to-noise level also decreased, so we had to manipulate the size and position of the magnetized granules and the smoothness of the film surface, as well as improve the signal processing and reading errors.
In order to ensure the reliability of the recorded data for decades, engineers have developed new recording heads that produce much stronger magnetic fields than conventional ones.
By combining all these developments, IBM engineers managed to achieve a recording density of 818,000 bits per linear inch (this density measurement has developed historically). The new technology allowed 246200 recording tracks to fit one inch and provided space for 201 gigabits per square inch. The cartridge with 1140 meters of film on the coil is able to save 330 terabytes of information. This can be compared to a whole cart of hard drives.
The data storage consortium , which includes HP, IBM, Oracle, Quantum, and several research groups, in 2015 released a document on tape storage development plans. According to the consortium's forecast, by 2025 the recording density per square inch will increase to 91 gigabytes, and by 2028 - to 200 gigabytes.
The authors of the document are professionally interested in such an optimistic forecast, but it is quite realistic. The IBM laboratory confirms that 200 gigabytes per square inch is a feasible goal for the next decade.
The film is a carrier of information, which "Moore's law" will press in the last place. Therefore, the benefits of storing data on film compared to hard drives will increase in the coming years.
The author of the article, Mark Lanz, works as a manager in an IBM lab in Zurich and deals with problems with storing data on film.
The article was originally published in print under the heading “Tape Storage Mounts a Comeback,” and then published on the website of the IEEE consortium.. The translation uses photos from the original article.