June 24, 2010 at 01:36The problem of dynamic allocation of disk space of virtual machines
Usually, ready-made articles with answers are written on the hub, but I want to raise a question, the answer to which, it seems to me, will be negative.
The essence of the problem: Virtualization systems with a high degree of virtualization (I say this, to sweep aside openvz) provide a place for virtual machines in the form of a virtual disk. Or, more precisely, a virtual block device. Sometimes a fairy tale is emulated (VMWare Server, Virtual PC), sometimes an “undefined block device”, sometimes the plain text “virtual disk” (Xen for PV machines). It is important that this device completely repeats the properties of a block device - a set of sectors that can be read and written.
Most virtualization systems allow you to resize disks. Many systems allow you to do this on the go, online.
The question is: can a guest OS use this place freely? This question goes beyond virtualization and is a general question: do we have file systems that can dynamically change in size in normal mode?
Firstly, I must say that most file systems can grow on the go. XFS, raiserfs, ext3-4 ... Some file systems can shrink (ext3, raiser ... xfs does not know how). Windows here looks like a full outcaster, since it does not support resizing the NTFS file system, especially on system disks. Third-party utilities can do this, but not on the go ...
But even the white and fluffy ext3 does not know how to do this in normal mode. You need to say “cut off” with your hands, and the process of raising (especially down) takes enormous resources. According to my personal impressions, on ordinary SATA disks, resizing from 5.5 TB to 4.5 TB of a file system with 4 TB of occupied space takes about 10-12 hours. The increase works out faster in a few minutes. But - all the same, with a loss of performance, heavy load and prohibitively long.
Yes, and the approach itself is incorrect. If we want to dynamically change the size of the used space, then we need a mechanism that would voluntarily free up unused space in the normal FS operation mode and request a new one if necessary.
As far as I understand, there are no such file systems for block devices yet. The only way out is to use file-based network access, in other words, use NFS / SMB for Unix and SMB for windows.
For block devices, as I understand it, only the “increase on demand” issue is partially and limitedly solved, this is done using a sparsed (sparse) disk image file, which is created obviously larger than it is planned to use - when writing additional information to such a disk, he begins to occupy a larger volume of actual space. For a guest OS, this disk does not change size (while it is larger than the actually occupied space). This method completely does not take into account the features of the file system, leads to fragmentation of the image file and does not allow to give up free space.
The file system should be as small as possible. If we have 2GB, then the size of the file system is 2GB + service data. If we decided to create a 100TB file in size, then the file system requests 25 billion blocks of 4 kb from a block device of the underlying level and places the file. When the file is erased, the file system will give the 25 billion back.
“LVM” comes to mind, but LVM does not currently support requests from the file system, and it’s not at all ready to cut holes from the middle of FS ...
Question 1: are there file systems in which there is a similar solution?
Question 2: what should the file system and block device interface look like?
Question 3: Is it possible to imagine replacing block devices with a type of virtual device that would allow you to create and use file systems on the one hand, and would be more adapted to dynamic resizing on the other hand?
Question 4: how will such a device look like?
The essence of the problem: Virtualization systems with a high degree of virtualization (I say this, to sweep aside openvz) provide a place for virtual machines in the form of a virtual disk. Or, more precisely, a virtual block device. Sometimes a fairy tale is emulated (VMWare Server, Virtual PC), sometimes an “undefined block device”, sometimes the plain text “virtual disk” (Xen for PV machines). It is important that this device completely repeats the properties of a block device - a set of sectors that can be read and written.
Most virtualization systems allow you to resize disks. Many systems allow you to do this on the go, online.
The question is: can a guest OS use this place freely? This question goes beyond virtualization and is a general question: do we have file systems that can dynamically change in size in normal mode?
Firstly, I must say that most file systems can grow on the go. XFS, raiserfs, ext3-4 ... Some file systems can shrink (ext3, raiser ... xfs does not know how). Windows here looks like a full outcaster, since it does not support resizing the NTFS file system, especially on system disks. Third-party utilities can do this, but not on the go ...
But even the white and fluffy ext3 does not know how to do this in normal mode. You need to say “cut off” with your hands, and the process of raising (especially down) takes enormous resources. According to my personal impressions, on ordinary SATA disks, resizing from 5.5 TB to 4.5 TB of a file system with 4 TB of occupied space takes about 10-12 hours. The increase works out faster in a few minutes. But - all the same, with a loss of performance, heavy load and prohibitively long.
Yes, and the approach itself is incorrect. If we want to dynamically change the size of the used space, then we need a mechanism that would voluntarily free up unused space in the normal FS operation mode and request a new one if necessary.
Existing solutions
As far as I understand, there are no such file systems for block devices yet. The only way out is to use file-based network access, in other words, use NFS / SMB for Unix and SMB for windows.
For block devices, as I understand it, only the “increase on demand” issue is partially and limitedly solved, this is done using a sparsed (sparse) disk image file, which is created obviously larger than it is planned to use - when writing additional information to such a disk, he begins to occupy a larger volume of actual space. For a guest OS, this disk does not change size (while it is larger than the actually occupied space). This method completely does not take into account the features of the file system, leads to fragmentation of the image file and does not allow to give up free space.
What you really need
The file system should be as small as possible. If we have 2GB, then the size of the file system is 2GB + service data. If we decided to create a 100TB file in size, then the file system requests 25 billion blocks of 4 kb from a block device of the underlying level and places the file. When the file is erased, the file system will give the 25 billion back.
“LVM” comes to mind, but LVM does not currently support requests from the file system, and it’s not at all ready to cut holes from the middle of FS ...
Questions
Question 1: are there file systems in which there is a similar solution?
Question 2: what should the file system and block device interface look like?
Question 3: Is it possible to imagine replacing block devices with a type of virtual device that would allow you to create and use file systems on the one hand, and would be more adapted to dynamic resizing on the other hand?
Question 4: how will such a device look like?