ext3cow file system

Recently an interesting file system - ext3cow caught my eye . I decided to do a little review. The ext3cow file system implements a function that is complementary to the ext3 functional - time shift.

Those. this is some combination of a version control system and a file system. This is what the documentation says:

In time-shifting, ext3cow introduces an interface to versioning that presents a continuous view of time. Users or applications specify a file name and any point in time, which ext3cow scopes to the appropriate snapshot or file version. The time-shifting interface allows user-space tools to create snapshots and access versions. Applications may access these tools to coordinate snapshots with application state. User-space tools are suitable for automation, using software as simple as cron.

One of the features of the system is its versatility - the section can be connected without changes in the kernel, only by loading the module dynamically. This is possible from the file system structure.

how it works on the part of the user: the

user at a certain point in time who wants to backup the file (s) executes a command (with or without a parameter - a specific file or all files) This command creates a version of the data on the disk that is current at the moment (the date is returned in Unix format epochs). This is called a snapshot epoch .
[user@host]: snapshot
Snapshot on . 1067992521




Further changes to the file can also be snapshot and will be saved already in another era (which is remarkable - file changes are not saved within the same era. I.e. backup at will. But, based on ext3 capabilities - no more than 1 snapshot per second . this is a significant difference between this file system and version control systems. subversion, for example).
Reading a file of a certain era looks like this: You can view the list of epochs in which versions of a certain file exist, i.e. as if on the file system all versions of the file are stored under the names file @ TIME , which is quite correct from the point of view of the file system model in the kernel. Because interesting things are possible -
[user@host]: cat somefile@1067992521


[user@host]: ls file@


• You can mount snapshots as directories. quote:
  

  Distinguished snapshots may be created using links, which allows time-shifting to emulate the behavior of systems that put snapshots in their own namespaces or mount snapshot
  namespaces in separate volumes. For example, an administrator might create a read-only
  version of a file system prior to installing new software.

  
  
• treelike past:
  

  it is permissible to have multiple time-shifts in a single path, eg / A / B @ time1 / C / D @ time2 / E.

  
  

it should be noted that backup versions are immutable and read-only.
the operating system (page cache) looks at one file, all backup copies of which are created only on disk.

how it works from the inside:
The main difference between ext3cow and ext3 is the increase in the amount of meta-information in i-nodes (offtop: oh, remembering Simonenko and the 3rd retake :)). Quote:

ext3cow does not interfere or replace the Linux common file model, therefore, it integrates easily, requiring no changes to the VFS data structures or interfaces. Modifications are limited to on-disk metadata and the in-memory file system specifications file of the VFS metadata. Ext3cow adds metadata to inodes, directory entries, and the superblock that allows it to scope requests from any point-in-time into a speci fi c file version and support scoping inheritance.

Further:

Both on-disk and in-memory inodes were retro filtered to support snapshot and copy-on-write by adding three filters: an inode epoch number , a copy-on-write bitmap , and filter pointing to the next inode in the version chain.

Those. file version is a linked list of i-nodes, whose head is the latest version of the file. Each of which contains an era number, a bitmap copy-on-write mask (by the way, cow probably came from here :)) and a link to the next i-node with the next version.

Details for each item:

• era number . the OS stores an epoch counter that corresponds to the current time. when there is a snapshot - the value of this counter is written to the i-node, and the counter itself is incremented. those. Nover of the era in the i-node - file version
• a copy-on-write bitmap . This map is intended for storing change marks. Those. the structure here resembles svn-like systems - each i-node stores links to file blocks. If the file block has not changed in the current version compared to the previous one, the link to the block is simply copied and two i-nodes point to one block. Links to changed / deleted blocks are stored in i-nodes of older versions. The bitmap contains the flags - whether to copy the link or whether the block has changed and the link will remain in the old version of the file.
• pointer to the next i-node . Then the developers proposed a not very beautiful solution. Because in ext3, the chain of i-nodes is limited in length, then we won’t be able to continue the chain of file versions forever. I must say that the meta-information of the directory is also expanded - birthepoch and deathepoch are stored in links to file i-nodes. This is necessary for storing files deleted in new versions, but remaining in old versions of directories. Therefore, the restriction on the length of the chain of i-nodes is bypassed by such a workaround:
  

  When the length of a version chain meets a threshold value, ext3cow terminates that chain by setting the death epoch of the directory entry used to access this chain to the current system epoch and creates a new chain (of length one) by creating a duplicate directory entry with a birth epoch equal to the system epoch. The stability of inodes ensures that other directory entries linking to the same data fi nd the new chain. Data blocks may be shared between inodes in the two chains. A long-lived, frequently-updated fi le is described by many short chains rather than a single long chain. While directory entries are also
  linear-search structures, this scheme increases search by a constant factor. It will improve the performance of version search from O (n) to O (1) when ext3 adopts extensible hashing for directories.

Here. The structure is quite simple and clear (the author received PhD on it).

By use. They say that the system is quite stable and can now be used, although the current implementation has a lot of TO-DO and sometimes lacks essential functionality ( bugs and todos ). Quote:

The authors have been running ext3cow to store data on their laptops and personal workstations since June 2003. We have not experienced a system crash or data loss incident in that period. Ext3cow has appeal beyond the regulatory environment for which it is designed; it has been adopted as the storage platform for several research projects.

There shouldn’t be any problems with speed either - in the documentation there is a comparative test with ext3, so ext3cow loses by 5-6 ms on two or three types of tests, which is already impressive.

wait a bit and you can do partitions and test. unless, of course, such functionality of the file system is in demand :)

by-the-way. I wonder how the gloriously famous new TimeMachine Apple is implemented. program log?

well, and at the end, of course:

Ext3cow is stable and ready for use under the GNU Public License. It is available for download at http://www.ext3cow.com .