File System Apple File System (APFS)
64-bit inodes, atomic transactions, time stamps in nanoseconds, directory cloning, embedded encryption
At yesterday’s WWDC 2016 presentation , Apple showed new versions of the macOS (Sierra) 10.12 operating system, iOS 10, tvOS 10, watchOS 3, the Swift Playgrounds programming app for teaching children and new emoji.
It would seem nothing interesting. However, Apple still rolled out something fundamental. The most significant development of all mentioned at the presentation is the file system of the new generation Apple File System (APFS) in the macOS (Sierra) 10.12 operating system.
On the developer site, shortly after the presentation, documentation was published with the main characteristics and file system description, which replicates the individual functions of the powerful free ZFS file system.
Apple computers now use the HFS + file system, an enhanced version of the HFS (Hierarchical File System) created over 30 years ago. Like its predecessor, HFS + uses a tree structure called the B * -tree to store most of the metadata. Hence the name "hierarchical file system."
The official presentation of HFS + took place on January 19, 1998, along with MacOS 8.1. Since 2002, the system has implemented journaling to increase the reliability of information storage. Since OS X 10.3, logging is enabled by default, it is possible to work in the mode with regard to the register of names.
Up to OS X 10.7, developers continued to refine HFS + and implement new functions for OS X at the file system level. But the fact remains: HFS was originally developed at the time of floppy disks and spinning hard drives when file sizes were measured in kilobytes or megabytes. Today, many work with SSD drives, where millions of files are stored - gigabytes or terabytes of data. There are completely different requirements for the file system. Instead of refining the old code, Apple decided to finally write a new file system from scratch.
The new generation APFS file system is still at the developer preview stage., that is, it is not planned to roll it out into mass use in the near future. At the moment, you cannot use the APFS volume as a boot disk, nor can you use it in the Time Machine backup system, in Fusion Drive, or with File Vault encryption. But it is possible for a regular non-bootable volume.
There is still a long refinement and testing, but only then APFS will become the main Apple file system for decades to come.
APFS, unlike HFS +, initially distinguishes the case of characters in the names of files and folders, and this feature cannot be disabled. This should be borne in mind by all who decide to use APFS.
In principle, Apple recommends to start experimenting with APFS on an external drive that does not store anything important. For this purpose it is proposed to use the utility.hdiutil .
Main characteristics
The official documentation lists the general characteristics of the APFS file system versus HFS +.
Containers and volumes
A container is the primary object for storing data in APFS. Containers usually completely coincide with GUID Partition Table (GPT) entries, they have their own scheme of protection against failures and allocation of disk space. Each container contains one or more volumes or file systems, each of which has its own namespace , that is, a set of files and directories.
APFS does not directly support software RAID, but it can be used with Apple RAID volumes to support Striping (RAID 0), Mirroring (RAID 1), and Concatenation (JBOD).
64-bit inodes (inodes)
64-bit inodes significantly increase the namespace, compared with the 32-bit identifiers in HFS +. The 64-bit APFS file system supports more than 9 quintillion files on each volume. That should be enough for everyone, as Bill Gates said.
Nanosecond time stamps
The accuracy of time stamps has increased significantly in APFS. APFS supports nanosecond time stamping. For comparison, in HFS + time stamps were set to the nearest second.
Nanosecond timestamps are very important in modern file systems, because they help to realize atomicity and atomic transactions - one of the basic requirements of ACID to a transactional system (for example, to a DBMS). Atomicity guarantees that no transaction will be partially fixed in the system. All its suboperations will be executed, or none of them will be executed.
Crash protection
APFS implements an innovative copy-on-write metadata scheme, which Apple calls “Crash Protection”. It ensures that changes to the file system and logging remain in a synchronized manner if something happens during the recording — for example, the power supply is lost.
ZFS copy-on-write scheme
Sparse files
A file with the attribute “sparse” assumes the content of blocks of zero bytes, not stored on the drive, but implied. HFS + did not support sparse files.
Extended Attributes
APFS has built-in support for extended file attributes, which in HFS + was implemented through the Attributes file, that is, through the B-tree.
Encryption
Apple says encryption is a fundamental property that is built into APFS at the file system level. For each volume in the APFS container, one of the encryption models is set: unencrypted, single key encryption, multiple key encryption. In the latter case, separate keys are used to encrypt files and metadata. Depending on the hardware, APFS uses AES-XTS or AES-CBC encryption mode.
Clone files and directories
Cloning - almost instant copying of a file or directory, which does not require additional space to store data. When a clone is modified, the file system only records data changes. Thus, the new file system can store many versions of large files, taking up less disk space.
Snapshots
Snapshots are open-only “snapshots” of the file system in the volume. The operating system can use snapshots for a more efficient backup procedure. That is, finally, Time Machine will work fine (fast).
Of course, APFS is significantly inferior in its capabilities to the 128-bit ZFS file system , which is supported by Linux, FreeBSD, and other free operating systems, but from Apple’s side, this is a step in the right direction.
It is strange that the preliminary documentation does not mention the compression function, which HFS +, by the way, supports.
Apple has long tried to transfer ZFS to the OS X system, an active discussion was conducted on the ZFS mailing lists, preliminary snapshots were published for the next version of OS X. Later, OpenZFS was made for OS X (O3X) and MacZFX .
The ZFS file system is open source, and Apple could easily borrow some ideas for the APFS file system. The implementation of open source for APFS is not yet ready, Apple plans to publish to document and publish the APFS format in 2017.
At the WWDC conference this evening, the first formal session will be held, where the developers will demonstrate in more detail the new features of APFS.