June 17, 2016 at 13:13Microsoft has published information about the implementation of VFS in the Linux subsystem on Windows 10
In previous posts, we covered some elements of the Linux subsystem implementation in Windows 10 (WSL). At the same time, we talked about the implementation mechanisms for Linux system services based on Windows 10 system modules. We pointed out that drivers such as LXss.sys and LXCore.sys are responsible for implementing the semantics of Linux system calls using the Windows kernel. If the semantics of the Linux and Windows system service coincide, the above drivers simply redirect the Linux system call to the corresponding Windows equivalent.
This post will be about implementationVFS virtual file system in WSL, which is used as an abstraction layer in Linux when accessing both disk files and other OS objects, including devices, ports, processes, a microprocessor, etc. Since the Windows 10 kernel has the structure of subsystems and originally designed to implement various types of environments, including POSIX, the VX semantics driver LXCore.sys addresses these kernel subsystems by implementing the corresponding semantics and directories like / dev, / proc, / sys.
Fig. General scheme for implementing VFS in WSL. At the top level are Linux applications that call the OS API, the kernel mode driver LXCore.sys is responsible for their implementation. To implement VFS semantics, the driver accesses various subsystems of the Windows kernel, including the object manager (ObMgr) for working with the Windows name system and the object manager namespace, the I / O manager (IoMgr) for implementing the / dev directory, as well as the file system driver NTFS for working with disk files.
WSL provides Linux applications with the necessary VFS semantics, while supporting the permissions system for files and directories, symbolic links, special FIFO files, as well as the above directories / dev, / proc, / sys, etc. When an application calls one of the functions like open, read , chmod , stat , the corresponding system call is taken to process it and transfers control to the VFS implementation layer (LXCore.sys). Further, when processing file paths (for example, when executing the open and stat functions ), VFS converts it to the internal format using a special cache (directory entry cache). In the absence of the required path element in the cache, one or several special plug-ins are called (see below) to create the well-known inode structure of this element. This structure will represent the file opened by the application in WSL.
Windows does not have an inode type structure to represent an open file descriptor; instead, a well-known structure called file object is used , which stores some information about the file (size, attributes, access mask, etc.). LXCore.sys relies on it internally when implementing inode. However, both Linux and Windows use file descriptors to represent an open file, details of which LXCore.sys hides. It defines the levels of the so-called plug-ins under the names VolFs and DrvFs, which are responsible for working with disk files, TmpFs for working with in-memory file system data, as well as pseudo-FS ProcFs, SysFs, and CgroupFs.
Volfs
The VolFs plugin is used by VFS to work with the disk file system, it is used to store Linux system files, as well as the contents of the / home directory. VolFs supports Linux file permissions, symbolic links, sockets, device files, and FIFO. The / root and / home directories are mounted in the% LocalAppData% \ lxss \ root and% LocalAppData% \ lxss \ home directories. When you remove the WSL subsystem, the files stored in these directories are not deleted. As you can see from the path, the Linux directory mount directories are user-specific, that is, each Windows user has its own WSL directory. Therefore, installing Linux applications in WSL by one user will not affect others.
WSL provides applications with two features of the Linux file system that are not directly supported on Windows. The first such feature is the sensitivity of the FS to the case of file names, in which case VolFs simply accesses the object manager, indicating a specific flag for the operation. Since NTFS internally distinguishes case-sensitive characters for file names, implementing this feature is a fairly simple task.
The second feature is support for almost all possible characters as applicable to file names. Windows has stricter restrictions on file names, with some characters in the file names being invalid, while others may have special meanings, such as the colon ":", representing an alternate stream of data in the file. When implementing this feature, VolFS avoids the use of invalid characters in names.
Linux inodes have a number of attributes that are missing from Windows, including information about the owner and group, as well as mode. These attributes are stored in the NTFS Ea extended attribute structure, which are associated with files on disk. The following information is stored in this NTFS Ea file attribute:
DrvFs
To facilitate interaction with Windows, WLS uses the DrvFs plugin. WSL automatically mounts all disk devices with supported file systems to the / mnt directory, for example, / mnt / c, / mnt / d. Volumes with NTFS and ReFS file systems are currently supported. DrvFs works in the same way as VolFs. When a stream opens a file descriptor, file object and inode structures are created. However, unlike VolFs, DrvFs adheres to Windows rules when working with the file system. It uses permissions for files from Windows, only legal NTFS file names, while special files such as FIFO and sockets are not allowed.
Linux is known to use a fairly simple permission model when a file owner, group, or anyone else is allowed to execute it, read or write data. Windows uses a more sophisticated model based on Access Control List (ACLs) that define complex access rules for each individual file or directory (Linux also has ACL support, but this feature is not currently supported by WSL).
When a file is opened through the DrvFs plugin, it uses the Windows permissions mechanism based on the access token of the user in the context of which the bash.exe shell process was launched. Thus, to access the files in the system directory C: \ Windows, it is not enough just to use the commandsudo , which can provide root privileges in WSL. The latter does not change the access token of the process, so to perform the specified operation, you must run the bash application with elevated privileges on Windows.
WSL can give the user a hint about the permissions that he has when accessing files, while DrvFs checks the current user permissions and converts them into read / write / execute bits, which can be seen when the ls -l command is executed. However, the possibility of direct conversion does not always exist; for example, Windows has separate permissions to create files or subdirectories in a directory. If the user has such permissions, DrvFs will indicate to him the presence of write-access to the directory, while some operations with it may not be available to him.
Since the specified WLS file access may vary depending on what rights the bash.exe process running on Windows has, the specified file permissions will change when switching between copies of bash.exe with different rights (one is run from a simple user, and second from the administrator). When calculating file permissions, DrvFs takes into account the read-only attribute. A file with the read-only attribute will be displayed in WSL as having no write permission. The chmod command can be used to set the read-only attribute (by deleting all write permissions, i.e. chmod aw some_file ) or by deleting it (by setting any write permissions, i.e. chmod u + w some_file) This WSL behavior is similar to the Linux CIFS file system, which is used when accessing Windows SMB shares.
Unlike VolFs, DrvFs does not store any additional file information. Instead, all inode attributes are generated based on the information that is used in Windows by querying the valid file attributes, valid permissions, and other information. DrvFs also prohibits the use of a special directory entry cache. This is done to keep it up to date, even if one of the other Windows processes modifies the contents of the directory. Thus, there are no restrictions on what Windows processes can do with files, as long as DrvFs also has access to them. DrvFs also uses Windows semantics when deleting files, so the file cannot be unlinked if there are any open descriptors on it.
ProcFs and SysFs
In the case of Linux, these types of special directories do not work with disk files, instead providing the information that the kernel of the OS has about running processes, threads, and devices used. These directories are dynamically generated when the client tries to read them. In some cases, the information for these directories is stored entirely in LXCore.sys memory. In other cases, for example, using a microprocessor by any of the processes, WSL requests this information from the Windows kernel. However, in both cases, plugins do not interact with Windows disk file systems.
This post will be about implementationVFS virtual file system in WSL, which is used as an abstraction layer in Linux when accessing both disk files and other OS objects, including devices, ports, processes, a microprocessor, etc. Since the Windows 10 kernel has the structure of subsystems and originally designed to implement various types of environments, including POSIX, the VX semantics driver LXCore.sys addresses these kernel subsystems by implementing the corresponding semantics and directories like / dev, / proc, / sys.
Fig. General scheme for implementing VFS in WSL. At the top level are Linux applications that call the OS API, the kernel mode driver LXCore.sys is responsible for their implementation. To implement VFS semantics, the driver accesses various subsystems of the Windows kernel, including the object manager (ObMgr) for working with the Windows name system and the object manager namespace, the I / O manager (IoMgr) for implementing the / dev directory, as well as the file system driver NTFS for working with disk files.
WSL provides Linux applications with the necessary VFS semantics, while supporting the permissions system for files and directories, symbolic links, special FIFO files, as well as the above directories / dev, / proc, / sys, etc. When an application calls one of the functions like open, read , chmod , stat , the corresponding system call is taken to process it and transfers control to the VFS implementation layer (LXCore.sys). Further, when processing file paths (for example, when executing the open and stat functions ), VFS converts it to the internal format using a special cache (directory entry cache). In the absence of the required path element in the cache, one or several special plug-ins are called (see below) to create the well-known inode structure of this element. This structure will represent the file opened by the application in WSL.
Windows does not have an inode type structure to represent an open file descriptor; instead, a well-known structure called file object is used , which stores some information about the file (size, attributes, access mask, etc.). LXCore.sys relies on it internally when implementing inode. However, both Linux and Windows use file descriptors to represent an open file, details of which LXCore.sys hides. It defines the levels of the so-called plug-ins under the names VolFs and DrvFs, which are responsible for working with disk files, TmpFs for working with in-memory file system data, as well as pseudo-FS ProcFs, SysFs, and CgroupFs.
Volfs
The VolFs plugin is used by VFS to work with the disk file system, it is used to store Linux system files, as well as the contents of the / home directory. VolFs supports Linux file permissions, symbolic links, sockets, device files, and FIFO. The / root and / home directories are mounted in the% LocalAppData% \ lxss \ root and% LocalAppData% \ lxss \ home directories. When you remove the WSL subsystem, the files stored in these directories are not deleted. As you can see from the path, the Linux directory mount directories are user-specific, that is, each Windows user has its own WSL directory. Therefore, installing Linux applications in WSL by one user will not affect others.
WSL provides applications with two features of the Linux file system that are not directly supported on Windows. The first such feature is the sensitivity of the FS to the case of file names, in which case VolFs simply accesses the object manager, indicating a specific flag for the operation. Since NTFS internally distinguishes case-sensitive characters for file names, implementing this feature is a fairly simple task.
The second feature is support for almost all possible characters as applicable to file names. Windows has stricter restrictions on file names, with some characters in the file names being invalid, while others may have special meanings, such as the colon ":", representing an alternate stream of data in the file. When implementing this feature, VolFS avoids the use of invalid characters in names.
Linux inodes have a number of attributes that are missing from Windows, including information about the owner and group, as well as mode. These attributes are stored in the NTFS Ea extended attribute structure, which are associated with files on disk. The following information is stored in this NTFS Ea file attribute:
- File mode (mode): includes the file type (regular, symbolic link, FIFO, etc.), as well as permission bits for the file.
- Information about the owner: the identifier of the group and user of the owners of the file.
- Device ID: for files that represent devices, the major and minor device numbers. So far, WSL does not allow users to create device files on VolFS.
- File timestamps: timestamps of the last access to the file and its modification. Linux uses a different format for timestamps, so information about them is stored in extended attributes.
DrvFs
To facilitate interaction with Windows, WLS uses the DrvFs plugin. WSL automatically mounts all disk devices with supported file systems to the / mnt directory, for example, / mnt / c, / mnt / d. Volumes with NTFS and ReFS file systems are currently supported. DrvFs works in the same way as VolFs. When a stream opens a file descriptor, file object and inode structures are created. However, unlike VolFs, DrvFs adheres to Windows rules when working with the file system. It uses permissions for files from Windows, only legal NTFS file names, while special files such as FIFO and sockets are not allowed.
Linux is known to use a fairly simple permission model when a file owner, group, or anyone else is allowed to execute it, read or write data. Windows uses a more sophisticated model based on Access Control List (ACLs) that define complex access rules for each individual file or directory (Linux also has ACL support, but this feature is not currently supported by WSL).
When a file is opened through the DrvFs plugin, it uses the Windows permissions mechanism based on the access token of the user in the context of which the bash.exe shell process was launched. Thus, to access the files in the system directory C: \ Windows, it is not enough just to use the commandsudo , which can provide root privileges in WSL. The latter does not change the access token of the process, so to perform the specified operation, you must run the bash application with elevated privileges on Windows.
WSL can give the user a hint about the permissions that he has when accessing files, while DrvFs checks the current user permissions and converts them into read / write / execute bits, which can be seen when the ls -l command is executed. However, the possibility of direct conversion does not always exist; for example, Windows has separate permissions to create files or subdirectories in a directory. If the user has such permissions, DrvFs will indicate to him the presence of write-access to the directory, while some operations with it may not be available to him.
Since the specified WLS file access may vary depending on what rights the bash.exe process running on Windows has, the specified file permissions will change when switching between copies of bash.exe with different rights (one is run from a simple user, and second from the administrator). When calculating file permissions, DrvFs takes into account the read-only attribute. A file with the read-only attribute will be displayed in WSL as having no write permission. The chmod command can be used to set the read-only attribute (by deleting all write permissions, i.e. chmod aw some_file ) or by deleting it (by setting any write permissions, i.e. chmod u + w some_file) This WSL behavior is similar to the Linux CIFS file system, which is used when accessing Windows SMB shares.
Unlike VolFs, DrvFs does not store any additional file information. Instead, all inode attributes are generated based on the information that is used in Windows by querying the valid file attributes, valid permissions, and other information. DrvFs also prohibits the use of a special directory entry cache. This is done to keep it up to date, even if one of the other Windows processes modifies the contents of the directory. Thus, there are no restrictions on what Windows processes can do with files, as long as DrvFs also has access to them. DrvFs also uses Windows semantics when deleting files, so the file cannot be unlinked if there are any open descriptors on it.
ProcFs and SysFs
In the case of Linux, these types of special directories do not work with disk files, instead providing the information that the kernel of the OS has about running processes, threads, and devices used. These directories are dynamically generated when the client tries to read them. In some cases, the information for these directories is stored entirely in LXCore.sys memory. In other cases, for example, using a microprocessor by any of the processes, WSL requests this information from the Windows kernel. However, in both cases, plugins do not interact with Windows disk file systems.