Samba, ZFS, VirtioFS: Setting up a file server with ACL and XATTR

Integrating Samba and ZFS with VirtioFS: Resolving ACL and XATTR Challenges

Deploying a high-performance SMB file server with ZFS storage in a KVM virtualized environment using VirtioFS presents unique challenges, especially regarding the correct handling of Access Control Lists (ACLs) and extended attributes (xattr) to ensure compatibility with Windows clients. This article details the configuration process and how to overcome key obstacles, ensuring full functionality and stable operation of the file share.

Choosing VirtioFS for Storage Virtualization

When creating a Network Attached Storage (NAS) based on ZFS and then setting up shared folders via Samba, an efficient method for connecting the ZFS pool to the virtual machine (VM) is required. Initially, various approaches were considered, including NFS, but direct directory pass-through offered by the KVM hypervisor garnered particular attention. Among the available options, such as virtio-9p and VirtioFS, the latter was chosen.

virtio-9p was ruled out due to its instability, limited POSIX support, and poor performance, making it unsuitable for critical file server tasks. VirtioFS, in contrast, showed significantly greater potential thanks to its architecture, which allows virtual machines to directly access the host's file system with high performance.

However, despite its clear advantages, VirtioFS had its quirks. Standard virt-manager settings did not provide options to enable support for extended attributes (xattr) and Access Control Lists (ACLs), which are crucial for Samba to work correctly with Windows clients. Extended attributes allow storing additional file information, while ACLs define detailed access rights for users and groups, forming the basis of NTFS permissions in Windows. Without these features, full integration of Samba and ZFS would be impossible.

Configuring VirtioFS for ACL and XATTR Support

The main challenge was to ensure the virtiofsd daemon, running on the host, started with the necessary --xattr and --posix-acl parameters. Directly editing the VM's XML configuration via libvirt or Qemu proved ineffective, as it was not possible to correctly pass all required parameters for virtiofsd startup.

The solution was to use a wrapper – a simple script that launches virtiofsd with the desired flags. This approach bypassed libvirt's limitations and guaranteed the activation of full ACL and xattr support for the passed-through directory. The process of creating and applying the wrapper involves the following steps:

Creating the Wrapper Script:

Create a file /usr/libexec/virtiofsdfix with the following content:

```bash

#!/bin/bash

exec /usr/libexec/virtiofsd "$@" --xattr --posix-acl --sandbox none

```

The --sandbox none parameter was added during experimentation and is intended to disable isolation, which can be important in some cases.

Setting Permissions:

Grant execute permissions to the script and copy security contexts from the original virtiofsd daemon:

```bash

sudo chmod +x /usr/libexec/virtiofsdfix

sudo chcon --reference=/usr/libexec/virtiofsd /usr/libexec/virtiofsdfix

```

Modifying VM Configuration:

In the virtual machine's XML configuration (via virt-manager or virsh edit), specify the path to the created wrapper script as the main VirtioFS module.

After these steps, VirtioFS will correctly pass the host's file system into the virtual machine with active xattr and POSIX ACL support. This forms the basic foundation for further integration with ZFS and Samba.

ZFS Configuration for Correct ACL Operation

Even with VirtioFS correctly configured to pass extended attributes and POSIX ACLs, the ZFS file system requires specific configuration for full support of these features. To enable a ZFS dataset to correctly handle ACLs similar to those used in Windows (NTFS ACLs), several key properties must be set:

zfs set acltype=posixacl pool/smbfs:

This property enables support for standard Linux POSIX ACLs for the specified dataset. It allows commands like getfacl and setfacl to work correctly, which is a prerequisite for Samba.

zfs set aclinherit=passthrough pool/smbfs:

This property defines ACL inheritance behavior. The passthrough value ensures that when new files or directories are created within the dataset, they automatically inherit access rights from the parent directory. This is critical for maintaining consistent permissions within the file structure, especially in the context of Windows shares.

zfs set aclmode=passthrough pool/smbfs:

This property sets ZFS's non-interference mode for ACLs. In passthrough mode, ZFS does not attempt to interpret or modify ACLs but simply passes them through as is. This ensures maximum compatibility with external systems, such as Samba, which manage their own ACLs.

After configuring ZFS and VirtioFS, the only remaining step is to mount the passed-through directory inside the guest VM. This can be done by adding the appropriate line to /etc/fstab:

fs   /fs   virtiofs   rw,noatime,_netdev,nofail

After executing mount -a, the ZFS directory will be accessible in the VM at /fs, ready for Samba's use.

Samba Issues with `acl_xattr` and `security.NTACL`

With the basic VirtioFS and ZFS setup, the next step is Samba configuration. A typical setup for a local directory, including vfs objects = acl_xattr, works perfectly, allowing Samba to write extended attributes, including those used for Windows ACLs. However, when attempting to apply this configuration to a directory passed through VirtioFS, a problem arises: Samba refuses to write XATTR attributes, specifically security.NTACL.

The issue manifests as standard setfacl commands for POSIX ACLs working correctly on both local disk and VirtioFS, but writing Samba-specific attributes like security.NTACL fails. This prevents full functionality with Windows permissions, as Samba cannot save the necessary security descriptor information. Even disabling SELinux on the host did not yield the desired result.

Tests conducted with setfattr to simulate Samba attribute writing revealed a key problem. When attempting to write the security.NTACL attribute as a domain user, the operation failed for both a local file and a file on VirtioFS. However, when performing the same operation as a superuser (root), writing security.NTACL on a local disk succeeded, while on VirtioFS, it still failed.

This indicates a fundamental difference in how extended attributes are handled:

security.NTACL: This attribute resides in the security namespace and is used by Samba to store Windows permissions (NTFS ACLs). Writing to this namespace is strictly restricted and requires CAP_SYS_ADMIN privileges, equivalent to root rights. Regular users, even file owners, cannot directly modify or delete this attribute.
user.NTACL: This attribute resides in the user namespace. Any user with write permissions to the file can modify attributes in this namespace.

The primary reason for the failure with VirtioFS is that even with the --sandbox none parameter and an attempt to write as root (via sudo), VirtioFS, at the kernel level or its own implementation, apparently blocks writing to the security.NTACL namespace. This prevents the correct saving of Samba's Windows permissions.

Solution: Redirecting `security.NTACL` to `user.NTACL` and ZFS Snapshot Integration

Since Samba cannot correctly write attributes to the security.NTACL namespace via VirtioFS, the only viable solution is to redirect these attributes to the user.NTACL user namespace. While this is a workaround rather than a complete solution to the root VirtioFS problem, it allows Samba to store all necessary Windows permission information. It's important to note that this approach is acceptable when users interact with the server exclusively via SMB and do not have direct access to the host's or guest VM's file system.

To implement this solution, the following parameters must be added to the Samba configuration file (smb.conf):

[VirtioFS]
    path = /fs
    writable = yes
    admin users = "domain\admin-domain" "superadmin"

    vfs objects = acl_xattr shadow_copy2

    # user.NTACL
    acl_xattr:security_acl_name = user.NTACL
    acl_xattr:ignore system acls = yes


    shadow:basedir = /fs
    shadow:snapdir = .zfs/snapshot
    shadow:format = GMT-%Y.%m.%d_%H.%M.%S
    shadow:sort = desc
    shadow:localtime = yes

Here, the key lines are:

acl_xattr:security_acl_name = user.NTACL: Instructs Samba to use user.NTACL instead of security.NTACL for storing security descriptors.
acl_xattr:ignore system acls = yes: This parameter tells Samba to ignore system ACLs, relying on its own mechanisms for storing permissions.

Integrating ZFS Snapshots with Samba

To provide Shadow Copies functionality for Windows clients, allowing users to restore previous file versions, the shadow_copy2 module is used. It integrates with ZFS's snapshot capabilities. In the provided configuration:

vfs objects = acl_xattr shadow_copy2: Activates the shadow_copy2 module.
shadow:basedir = /fs: Specifies the base directory for shadow copies.
shadow:snapdir = .zfs/snapshot: Defines the subdirectory where ZFS stores its snapshots. For correct operation, the ZFS pool must have the listsnapshots=on property set (e.g., zpool set listsnapshots=on pool_name), which allows Samba to discover and display available snapshots.
shadow:format, shadow:sort, shadow:localtime: Configure the display format and sorting of snapshots.

Thus, this configuration allows for the creation of a fully functional Samba file server on ZFS via VirtioFS, ensuring correct operation with Windows ACLs and shadow copies, despite the initial difficulties with security.NTACL.

Key Takeaways

VirtioFS and ACL/xattr: For full Samba functionality with Windows ACLs via VirtioFS, the virtiofsd daemon must be launched with --xattr and --posix-acl parameters, often requiring a wrapper script.
ZFS Configuration: ZFS datasets used for the Samba share must be configured with acltype=posixacl, aclinherit=passthrough, and aclmode=passthrough properties for correct ACL handling.
security.NTACL Issue: Samba by default attempts to write Windows security descriptors to security.NTACL, which requires root privileges and can be blocked by VirtioFS.
Solution via user.NTACL: An effective workaround is to redirect Samba to use user.NTACL for storing security attributes via the acl_xattr:security_acl_name = user.NTACL parameter.
ZFS Snapshots and Samba: Integration of ZFS snapshots for Windows Shadow Copies is achieved through the shadow_copy2 module in Samba, requiring appropriate ZFS pool property configuration.