Isolating daemons with systemd or “you don’t need Docker for this!”

Recently, I have seen a fairly large number of people use container virtualization only to lock up a potentially unsafe application inside a container. As a rule, they use Docker for this because of its prevalence, and do not know anything better. Indeed, many daemons are initially launched as root, and then either lower their privileges or the master process spawns processing processes with reduced privileges. And there are those that work exclusively from root. If a vulnerability is found in the daemon that allows access with maximum privileges, it will not be very pleasant to detect intruders who have already managed to download all the data and leave the viruses.
The containerization provided by Docker and other similar software really saves from this problem, but it also brings new ones: you need to create a container for each daemon, take care of the safety of the changed files, update the base image, and the containers themselves are often based on different OSs that are necessary store on disk, although you, in general, are not particularly needed. What to do if you don’t need containers as such, the application is not built in the Docker Hub the way you need, and the version is deprecated, SELinux and AppArmor seem too complicated for you, and you would like to run it in your environment, but using the same Is the isolation that Docker uses?

Capabilities

What is the difference between a regular user and root? Why can root manage the network, load kernel modules, mount file systems, kill processes of any users, and the ordinary user is deprived of such opportunities? It's all about capabilities - a tool for managing privileges. All these privileges are given to a user with UID 0 (i.e. root) by default, and a regular user does not have any of them. Privileges can be either given or taken away. So, for example, the usual ping command requires creating a RAW socket, which cannot be done on behalf of a regular user. Historically, an SUID flag was set on ping, which simply ran the program on behalf of the superuser, but now all modern distributions set CAP_NET_RAW capability, which allows you to run ping from under any account.
You can get the list of installed capabilities of a file using the command getcapfrom libcap.

% getcap $(which ping)
/usr/bin/ping = cap_net_raw+ep

The p flag here means permitted , i.e. the application has the ability to use the given capability, e means effective - the application will use it, and there is also the i flag - inheritable , which makes it possible to save the capabilities list when the function is called execve().
Capabilities can be set both on the FS level and simply on a separate program stream. It is impossible to obtain capability, which has not been available since launch, i.e. privileges can only be reduced, but not increased.
There are also Secure Bits, there are three of them: KEEP_CAPS allows you to save capability when calling setuid, NO_SETUID_FIXUP disables the reconfiguration of capability when calling setuid, and NOROOT prohibits the issuance of additional privileges when starting suid programs.

Namespaces

The ability to put an application into your namespaces is another Linux kernel feature. Separate namespaces can be defined for:

• File system
• UTS (hostname)
• System V IPC (interprocess communication)
• Networks
• PID
• Users

If we put the application, for example, in a separate network space, it will not be able to see our network adapters that are visible from the host. The same thing can be done with the file system.

systemd

Fortunately, systemd supports everything you need to isolate applications and distinguish between rights.
We will use these features, but first we’ll think a bit about what rights our application needs.
So what are demons? There are those that generally do not need superuser rights, and they only use them to listen to a port below 1024. It is enough to issue such programs capability CAP_NET_BIND_SERVICE, which will allow them to listen to any ports without restrictions, and immediately start them from an unprivileged user. You can install capability on a file with the command setcap. As an experimental “service” we will have ncat from nmap, which will give shell access to anyone who wants it - you can’t imagine worse:

% sudo setcap CAP_NET_BIND_SERVICE=ep /usr/bin/ncat
% getcap /usr/bin/ncat
/usr/bin/ncat = cap_net_bind_service+ep

Now we write the simplest systemd unit that will run ncat with the necessary parameters on port 81 on behalf of the nobody user:

[Unit]
Description=Vuln
[Service]
User=nobody
ExecStart=/usr/bin/ncat --exec /bin/bash -l 81 --keep-open --allow ::1

We save it in /etc/systemd/system/vuln.serviceand run the usual sudo systemctl start vuln .
We are connected to it:

% ncat ::1 81
whoami
nobody

Works great!
It is time to protect our service, for this systemd has the following directives:

• CapabilityBoundingSet = - controls the capabilities. Sets only those that were transferred in this parameter, or vice versa, takes the transferred ones if the tilde character "~" is before the first.
• SecureBits = - sets the security bits.
• Capabilities = - also controls capabilities, but in such a way that the capabilities specified in the file at the FS level have an advantage, so it’s practically useless.
• ReadWriteDirectories =, ReadOnlyDirectories =, InaccessibleDirectories = - manage the file system namespace. Remount the file system inside the daemon namespace so that the specified directories are read / write, read-only, or not accessible at all (they become empty).
• PrivateTmp = - remounts / tmp and / var / tmp in their own tmpfs inside namespace.
• PrivateDevices = - selects access to devices from / dev, leaving access only to standard devices, such as / dev / null, / dev / zero, / dev / random and others.
• PrivateNetwork = - Creates an empty network namespace with a single lo interface.
• ProtectSystem = - mounts / usr and / boot into read-only mode, and when passing the “full” argument, does the same with / etc.
• ProtectHome = - makes the / home, / root and / run / user directories inaccessible, or remount them in read-only mode with the “read-only” parameter
• NoNewPrivileges = - allows you to make sure that the application does not receive additional privileges. According to the authors, it is more powerful than the corresponding capability.
• SystemCallFilter = - filters system calls using seccomp technology. More about this later.

Let's rewrite our unit file using these options:

[Unit]
Description=Vuln
[Service]
User=nobody
ExecStart=/usr/bin/ncat --exec /bin/bash -l 81 --keep-open --allow ::1
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
InaccessibleDirectories=/sys
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=full

So, we gave our application one capability CAP_NET_BIND_SERVICE, created separate / tmp and / var / tmp, selected access to devices and home directories, remounted / usr, / boot and / etc in read-only mode and separately blocked / sys, t .to. a typical daemon is unlikely to get there, and all this is done on behalf of the user.
It should be noted that CapabilityBoundingSet does not allow even suid-applications like su or sudo to get additional capabilities, so we will not be able to access on behalf of another user or root, even knowing their passwords, because the kernel will not allow setuid and setgid calls:

% ncat ::1 81           
python -c 'import pty; pty.spawn("/bin/bash")'   # создает новый pty, без него не получится использовать sudo или su
[nobody@valaptop /]$ sudo -i    # запрет setuid() и setgid()
sudo: unable to change to root gid: Operation not permitted
sudo: unable to initialize policy plugin
[nobody@valaptop /]$ ping   # запрет получения capability cap_net_raw
bash: /usr/sbin/ping: Operation not permitted
[nobody@valaptop /]$ cd /home
bash: cd: /home: Permission denied
[nobody@valaptop /]$ ls -lad /home
d--------- 2 root root 40 Nov  3 11:46 /home
[nobody@valaptop tmp]$ ls -la /tmp
total 4
drwxrwxrwt  2 root root   40 Nov  5 00:31 .
drwxr-xr-x 19 root root 4096 Nov  3 22:28 ..

Consider the second type of daemon, those that run as root and lower their privileges. This approach is used for many purposes: reading confidential files that are accessible only from the superuser (for example, a private key for using TLS by the web server), maintaining logs that will not be available if the non-root fork is compromised, and just applications that are arbitrary change UID (ssh-servers, ftp-servers). If such programs are not isolated, then the worst thing that can happen is that the attacker will gain full access on behalf of the superuser. Although the lack of capabilities inherent in root makes him an almost ordinary unprivileged user, root still remains root with a bunch of files belonging to him that he can read,

[Unit]
Description=Vuln
[Service]
ExecStart=/usr/bin/ncat --exec /bin/bash -l 81 --keep-open --allow ::1
CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_SETUID CAP_SETGID
NoNewPrivileges=yes
InaccessibleDirectories=/sys
InaccessibleDirectories=/etc/openvpn
InaccessibleDirectories=/etc/strongswan
InaccessibleDirectories=/etc/nginx
ReadOnlyDirectories=/proc
PrivateTmp=true
PrivateDevices=true
ProtectHome=true
ProtectSystem=full

Here we added capability CAP_SETUID and CAP_SETGID so that our daemon could lower privileges, use NoNewPrivileges so that it could not increase its capabilities, block access to directories that it should not read, and allow read only access to / proc so that sysctl could not be used. You can also mount the entire root at once in read-only, and write permissions only in those directories that the program uses.
You should separately verify the permissions of the / etc / shadow file . In modern distributions, it is not readable even by root, and the capability CAP_DAC_OVERRIDE is used to work with it, which allows you to ignore access rights.

% ls -la /etc/shadow
---------- 1 root root 1214 ноя  3 19:57 /etc/shadow

Check our settings!

python -c 'import pty; pty.spawn("/bin/bash")'   # создает новый pty
[root@valaptop /]# whoami
root
[root@valaptop /]# ping   # запрет получения capability cap_net_raw
bash: /usr/sbin/ping: Operation not permitted
[root@valaptop /]# cat /etc/shadow   # нет CAP_DAC_OVERRIDE
cat: /etc/shadow: Permission denied
[root@valaptop /]# cd /etc/openvpn
bash: cd: /etc/openvpn: Permission denied
[root@valaptop /]# /suid   # SUID shell
[root@valaptop /]# cat /etc/shadow   # уже из-под нового shell, прав не прибавилось
cat: /etc/shadow: Permission denied

Unfortunately, systemd (so far) does not know how to work with the PID namespace, so our root daemon can kill other programs running as root.
In general, this can be finished, capabilities and namespace settings do a good job of isolating applications, but there is one more thing that would be great to configure.

seccomp

Seccomp technology prevents the program from making certain system calls, killing it immediately when trying to do it. Although seccomp appeared a long time ago, in 2005, it really began to be used relatively recently, with the release of Chrome 20, vsftpd 3.0 and OpenSSH 6.0.
There are two approaches to using seccomp: blacklist and whitelist. Blacklisting potentially dangerous calls is noticeably easier than white, so this approach is used more often. The firejail project prohibits the following syscalls from executing by default (the tilde enables the blacklist mode):

SystemCallFilter=~mount umount2 ptrace kexec_load open_by_handle_at init_module \
finit_module delete_module iopl ioperm swapon swapoff \
syslog process_vm_readv process_vm_writev \
sysfs_sysctl adjtimex clock_adjtime lookup_dcookie \
perf_event_open fanotify_init kcmp add_key request_key \
keyctl uselib acct modify_ldt pivot_root io_setup \
io_destroy io_getevents io_submit io_cancel \
remap_file_pages mbind get_mempolicy set_mempolicy \
migrate_pages move_pages vmsplice perf_event_open

In systemd prior to version 227 inclusive, there is a bug that requires setting NoNewPrivileges = true to use seccomp.
The whitelist can be compiled as follows:

1. Run the required program under strace:
   

   % strace -qcf nginx

   
   We get a large table of syscalls:
   

    time     seconds  usecs/call     calls    errors syscall
   ------ ----------- ----------- --------- --------- ----------------
     0.00    0.000000           0        24           read
     0.00    0.000000           0        27           open
     0.00    0.000000           0        32           close
     0.00    0.000000           0         6           stat
   …
     0.00    0.000000           0         1           set_tid_address
     0.00    0.000000           0         4           epoll_ctl
     0.00    0.000000           0         3           set_robust_list
     0.00    0.000000           0         2           eventfd2

   
   
2. We rewrite them all, set as SystemCallFilter . Most likely, your application will crash because strace did not find all the calls. We look at the execution of which call the application terminated in the logs of the audit daemon:
   

   type=SECCOMP msg=audit(1446730375.597:7943724): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=11915 comm="(nginx)" exe="/usr/lib/systemd/systemd" sig=31 arch=40000003 syscall=191 compat=0 ip=0xb75e5be8 code=0x0

   The number of the syscall we need is 191. We open the call table and look for the name of this call by number.
3. Add it to the allowed calls. In case of a fall, we return to point 2.

Tips & Tricks

You can check the current privileges and the possibility of increasing them using the captest command .
filecap displays a list of files with installed capabilities.
Using netcap, you can get a list of running network programs that have at least one socket and one capability, and pscap will display not only the network running software.
It is not necessary to completely edit the systemd unit and track its changes during the upgrade, but it is better to add the necessary directives through systemctl edit .