Black mark - how OpenShift protects against container vulnerabilities with SELinux
Have you ever done a difficult job for the good of society, but didn’t notice your efforts, because you have benefited for so long that you are all used to it? This is the kind of work for you that all SELinux community members do .
And on February 18 this year, thanks in large part to their work, the world was saved from the dangerous container-hitting vulnerability CVE-2019-5736 .
Although there are other operating systems and other open source projects that use type and category controls, it is rare that all components configured with SELinux are included out of the box, by default, and ready to go. Even more rarely, this configuration covers all levels, right up to the solution for container orchestration, on the basis of which the public cloud works.
For eight years now, Red Hat OpenShift has been using Linux forced access control mechanisms such as type enforcement (TE) and multi-category security (MCS). SELinux has been used in OpenShift since 2011. At Red Hat OpenShift Online, a publicly available hosting service where thousands of developers run container code daily, SELinux has been used from the start. What about the version of OpenShift that is used, for example, in the data center of your favorite mobile operator? In fact, the SELinux security module is included in the Red Hat OpenShift Container Platform by default, by default! And not just turned on, but fully configured and ready to protect against real threats.
Unlike other Kubernetes distributions, Red Hat closes the gap between Linux and the container orchestration platform installed on top of it. That is, Red Hat OpenShift monitors and eliminates security threats throughout the stack, and not just in one layer. And this is done by default - from the first day of work.
OpenShift uses this configuration by default in Red Hat Enterprise Linux (you don’t even need to know what it is there). The matter is not limited to running setenforce 1 on a laptop. The access control rules for the types and categories that a tenant applies to working with containers on one Kubernetes cluster can be extended to hundreds of nodes that can be used by thousands of other tenants.
Think about how the SELinux configuration with MCS will look after a few years of use in a large company that distributes OpenShift credentials left and right. Now imagine that you provide your credentials to log into your OpenShift cluster, as we do at openshift.com. SELinux loyalty often fails to be recognized for everything it does in an OpenShift solution. If it seems to you that the operating system is not so important these days, consider whether you were protected from the CVE-2019-5736 vulnerability until this February. In OpenShift, CVE-2019-5736 is protected from the very beginning, and you can move on to this solution now .
One of the most effective default security features implemented in Red Hat OpenShift is Security-Enhanced Linux (SELinux). SELinux is a Linux kernel security module that provides security policy-based access control. The way SELinux works is to assign labels (names) to all processes and objects of the operating system. Thus, each element involved in kernel operations is marked and classified, and then it is granted access based on an existing set of rules.
Policy rules define the relationship between tagged processes and tagged objects. Rules defined by the user in the policies are applied at the kernel level. By default, anything that is not allowed is automatically denied - by analogy with a firewall that denies access to all processes for which explicit permissions are not configured. The illustrations below illustrate simple use cases.
Imagine a system in which you need to define types for objects such as cats and dogs. Cat and dog are types of processes.
* The author of all the drawings is Máirín Duffy.
The class of objects with which the processes will interact is feed. Add feed types: cat_chow and dog_chow (Ohm-nom-nom).
We set permission for the dog to eat dog food (dog_chow), and for the cat - cat food (cat_chow). We write these permissions as a SELinux policy rule:
According to these rules, the “cat” process will be allowed to eat food with the cat_chow label at the kernel level, and the dog with dog_chow label to be allowed to eat food.
But we remember that in SELinux, everything is prohibited by default. Therefore, if the dog tries to eat cat_chow, the core will not allow it.
This is type control, which plays a major role in protecting the host system from container processes. Container processes can only read and run files from the / usr directory and write data only to container files. This restriction reliably protects the host from containers, but does not protect some containers from others, because they are all marked with the same type. To protect containers from each other, you will need to implement MCS tag control.
The use of MCS is not directly related to the protection of OpenShift from CVE-2019-5736 , but it is useful to familiarize yourself with this topic in order to better understand the principles of using SELinux in OpenShift. Assigning MCS labels from the point of view of the user or system administrator is quite simple. You just need to configure a set of categories that are simple text labels (for example, Fido or Spot), and add users to them. The system administrator first configures the categories and then adds users to them, after which users can use these labels as they wish. This is convenient because MCS allows you to use standard SELinux tags for managing objects. Let us again turn to the imaginary system from the example above.
Add another part of the label that will be applied to the dog process and dog_chow feed. Assign the process “dog” to dog: random1 (Fido) and dog: random2 (Spot).
Let's assign the dog_chow: random1 (Fido) and dog_chow: random2 (Spot) labels to the dog food.
According to the principles of operation of MCS, if the rules of forced control are met by type and arbitrary MCS tags exactly match, then access is allowed, and in all other cases it is denied.
Attempts by Fido (dog: random1) to eat cat_chow: food will be denied due to forced type controls.
The default SELinux security module used by OpenShift is a prime example of defense in depth. OpenShift, like many other Kubernetes-based platforms, uses SCC / PSP policies that prohibit running containers with root privileges. This limitation also protects against CVE-2019-5736 . In OpenShift, containers that are owned by the root user are blocked by default, but this parameter can be changed . Even if you allow containers to run as root, the standard SELinux configuration in OpenShift still protects against CVE-2019-5736. This is another level of protection that really pays off in this situation and it is far from the only one in OpenShift. Further information can be found in the document 10 levels of container security .
Further information on CVE-2019-5736 , including the Red Hat Enterprise Linux patch for the container runtime, can be found in the Red Hat vulnerability review .
And on February 18 this year, thanks in large part to their work, the world was saved from the dangerous container-hitting vulnerability CVE-2019-5736 .
Although there are other operating systems and other open source projects that use type and category controls, it is rare that all components configured with SELinux are included out of the box, by default, and ready to go. Even more rarely, this configuration covers all levels, right up to the solution for container orchestration, on the basis of which the public cloud works.
For eight years now, Red Hat OpenShift has been using Linux forced access control mechanisms such as type enforcement (TE) and multi-category security (MCS). SELinux has been used in OpenShift since 2011. At Red Hat OpenShift Online, a publicly available hosting service where thousands of developers run container code daily, SELinux has been used from the start. What about the version of OpenShift that is used, for example, in the data center of your favorite mobile operator? In fact, the SELinux security module is included in the Red Hat OpenShift Container Platform by default, by default! And not just turned on, but fully configured and ready to protect against real threats.
Unlike other Kubernetes distributions, Red Hat closes the gap between Linux and the container orchestration platform installed on top of it. That is, Red Hat OpenShift monitors and eliminates security threats throughout the stack, and not just in one layer. And this is done by default - from the first day of work.
OpenShift uses this configuration by default in Red Hat Enterprise Linux (you don’t even need to know what it is there). The matter is not limited to running setenforce 1 on a laptop. The access control rules for the types and categories that a tenant applies to working with containers on one Kubernetes cluster can be extended to hundreds of nodes that can be used by thousands of other tenants.
Think about how the SELinux configuration with MCS will look after a few years of use in a large company that distributes OpenShift credentials left and right. Now imagine that you provide your credentials to log into your OpenShift cluster, as we do at openshift.com. SELinux loyalty often fails to be recognized for everything it does in an OpenShift solution. If it seems to you that the operating system is not so important these days, consider whether you were protected from the CVE-2019-5736 vulnerability until this February. In OpenShift, CVE-2019-5736 is protected from the very beginning, and you can move on to this solution now .
SELinux marks
One of the most effective default security features implemented in Red Hat OpenShift is Security-Enhanced Linux (SELinux). SELinux is a Linux kernel security module that provides security policy-based access control. The way SELinux works is to assign labels (names) to all processes and objects of the operating system. Thus, each element involved in kernel operations is marked and classified, and then it is granted access based on an existing set of rules.
Policy rules define the relationship between tagged processes and tagged objects. Rules defined by the user in the policies are applied at the kernel level. By default, anything that is not allowed is automatically denied - by analogy with a firewall that denies access to all processes for which explicit permissions are not configured. The illustrations below illustrate simple use cases.
Imagine a system in which you need to define types for objects such as cats and dogs. Cat and dog are types of processes.
* The author of all the drawings is Máirín Duffy.
The class of objects with which the processes will interact is feed. Add feed types: cat_chow and dog_chow (Ohm-nom-nom).
We set permission for the dog to eat dog food (dog_chow), and for the cat - cat food (cat_chow). We write these permissions as a SELinux policy rule:
allow cat cat_chow:food eat;
allow dog dog_chow:food eat;
According to these rules, the “cat” process will be allowed to eat food with the cat_chow label at the kernel level, and the dog with dog_chow label to be allowed to eat food.
But we remember that in SELinux, everything is prohibited by default. Therefore, if the dog tries to eat cat_chow, the core will not allow it.
This is type control, which plays a major role in protecting the host system from container processes. Container processes can only read and run files from the / usr directory and write data only to container files. This restriction reliably protects the host from containers, but does not protect some containers from others, because they are all marked with the same type. To protect containers from each other, you will need to implement MCS tag control.
MCS doesn't pull a cat by the tail
The use of MCS is not directly related to the protection of OpenShift from CVE-2019-5736 , but it is useful to familiarize yourself with this topic in order to better understand the principles of using SELinux in OpenShift. Assigning MCS labels from the point of view of the user or system administrator is quite simple. You just need to configure a set of categories that are simple text labels (for example, Fido or Spot), and add users to them. The system administrator first configures the categories and then adds users to them, after which users can use these labels as they wish. This is convenient because MCS allows you to use standard SELinux tags for managing objects. Let us again turn to the imaginary system from the example above.
Add another part of the label that will be applied to the dog process and dog_chow feed. Assign the process “dog” to dog: random1 (Fido) and dog: random2 (Spot).
Let's assign the dog_chow: random1 (Fido) and dog_chow: random2 (Spot) labels to the dog food.
According to the principles of operation of MCS, if the rules of forced control are met by type and arbitrary MCS tags exactly match, then access is allowed, and in all other cases it is denied.
Attempts by Fido (dog: random1) to eat cat_chow: food will be denied due to forced type controls.
Defense in depth
The default SELinux security module used by OpenShift is a prime example of defense in depth. OpenShift, like many other Kubernetes-based platforms, uses SCC / PSP policies that prohibit running containers with root privileges. This limitation also protects against CVE-2019-5736 . In OpenShift, containers that are owned by the root user are blocked by default, but this parameter can be changed . Even if you allow containers to run as root, the standard SELinux configuration in OpenShift still protects against CVE-2019-5736. This is another level of protection that really pays off in this situation and it is far from the only one in OpenShift. Further information can be found in the document 10 levels of container security .
Further information on CVE-2019-5736 , including the Red Hat Enterprise Linux patch for the container runtime, can be found in the Red Hat vulnerability review .