
Security Monitoring with Sysdig Falco
- Tutorial

About Sysdig — a tool for tracing a kernel — we talked about two years ago . More recently, in May of this year, Sysdig developers presented another interesting product: Falco anomaly detection system .
Falco consists of two main components: the sysdig_probe kernel module (which also runs Sysdig) and the daemon that writes the collected information to disk.
Based on user-defined rules, Falco monitors the operation of applications and, upon detection of anomalies, writes information to standard output, syslog or a user-specified file. Developers in his blog jokingly called Falco «hybrid snort, ossec and strace» and position it as a simple IDS, almost does not overload the system.
We would briefly describe Sysdig Falco in a slightly different way: it is an advanced audit tool. It can track the same events as the Linux audit subsystem - but not only. Here is a far from complete list:
- launching the command shell inside the container;
- writing to the / dev directory files that have nothing to do with devices (for example, some rootkits do this);
- atypical network connections initiated by applications;
- Attempts to modify files in critical directories - for example, / etc / passwd;
- atypical events in the operation of individual applications.
Falco itself does not provide any protection, but only collects information about system events that meet specified conditions. Based on this information, it is possible to draw certain conclusions and, if necessary, take additional measures.
Installation
Before installing Falco, you need to add the appropriate repository (hereinafter all example commands are provided for the Ubuntu 16.04 OS):
$ curl -s https://s3.amazonaws.com/download.draios.com/DRAIOS-GPG-KEY.public | apt-key add -
$ curl -s -o /etc/apt/sources.list.d/draios.list http://download.draios.com/stable/deb/draios.list
$ sudo apt-get update
We also need to set the kernel headers:
$ sudo apt-get -y install linux-headers-$(uname -r)
After that, install Falco and add the sysdig_probe module to the kernel:
$ sudo apt-get -y install falco
$ modprobe sysdig-probe
This completes the installation. After that, Falco can run:
$ sudo service falco start
Information about all detected events will be recorded in syslog. You can run Falco in interactive mode:
$ falco
All information about suspicious events will be immediately written to standard output.
The default settings and rules for getting started will be more than enough.
The /etc/falco_rules.yaml file already contains rules for all occasions. There are even ready-made rules for a wide variety of applications and services: MySQL, MongoDB, CouchDB, Fluentd, Elasticsearch and others.
If necessary, you can always change existing rules and even add new ones. Let us examine the structure of Falco configuration files in more detail.
Basic settings
Basic Falco settings are stored in the /etc/falco.yaml file. By default, it looks like this:
# File containing Falco rules, loaded at startup.
rules_file: /etc/falco_rules.yaml
# Whether to output events in json or text
json_output: false
# Send information logs to stderr and/or syslog Note these are *not* security
# notification logs! These are just Falco lifecycle (and possibly error) logs.
log_stderr: true
log_syslog: true
# Where security notifications should go.
# Multiple outputs can be enabled.
syslog_output:
enabled: true
file_output:
enabled: false
filename: ./events.txt
stdout_output:
enabled: true
program_output:
enabled: false
program: mail -s "Falco Notification" someone@example.com
As you can see, it is indicated here in which file the rule is stored, in which format the output should be presented (plain text or json) and where to record information about the detected anomalies. Falco can write messages to standard output, to syslog, as well as to a user-specified text file.
Rules and their syntax
The /etc/falco_rules.yaml file contains rules indicating which specific features in the behavior of the Sysdig Falco system should pay special attention to. Here is a snippet of this file:
- rule: write_etc
desc: an attempt to write to any file below /etc, not in a pipe installer session
condition: write_etc_common and not proc.sname=fbash
output: "File below /etc opened for writing (user=%user.name command=%proc.cmdline file=%fd.name)"
priority: WARNING
Everything is simple and clear here: the rule indicates that Falco should inform about any attempt to open any file in the / etc directory for writing (unless the files in / etc are created during the installation of the programs.
Each rule consists of the following fields:
- desc - description of the rule in any form;
- condition - the condition under which the rule is triggered (the standard Sysdig syntax is used to write the conditions; for more details, see the official documentation) , as well as in our article );
- output - the output that will be output when the rule is triggered;
- prioriry - priority (INFO, WARNING, ALERT, DEBUG, CRITICAL).
Let's see how this rule works. Launch Falco in the mode of "live observation":
$ falco
In another terminal, try to open some file in the / etc directory. We will see that messages of the form:
12:43:52.640375428: Warning File below /etc opened for writing (user=useri command=nano /etc/default/grub file=/etc/default/.grub.swp)
12:43:52.640973730: Warning File below /etc opened for writing (user=useri command=nano /etc/default/grub file=/etc/default/grub)
If we stop Falco by pressing Ctrl + C, a brief summary of all detected events will be displayed on the console:
Events detected: 2
Rule counts by severity:
Error: 0
Warning: 2
Informational: 0
Triggered rules by rule name:
write_etc: 2
Let's look at another example and show how Falco can be used to audit system events in containers.
Watching the container
Sysdig Falco is well suited for monitoring what happens inside containers. Let's see how it works.
Create a Docker container:
$ docker pull:ubuntu 14.04
After that, add an additional rule to /etc/falco_rules.yaml (an example is taken from here ):
- rule: system_binaries_network_activity_container
desc: any network activity performed by system binaries that are not expected to send or receive any network traffic in a container
condition: ((inbound or outbound) and (fd.sockfamily = ip)) and fd.name != '' and container
output: "Suspicious binary sent/received network traffic from container=%container.id (user=%user.name command=%proc.cmdlin
e connection=%fd.name type=%evt.type)"
priority: WARNING
Save the changes and restart Falco. After that, enter the container:
$ docker run --rm -it ubuntu:14.04 /bin/bash
Let's execute the command in the container:
$ ping ya.ru
The following messages appear on the main host in syslog:
16:08:56.944164593: Warning Suspicious binary sent/received network traffic from container=0b86d8efdf0a (user=root command=ping ya.ru connection=172.17.0.2:47776->123.45.67.89:53 type=connect)
16:08:56.945398068: Warning Suspicious binary sent/received network traffic from container=0b86d8efdf0a (user=root command=ping ya.ru connection=172.17.0.2:38643->123.45.67.89:1025 type=connect)
They contain the container id, username and command, as a result of which a network connection was initiated.
Conclusion
Sysdig Falco is an interesting and promising tool. It has the same advantages as Sysdig: flexibility, convenient rules syntax, human-readable form of conclusions. With its help, you can get a lot of valuable information about the system, which cannot be obtained using other tools.
If you already use Falco in practice, we invite you to share your experience in the comments.
In conclusion, we present a selection of useful links for those wishing to learn more:
- https://github.com/draios/falco - the official Falco repository on GitHub;
- https://github.com/draios/falco/wiki - official Falco documentation;
- https://sysdig.com/blog/tag/falco/ - a series of articles about Falco in the Draios company blog;
- https://www.youtube.com/watch?v=flcu2Mz9bpo - an overview lecture about Sysdig Falco.
If you for one reason or another can not leave comments here - welcome to our corporate blog .