CFEngine

We continue the story about CFEngine started by user alex_www in two previous articles. This article will focus on the practice of using CFEngine and some nuances of its configuration in real-world conditions. To reduce the volume of the text, I assume that you know the basic concepts from the CFEngine world, perhaps even tried to use it somewhere. As a primer, I can recommend Diego Zamboni's book Learning CFEngine 3 , which is small, very understandable and read in one breath.

This article gives examples for setting up from scratch on Debian GNU / Linux using Git. If you want to supplement the article with examples for your favorite distributions and VCS, then send personal messages or express yourself in the comments. If possible, I will add them to the main text indicating authorship.

Installation

The easiest way to install CFEngine3 is to do this through the official repository. To do this, you first need to add the key with which the packages are signed:

cd /tmp
wget http://cfengine.com/pub/gpg.key
cat gpg.key # не ныряй в незнакомых местах
apt-key add gpg.key
rm gpg.key # чисто там, где не сорят

... and then add the repository and put CFE:

echo "deb http://cfengine.com/pub/apt community main" > /etc/apt/sources.list.d/cfengine-community.list
chmod 644 /etc/apt/sources.list.d/cfengine-community.list # некоторые любят umask 0700
apt-get update
apt-get install cfengine-community

Initialization

To initialize CFE, you must perform (for example, manually) the first start of cf-agent. If you often have to commission new servers, then installing CFE and bootstrap is best done from preseed or from a script that runs once at the first start of the system; A good example of such a script is a script that creates server keys for OpenSSH.

/var/cfengine/bin/cf-agent -IC --bootstrap 

Instead, you need to substitute the appropriate IP address or domain name of your policy hub, and if you initialize the policy hub itself, you need to specify the IP on which it will serve clients later. In the case of a domain name, it will be resolved to an IP address, which will be used in the future, so that problems with the possible unavailability of DNS servers will not become an obstacle.

The parameter -Iincludes the creation of a brief report on the implementation, and -C- coloring the output in the console. Both are optional, but I use them in interactive sessions for my own convenience. Another useful launch option is -vverbose mode. It takes precedence over -Iand provides very detailed information on the progress of the promises. Great for debugging.

Initial setup

After the initialization of the policy hub, even before connecting to the first client, you need to configure some little things. The fact is that the default parameters in the file def.cf(hereinafter for all relative paths are considered to be the root /var/cfengine/masterfiles, unless otherwise indicated) are well suited for “on-the-knee” experiments or to demonstrate capabilities, however, these parameters are not suitable for selling. Below, I’ll write about the organization of the process of developing promises and rolling them into products, but for now, it’s best to make a local copy /var/cfengine/masterfilesand work with it.

First of all, we indicate our fully qualified domain name. Despite the efforts of the development team to make a good system for automatically determining the current parameters of the system, it is imperfect, so wherever you can and in the best sense it is best not to rely on automation and specify data manually. To begin with bundle common def, and specify domain, mailto, mailfromand smtpserver:

'domain'  string => 'example.org';
'mailto' string => 'sysadmin-queue@${def.domain}';  # на этот адрес CFE будет пытаться слать сообщения в случае необходимости
'mailfrom' string => 'root@cfe-policy-server.${def.domain}';
'smtpserver' string => 'internal-mail-collector.${def.domain}';

Security

CFE can provide connection security and client authentication, however, it does not have any exclusively developed means for this, since there are enough available for the eyes to complete the task. One way or another, in communication with clients, a system with trusted keys is used, similar to OpenSSH, and lists of trusted IP networks and domains. By default, connections are allowed for all hosts in the policy hub domain and for / 16 of its primary IP. All keys received through successfully established connections are considered trusted. Such an approach in a trusted network environment can greatly facilitate deployment from scratch and is very convenient at the R&D stage, but is extremely unsafe in real life. Given the ephemerality of IP addresses and the geographical distribution of controlled machines, I prefer to use the following approach: CFEngine (cf-serverd, to be more precise) accepts connections from any IP address and trusts only those keys that are known to it in advance (that is, are in /var/cfengine/ppkeys):

  'acl'     slist => {
      '0.0.0.0/0',
  };
  comment => 'Connections are allowed from any IP',
  handle => 'common_def_vars_acl';
  'trustkeysfrom' slist => {
        # NEVER ADD ANYTHING HERE. DON'T TRUST STRANGERS!
  },
  comment => 'Only keys in /var/cfengine/ppkeys are trusted',
  handle => 'common_def_vars_truskeysfrom';

In the case when you need to restrict access by IP addresses, I prefer to use a firewall as a more suitable tool for solving the problem.

To add the client key to the trusted ones, you need to copy the contents of the file /var/cfengine/ppkeys/localhost.pub(the usual RSA key in Base64) to the policy hub and run it cf-key -t /path/to/client_key.pub. The program cf-keyitself will add it /var/cfengine/ppkeyswith the correct name and rights.

Standard library

Automation of control over configurations, which allows easy to make large-scale changes, with the same ease leads to large-scale errors. Therefore, a reasonable amount of “seat belts” and “big red buttons” are needed. One of these “straps” is to add a standard library to VCS along with your code. When you install a package with the new version, the contents of the directory /var/cfengine/masterfilesand /var/cfengine/inputsis not updated as a result of it will be impossible to ensure the consistency of the configuration. Therefore, one of the important stages of updating is to merge the changes in the standard library with your copy, and it is here that all the help that your VCS can offer you, as well as the ability to use utilities diffand patch.

package_latests

One of the mechanisms that I often use to guarantee updates to some packages is the bundle package_latestfrom the standard library. Unfortunately, there is a bug due to which the bundle in Debian does not work. Fix is ​​very trivial. In the file lib/3.6/packages.cfyou need to find the bundle code packages_latestand bring it to this form (you can use the patch from the bug report):

debian::
  "$(package)"
  package_policy => "addupdate",
  package_version => "999999999:9999999999",
  package_method => apt_get_permissive;

Bug 6870

Another annoying bug that I found in the prod is bug 6870 . The essence of the problem is that CFEngine sets some classes based on PTR records of IP addresses on interfaces. As you might guess, such a behavior of the system is very, very unsafe, and it contradicts CFEngine's postulate on the unacceptability of external knowledge. However, in his book, Diego Zamboni teaches us to determine the execution host by view classes host1_example_org, and fixing this bug can break too many systems that are currently working. Therefore, until the developers have provided a more reliable way, we will create it ourselves. Here is the code that you can add to your version of the standard library in a file lib/3.6/bug_6870.cf:

bundle common bug_6870_workaround {
    classes:
        'bug6870_workaround_${sys.host}' expression => 'any';
}

Then you need to add the file name to, ${stdlib_common.inputs}by analogy with the files already listed there and then use the class bug6870_workaround_host1_example_orgwithout fear of unexpected intersections with PTR records of other IPs.

Life cycle

Now that everything is ready for creativity, I’ll talk a little about organizing the process of developing promises and a little about more mundane, everyday things.

Instruments

First of all, promises need to be written in something. For this, most likely, your favorite text editor is suitable . I prefer Vim and use plugins written by Niel Watson, and my colleague Valera Astraverkhau created a very good plugin to support CFEngine in Sublime Text 2 and 3. Emacs users will be interested in a lecture by Ted Zlatanov on Ted Zlatanov using Emacs as a CFEngine IDE.

You will also need a good version control system. Git is fine with everyone, but I'm sure any modern VCS will do. The requirements here are the same as for the development of conventional software, so take what you are comfortable with.

File Organization and Entry Point

I will say right away that the method I proposed is not the only true one. Familiar to Perl programmers, the TIMTOWTDI principle applies here; and nowhere else is polemic relevant here. Here is an example directory structure relative to the project root:

/bin
/masterfiles
/masterfiles/cfe_internal
/masterfiles/cfe_internal/ha
/masterfiles/controls
/masterfiles/controls/3.4
/masterfiles/inventory
/masterfiles/example_org
/masterfiles/lib
/masterfiles/lib/3.5
/masterfiles/lib/3.6
/masterfiles/services
/masterfiles/services/autorun
/masterfiles/sketches
/masterfiles/sketches/meta
/masterfiles/templates
/masterfiles/update
/static
/static/bird-lg
/static/firewall-configs
/static/ssh-keys
/templates

The directory /masterfiles/example_orgcontains the code that we write. The remaining subdirectories in /masterfilesare parts of the standard delivery, which I try not to change unless absolutely necessary. All non-standard templates are placed in /templates, and /static, as the name implies, "static" information is stored - public SSH keys, firewall settings, user settings, configuration files, etc. that does not change from host to host. There are a /bincouple of service scripts in the directory , including the "big red button" - a script that transfers all the necessary files to where CFEngine can distribute them to clients.

The entry point is located at /masterfiles/example_org/main.cfwhere two promises are contained: bundle common example_orgin which the files used are listed and the servers are classified, andbundle agent example_org_main, where, depending on the class, control is transferred to the desired bundle, which describes how exactly the servers of this class should be configured.

To specify an entry point in a file promises.cf, the following changes must be made:

body common control {
    bundlesequence => {
        # [...]
        @{example_org.bundles},
    };
    inputs => {
        # [...]
        'example_org/main.cf',
        @{example_org.inputs},
    };
}

Itself example_org/main.cflooks something like this:

bundle common example_org {
    vars:
        'inputs' slist => {
            'example_org/add_default_users.cf',
            'example_org/basic_packages.cf',
            'example_org/configure_dns.cf',
            'example_org/configure_firewall.cf',
            'example_org/configure_ftp.cf',
            'example_org/configure_ssh.cf',
            'example_org/cve_2015_0235.cf',
            'example_org/lib.cf',
        };
        'bundles' slist => {
            'example_org',
            'example_org_main',
        };
    classes:
        'ftp_server' or => {classmatch('BUG6870_ftp.*')};
        'dns_server' expression => classmatch('BUG6870_dns.*');
    reports:
        verbose_mode::
            '${this.bundle}: defining inputs="${inputs}"';
            '${this.bundle}: defining bundles="${bundles}"';
        ftp_server::
            'This host assumes FTP server role';
        dns_server::
            'This host assumes DNS server role';
}
bundle agent example_org_main {
    methods:
       any::
            'example_org_update_motd' usebundle => 'update_motd';
            'example_org_basic_packages' usebundle => 'basic_packages';
            'example_org_add_default_users' usebundle => 'add_default_users';
            'example_org_configure_firewall' usebundle => 'configure_firewall';
            'example_org_configure_ssh' usebundle => 'configure_ssh';
            'example_org_cve_2015_0235' usebundle => 'cve_2015_0235';
        any.Min30_35::
            'heartbeat' usebundle => 'heartbeat';
        # FTP servers configuration
        ftp_server::
            'example_org_configure_ftp' usebundle => 'configure_ftp';
        # DNS servers configuration
        dns_server::
            'example_org_configure_dns' usebundle => 'configure_dns';
}

This example was composed of several real projects to demonstrate, in reality, of course, everything is a little more complicated and bigger. The purpose of this approach is to minimize the number of changes in the standard library, so that later it will be easier to maintain changes in it.

Debugging and testing

Since the price of the error is high, before you click on the “big red button”, you should test your promises. For tests, I have a small testing ground: several virtual machines that I run on my work computer. On them I check the correctness of the fulfillment of promises and engage in experiments.

In developing promises, I adhere to the style of the debugging school via printf, that is, I use promises of the type reportvery widely. Another indispensable tool is a complete utility cf-promises. In addition to formal syntax validation, it can show all classes (parameter --show-classes) available at runtime and variables with their contents (parameter --show-vars). And of course, launch cf-agentin verbose mode (parameter --verbose).

Updates

There are two ways to update the CFEngine version on all controlled machines: either through the package manager, or using the mechanisms of CFEngine itself. I prefer a package manager, for which I have a special promise that I connect only for the duration of the updates, its essence boils down to a call package_latest. In my opinion, this approach is best suited to CFE concepts.

Big Red Button

Rolling out into the product is always a little exciting moment, even if it happens many times a day, and I do not blame people who have some kind of ritual for this. In the case of massive configuration changes, this can determine the meaning of life for the next few days if something goes wrong. Therefore, no automation, no hooks for Git. Only manual mode, as a guarantee of confidence that everything is done correctly, tested and ready for sale. I have a script deploy.shwith rights as a button 0600, so that it could not be started by accident. Typing hands in the console bash bin/deploy.shis my ritual and the last opportunity to cancel the launch. The script itself is very trivial: with the help of rsyncit it synchronizes masterfiles, staticand templateswith the contents/var/cfengine/{masterfiles,static,templates}and runs two commands: cf-agent -KIC -f update.cfand cf-agent -KIC -f promises.cf. So I can be sure that at least the policy hub can fulfill promises and distribute them to all clients.

Conclusion

This is far from all the subtleties and wisdom, but this is quite enough to start the implementation of CFEngine at home. Outside of the article there were such interesting topics as " Design Center ", reports, internal design, different usage scenarios and much more. If CFEngine is of any interest to the habrasociety, I will be happy to talk more about it, but for now, if you have any urgent questions right now, do not hesitate to ask them in the comments. My colleague and lastops cagliostro and I will try to answer them.