Installing and Configuring Puppet + Foreman on Ubuntu 14.04 (Walkthrough)

Good day to the residents of Habr!

When the number of managed servers reaches several tens, or even hundreds, you have to look for a solution to automatically configure and manage such a fleet. Puppet comes to the rescue here. Why Puppet? Puppet is cross-platform, has a rich community, has many ready-made modules (4800+), has Enterprise versions. All these advantages do not give doubts about the power of this product. But managing such a “combine” from the console is not so simple. Therefore, for convenient control and configuration of Puppet, Foreman was developed. Next, install and configure this bundle using the SSH key management task example.

Requirements:

• pure server for puppet-master ;
• Commands on the puppet master server run as root ;
• commands on puppet-agent servers are executed via sudo .

Used software:

• OS Ubuntu 14.04.5 LTS;
• Puppet 3.8.7;
• Foreman 1.11.4.

Objectives:

• Get a convenient way to automate network infrastructure management
• Get a convenient way to manage ssh keys.

Note

All screenshots and a piece of configuration are hidden by spoilers. For a better understanding of where the commands are executed, I added a server type (master or agent) before each command.

1. Installing Foreman + Puppet on puppet masters

Add the Foreman / Puppet installer repository and install it on the system:

master ~ $ apt-get -y install ca-certificates
master ~ $ cd ~ && wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
master ~ $ dpkg -i puppetlabs-release-trusty.deb
master ~ $ sh -c 'echo "deb http://deb.theforeman.org/ trusty 1.11" > /etc/apt/sources.list.d/foreman.list'
master ~ $ sh -c 'echo "deb http://deb.theforeman.org/ plugins 1.11" >> /etc/apt/sources.list.d/foreman.list'
master ~ $ cd ~ && wget -q http://deb.theforeman.org/pubkey.gpg -O- | apt-key add -
master ~ $ apt-get update && apt-get -y install foreman-installer

Run the installer:

master ~ $ foreman-installer

The result should be similar to the following:


Link view puppet.and username with password will come in handy further.

Let's configure the configuration for viewing in Foreman differences of file changes:

master ~ $ nano /etc/puppet/puppet.conf
> show_diff = true

Open the link recommended in the previous step in the browser: puppet.
And enter the username: admin and the password that we saw in the console after installation.


If authorization is successful, then Foreman is installed and working properly. You can now move on to the next chapter.

2. Setting up Foreman

By default, Foreman uses its SSL certificate generated by Puppet and your browser will not accept it. You can add the root certificate ( /var/lib/puppet/ssl/certs/ca.pem) to your browser so that insecure connection warnings disappear (for Chromium add here: Settings / SSL / Certificate Authorities).

The first time you log in, you will see the Dashboard page , which will show general statistics for all nodes on the network. When adding hosts, there will be useful statistical information.


On subsequent logins, you will be redirected to the host list page.

2.1. password change

First of all, you need to change the user password:


The default password is already complicated, but it’s better to make your own.

2.2. Adding a module using NTP as an example

The time must be precisely set on the puppet master server. To do this, you must use NTP. If the time is wrong, the puppet master may erroneously issue agent certificates from the distant past or future, which other nodes will consider obsolete.

Sometimes, in order to be able to manage Puppet modules through Foreman, you need to install modules whose developers are not Puppet-Labs, but the developers of the Puppet community. This follows from the fact that Foreman uses Restful API HTTP requests for Puppet, but not all modules define management using this API.

Install the saz / ntp module on puppet master:

master ~ $ puppet module install saz/ntp

Note

The saz / ntp module works great on Foreman version 1.11. For other versions of Foreman, you can use the modules from the website forge.puppetlabs.com to search for ntp .

You should see the following:


Now the module has been installed only for puppet-master. Now you need to enter the web interface and add it to Foreman. Go to Configure → Classes menu and click Import from puppet.:


As a result, you will see a list of available classes, select the required ones and click Update :


In order to use the ntp servers closest to you, go to www.pool.ntp.org . There, in the right block, select the pool we need (Africa, Asia, etc.) and pick up the list of servers in the clipboard.

Next, go to the settings of the ntp class by clicking on its name. Go to the Smart Class Parameter tab and look for the server list tab in the left list :


We mark the Override item in the Default value according to the example of the previous value, add the server from the step above. I added the following value:

["0.asia.pool.ntp.org","1.asia.pool.ntp.org","2.asia.pool.ntp.org","3.asia.pool.ntp.org"]

Click Submit at the bottom of the page, thereby overriding the class parameter.

2.3. Adding the accounts and ssh module

Using the previous module as an example, install the accounts module :

master ~ $ puppet module install camptocamp-accounts

If the installation was successful, then you will see the following:


Install the ssh module :

master ~ $ puppet module install saz/ssh

After that, go to Foreman and import new classes. Later, after creating host groups, we will configure the accounts and ssh classes .

2.4. Adding mysql and apache module

To explain the subsequent names of the database and web groups, add the apache and mysql modules . We add modules following the example of the previous ones. You can download them with the commands:

master ~ $ puppet module install puppetlabs-apache
master ~ $ puppet module install puppetlabs-mysql

3. Adding hosts

To add a host to Puppet, you must install the puppet agent on that host. To install the puppet agent, download and install the puppet-labs repository :

agent ~ $ cd ~ && wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
agent ~ $ sudo dpkg -i puppetlabs-release-trusty.deb
agent ~ $ sudo apt-get update

Then install the puppet agent:

agent ~ $ sudo apt-get -y install puppet

To run Puppet as an agent, you need to comment out the puppet master zone settings. Also add the configuration for the agent, which will set the address of our puppet master. Let's bring the configuration file /etc/puppet/puppet.confinto the form:


Replace the value of the START variable with no to yes to start the puppet agent after rebooting the OS. And also run the puppet agent:

agent ~ $ sudo sed -i s/START=no/START=yes/g /etc/default/puppet
agent ~ $ sudo service puppet start

With a small infrastructure, the puppet agent can be run as a daemon. There is also a way to run through CRON: docs.puppet.com/puppet/3.6/services_agent_unix.html#running-puppet-agent-as-a-cron-job .

Note

By default, the Puppet agent searches for the puppet master domain in its zone, unless the server parameter is explicitly specified (in the puppet.conf file). For example: server.domain.com will search for puppet.domain.com server . Therefore, if you still follow the instructions, then everything should work for you.

After that, go to Foreman in Infrastructure → Smart Proxies → Certificates :


There should appear a host on which we just installed the puppet agent. You can use the filter (top left) to see only unsigned certificates. To sign, click the Sign button :


Within a few minutes, the server server.(the server on which we just installed the agent) appears in the list Hosts → All hosts .

4. Adding host groups

Go to the menu item Configure → Host Groups . Click New Host Group . The Host Group tab should be as follows:


The root group will be the root group . She will be the parent of all other groups. She will have full access to everything. And the main classes will be included in it.

Next, go to the Puppet Classes tab and add the necessary classes by clicking on + :


Click Submit .

We add two more groups according to the same principle. Only now we will choose the root group as the Parent , because the classes accounts , ntp and ssh are inherited and you do not need to add them again. We add the mysql :: server class only for the database group , and the apache class for the web group .

5. Adding a node to a group

To include a node in a group, you need to go to its settings.


After that, add the group in the first tab, as in the screenshot below:


After that, click Submit and within a few minutes mysql will appear on the host . In the same way, you can assign the other two servers to the web group :


The entire configuration extends to puppet agents automatically for a short time.

If you do not want to wait, then you can run the command on the clients puppet agent --testand see with your own eyes how the configuration is created.

6. Setting permissions using the accounts module

Actually now you can once again look at the circuit that we presented at the beginning and create logic on the basis of it.

Go to the menu item Configure → Classes . Click on accounts to go to the module settings. Of all the settings, we will need the tabs accounts , ssh keys , users .

Note
The accounts tab — it contains the hashes “server user → public key names from the ssh keys tab ”. Ssh keys tab - it contains the hashes “key name → type and value”. Users tab- it contains users who need to be created or specify some parameters for existing ones.

Open the last users tab and set it as in the screenshot:


This setting configures the user's home directory. Here we used Merge overrides and Merge default parameters, which will allow us to combine the configuration for the final host.

We fill in the ssh keys tab as follows:


In the Default value field , enter all the public keys of the accounts that will be used in the accounts tab . These are public keys of users who will have access to one or another server. An indent of two spaces before the type and public parameters is required.

An example of how one public key looks like (the rest are added one after another below):

admin:
  type: ssh-rsa
  public: AAAAB3NzaC1yc2EAAAADAQABAAABAQDXibuyi2MFzERps7mD2J38mhd4phXQlOEZrmui9rDdcYD0XeEnvdRTZPcsMOw6DRT1ERpzbcFehj+G29YxoiXZ541gVjVvsATAqojN3zEkMz5b0AgBNcKDFi9h/qwlK9YDv2trKEcRHQ4kBN332Z6oqdBFerUMys5dvc3RVlE+x2kVmYNmGIlma5twC9w/wRNoD+nUK+3bk+I+Og40f//uFAKFeY4DMoCrdOsHJrPak5nD9vL6a2m/Fe3jfgmpBCcnV3LS2mr+PdRYbtju7nzfu8WT0ugMAUi+dDMRFh3DmfCzXbOi2TPi+mP//L/A19thXffd/QzW7wmAgxlj+km1

Fill the top tab accounts as follows:


Of this parameter should be: root has access anywhere with the account root (login root - it is an element of the tab, the ssh keys ), account dbadmin have root-access only to servers from the group database , and the user admin is only at the group web and account admin can connect only for admin user .

On the users tab, add the admin user to the www-data group .

6.1 Configuring the ssh class

In the accounts class, we configured ssh key access. Therefore, for more complete security, it is necessary to prohibit password access. This is done using the ssh class . Go to its settings and open the Smart Class Parameter tab . Next, client options lead to the following form:


The server options tab is as follows:


And fill the storeconfigs enabled tab like this:


In storeconfigs all customer facts are stored, so you can query the database and get a list of hosts that meet certain criteria. For greater security, we turned it off.

7. Results

As you complete this guide, your infrastructure added to Puppet will become quickly configurable and scalable. And the main goal - managing public ssh keys will be as convenient as possible.

Screenshot of the admin user key list on one of the machines in the root / web group :


Remember, when setting up the accounts class for the ssh keys parameter, we included Merge overrides and Merge default . This is necessary so that at the end for a certain network node one structured file with ssh keys is collected.

Check if it is really possible to log in as user “ admin ” using the added key:


If your test was also successful, then the infrastructure is ready and you can gradually connect your other servers to the puppet-master and configure other services through Puppet.

Used resources: documentation Puppet's , documentation Foreman of .