Preparing Servers with Chef Solo

In this article I want to talk about using Chef Solo both to prepare the developer environment (dev-servers) and to prepare battle servers. There have already been several articles on Habré ( Rapid deployment of the development environment and Development Environment using Vagrant and Chef ), devoted to the deployment of dev-servers using Vagrant and Chef Solo. I want to show how we use Chef Solo in our small company.

Our web-project requires a rather complicated environment, since a multi-server architecture is used. Therefore, it was vital for us to automate the preparation of such an environment. To solve this problem, we use Vagrant and Chef Solo.


VagrantIs an open-source tool for creating a virtual environment (using VirtualBox, VMware, AWS, etc.). In other words, it is a program for easily managing virtual machines. Among other goodies, Vagrant supports several configuration management systems (Chef, Puppet, Ansible). By configuration we mean the state of the server (in this case, the virtual machine), namely: a set of installed packages, a list of running services, the contents of configuration files, the composition of users, etc. That is, Vagrant can start the process of bringing the machine to the necessary state immediately after starting.

Vagrant was perfect for creating a developer environment. It is enough for the developer to make a clone of the Vagrant project and execute the “vagrant up” command. Everything else is done automatically: downloading the image of the virtual machine (if it is not on the developer's machine), starting the virtual machine with the configured network parameters and shared directories, launching the configuration management system. This provides a unified environment for all developers, which is very important for large teams or complex projects.

Chef- server configuration management system. Chef has a client-server architecture. The Chef server stores information about connected Chef clients and sets of “recipes” for their “preparation” (that is, to bring them to the required state). You have to pay for using a Chef server (up to 5 clients can be connected for free).

Why Chef was chosen, and not another configuration management system (only looked seriously at Puppet):

• large community;
• many ready-made recipes;
• recipes are written in Ruby;
• predictable recipe order (as opposed to Puppet);
• supported by Vagrant.

Chef Solo is an open-source version of the Chef client that allows you to use recipes without a Chef server (recipes must be physically located on the same machine). Chef Solo is free, but has several limitations compared to the client-server Chef. For example, there is no way to use the server search conditions in recipes.

Why Chef Solo was chosen, and not the client-server Chef:

• free (we have more than 5 servers, but not yet thousands);
• safe (data about our servers are not stored on other people's servers);
• simple (working with local files instead of console commands to the Chef server);
• there is still the possibility of easily switching to the Chef server (if there are too many servers).

Chef Solo allowed to fully automate the installation and configuration of the project on the new server. The deployment of a combat server for our application usually takes about 10-15 minutes (depending on the complexity of the recipes).

The following project is intended for server management . From the catalog of this project, virtual machines are launched using Vagrant, and the preparation of real servers is also carried out. In this project, one virtual server (demo) has already been added, on which the latest version of nginx is installed.

Project Structure:

• cookbooks / : a directory of other people's recipes downloaded externally and not editable. More details .
• data_bags / : directory for storing additional data available from recipes. More details .
• environments / : directory of environments. Under the environment refers to a specific set of settings used for the server, deployed with the specified environment. This is where development / production settings are shared. To be able to use such environments, a Chef Solo version of at least 11.6.0 and a Vagrant version of at least 1.3.0 are required. More details .
• nodes / : files with server attributes. For each server, there should be a file with a name equal to its IP address containing the server attribute values, as well as a list of roles and recipes that must be applied to the server. More details .
• roles / : roles directory. A role is defined as a specific set of settings used for the server to which this role is assigned. A server can be assigned multiple roles. More details .
• site-cookbooks / : directory of your recipes. It contains recipes written with one's own hand to ensure specific requirements (for example, deployment of work projects). This directory is specified in the solo.rb and Vagrantfile files as an additional storage for recipes.
• deploy.sh : script for preparing a combat server (details below).
• install.sh : a script to install Chef Solo and run it on a combat server (details below).
• solo.rb : Chef Solo configuration file.
• Vagrantfile : Vagrant settings file. It lists all the virtual servers.

To deploy a new virtual server for development, you must:

1. In the “Vagrantfile”, associate the server name (for example, “frontend”) with its IP address (for example, “10.2.2.100”).
2. In the “nodes” directory, create a JSON file for the attributes of the new server (for example, “10.2.2.100.json”).
3. Start the virtual machine using the “vagrant up” command with the machine name (for example, “vagrant up frontend”). At the first start, Chef Solo will be launched, which will execute all the recipes specified in the corresponding JSON attribute file. In this case, the "development" environment will be used (this is indicated in the "Vagrantfile").

To raise a new battle server, you must:

1. In the “nodes” directory, create a JSON file for the attributes of the new server (for example, “23.144.12.15.json”).
2. Run the script "deploy.sh" with the user and IP server (for example, "./deploy.sh root@23.144.12.15"). The deploy.sh script internally provides an SSH connection to the server, therefore, all additional parameters specified when running this script will be used as parameters of the ssh command (this is important when entering by key). Under Windows, we run this script in Git Bash, since it requires some * nix commands, which are not in cmd.exe, to execute it.

The script "deploy.sh" performs the following actions:

1. connects to the server via SSH;
2. archives the entire project directory and transfers the archive via SSH to the server;
3. unpacks the archive on the server into the directory "~ / chef";
4. runs the install script “install.sh”.

The script "install.sh" performs the following actions:

1. checks if Chef Solo is installed and installs it if necessary. We use Ubuntu on the servers, so for other systems you may need to correct the commands to install Chef.
2. starts Chef Solo with the configuration file “solo.rb” and the JSON file for server settings. In this case, the "production" environment will be used (this is indicated in "solo.rb").

In addition to the initial preparation of the server, it is often necessary to update its configuration. For example, you need to update some packages, post a new version of the application, or apply the latest changes in recipes. For a virtual server, this operation is performed by the "vagrant provision # machine_name #" command. For the combat server, you need to restart "./deploy.sh root @ # ip_server #".

The “deploy.sh” and “install.sh” scripts are written based on the article by Chef Solo tutorial: Managing a single server with Chef .

Instead of deploy.sh, you can look at knife-solo , but it has several drawbacks compared to the proposed method:

• Chef will have to be installed on the machine from which the server is being prepared;
• Windows requires cygwin + rsync (non-trivial setup);
• Chef is installed on the server through Opscode Installer, which has problems installing some gem packages (this problem is solved by specifying your installation script).

So, with the help of simple devices, you can relatively easily deploy virtual servers for development and combat servers for working projects. I hope my article will be useful to someone.