Migrating to Chef Server 11 / EPAM Blog

As part of our EPAM Private Cloud, we initially used Chef Server 10 to autoconfigure virtual machines. The
list of supported roles has stepped over 60 and included both simple and rather complex cluster solutions.
And when the number of server clients grew to 750, we noticed a significant decrease in performance.
It was not advisable to increase the power of the virtual machine on which Chef Server 10 is installed, and it was not so small anyway (2x Intel® Xeon® CPU L5640 @ 2.27GHz and 8Gb of RAM).
Manipulations with tuning chef-solr and chef-expander also did not give the desired performance boost.

And then it was decided to migrate from Chef 10 to Chef 11.
Opscode.com says it’s faster, easier to scale, and easier to configure and manage.

So, first, a quick installation guide for Chef Server 11 for Ubuntu12.04.

#Скачиваем и устанавливаем пакет
wget -O chef-server-11.deb https://opscode-omnitruck-release.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.0.6-1.ubuntu.12.04_amd64.deb
sudo dpkg -i chef-server-11.deb
#Конфигурируем и запускаем
sudo chef-server-ctl reconfigure
#Проверяем, что установка прошла успешно
sudo chef-server-ctl test

When the server is installed and running, you need to create a user with administrator rights.
Create the .chef directory in the user's home folder and copy the necessary keys.

mkdir ~/.chef
cp /etc/chef-server/admin.pem ~/.chef
cp /etc/chef-server/chef-validator.pem ~/.chef

We start setting up the configuration file for knife.

knife configure -i
#В итоге получаем что-то типа
cat ~/.chef/khife.rb
log_level                :info
log_location             STDOUT
node_name                'chefuser'
client_key               '/home/chefuser/.chef/chefuser.pem'
validation_client_name   'chef-validator'
validation_key           '/home/chefuser/.chef/chef-validator.pem'
chef_server_url          'https://192.168.0.1'
syntax_check_cache_path  '/home/chefuser/.chef/syntax_check_cache'

Data transfer with Chef 10.

First you need to install knife-essentials to expand the functionality of the utility knife

/opt/chef-server/embedded/bin/gem install knife-essentials

Create a directory that we will use as a transit point. And in it we will create a folder for knife configuration files.

mkdir  -p ~/transfer/.chef

Create separate knife configuration files for Chef 10 and Chef 11


It remains to copy webui.pem from Chef Server 10 to /home/chefuser/transfer/.chef/chef-webui.pem and
/home/chefuser/.chef/chefuser.pem to /home/chefuser/transfer/.chef/admin. pem.

When this is ready, you can try to download information about nodes, clients, roles, etc.

/opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb /
#Результатом будет создание папок /nodes /clients /roles /cookbooks и т.д. с соответствующим содержимым.
ls transfer/
clients  cookbooks  data_bags  environments  nodes  roles  users

In the same way, you can download information about an individual node or client.
For instance:

/opt/chef-server/embedded/bin/knife download -c .chef/knife-chef10.rb /nodes/server1.json

Before you start uploading data to Chef Server 11, I recommend checking that you do not have the same client names or nodes on both servers. Otherwise, information about them may be erased.

Download data as follows

/opt/chef-server/embedded/bin/knife upload -c .chef/knife-chef11.rb /

You can also upload by one node, client, etc.

Switching customers from Chef 10 to Chef 11.

This is perhaps the most interesting stage in migration.
If you used the DNS name as chef_server_url , then there are no problems. You just need to reassign it to the new server.
But if you used, as in our case, the name of the virtual machine, which is unique and cannot be assigned to another machine, then you have to think about it.
I decided this question by creating a simple cookie, the purpose of which was to change the value of chef_server_url to the desired one and restart the chef-client.
If anyone is interested in such an implementation, the cookie will be posted on GitHub.

Summary: After migrating to Chef 11, we got a performance boost at times. The previous server was constantly “on the verge of a swap” and could process a simple request for a list of roles up to 2 minutes. On the current server with the same number of clients, 3 GB of RAM is free and requests are processed within a few seconds.

If anyone has questions - write in the comments, I will answer with pleasure.