Developing and testing chef cookbooks with the Sparrowdo tool

Hello! A lot has been written about the development of chef cookbooks and the related infrastructure, and there are already plenty of tools in this area. Among them you can list such solutions as vagrant , test kitchen , food critic , chef spec , minitest-chef-handler , serverspec , inspec . All of them, to one degree or another, simplify and accelerate the industrial development and testing of chef cookbooks and the infrastructure they configure.

If this area is close to you and you also have something to do with the Perl language (more precisely, Perl6 ), then welcome to the topic.

So, today I will tell you how I use Sparrowdo when developing and testing chef cookbooks.

First, a little introductory.

What is a Sparrowdo?

Sparrowdo is an add-on on the sparrow plug-in system that allows you to run these plug-ins via ssh on remote servers. In turn, sparrow plugins are various automation scripts written mainly (but not limited to these languages) in Perl5 and Bash . A whole series of articles on habrahabr has already been written about all this. If you are interested, you can type a search query with the word sparrow and read accordingly. But, so that it would be clear to an uninitiated reader, all this is very similar to the ansible ideology , when you have a centralized server from which you start different tasks on other remote servers.

So, you can run various sparrow plugins after or before running the chef client on the server, and perform various checks to determine that the deployment procedure was successful or satisfies certain conditions.

All this is very similar to scripting in the style of serverspec or inspec with the only difference being that sparrow plugins are multi-purpose and are not limited to testing and audit tasks, in principle, you can implement any other logic through sparrow that you for one reason or another do not would like to stuff in chef cookbooks.

In fact, sparrowdo so conveniently “fit in” with the tasks of automatically setting up servers that I even launch the chef client itself ... But, quite theory, let's look at a specific simple, but significant example ...

We write a cookbook for installing nginx + tomcat and test it with sparrowdo

For simplicity, I’ll use an ec2 instance with a pre-installed chef client and configured on a local chef server. Those. all issues related to the initial setup of the system will be left “outside the brackets”. Further we have an ssh key to our instance and login with sudo:

$ ssh remote.server 

Now let’s move a little away from server access issues and write our symbolic cookie, we need:

• install nginx server
• install tomcat server
• if the tomcat_redirect attribute is set, configure proxying of all requests coming to nginx to the tomcat server running on localhost port 8080

Writing a cookbook

The code is trivial, again, the focus of this article is not on learning to write chef cookbooks, but on using chef in conjunction with sparrowdo / sparrow

# recipes/default.rb
package 'nginx'
package 'tomcat7'
service 'nginx' do
  action [ :enable, :start ]
end
service 'tomcat7' do
  action [ :enable, :start ]
end
template '/etc/nginx/conf.d/default.conf' do
   source 'nginx.conf'
   mode '644'
   variables :tomcat_redirect => node[:tomcat_redirect]
   notifies :restart, 'service[nginx]', :delayed
end
# attributes/default.rb
default[:tomcat_redirect] = false
# cat templates/default/nginx.conf
server {
    listen       80 default_server;
    server_name  _;
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    <% if @tomcat_redirect %>
    location / {
      proxy_pass http://127.0.0.1:8080;
    }
    <% else %>
    location / {
      try_files $uri $uri/ $uri.html =404;
    }
    <% end %>
    error_page  404              /404.html;
      location = /404.html {
    }
    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
      location = /50x.html {
    }
}

I think that comments on the cookbook code are superfluous, I only note that by default nginx does not perform proxying on tomcat, since the attribute is tomcat_redirectset to false. And finally, let's call the nginx-app cookie and upload it to the chef server:

# cat metadata.rb
name 'app-nginx'
version '0.0.1'
$ knife cookbook upload app-nginx -o ../

Now, actually run the created cookie on our server.

Writing a sparrowdo script

Sparrowdo script - Perl6 code that runs tasks on a remote server. Tasks are nothing more than sparrow plugins with parameters. As I said, we will use sparrow to launch the chef client as well

$ cat  sparrowfile
task_run %(
  task => "set up chef run list an attributes",
  plugin => "file",
  parameters => %(
    target => "/tmp/chef.json",
    content => q:to/JSON/,
    {
      "run_list" : [
        "recipe[nginx-app]",
      ],
        "tomcat_redirect" : false
    }
    JSON
  ),
);
task_run %(
   task => "run chef-client",
   plugin => "bash",
   parameters => %(
   command => "chef-client --color --json /tmp/chef.json"
 ),
);

A little explanation on the code.

• The script file should be named sparrowfileand located in the same directory from where you will run the client sparrowdo, it makes sense to keep it in the cookbook directory - clearly and always at hand.

  
  
• In this example, two sparrow plugins are used: file - to create files and bash - to run arbitrary commands on Bash, as already mentioned, they are launched with parameters using the function task_run, also the parameter taskin the calls to this function adds an arbitrary textual description of the task.

Now you can run the script. Because on the given sparrow server, the client is not yet installed, we start with the --bootstrap option, asking sparrowdo to install the client first, which will perform all the tasks:

$ sparrowdo --http_proxy=$http_proxy --https_proxy=$https_proxy --host=remote.server  --ssh_user=root  --no_sudo --no_index_update
running sparrow tasks on remote.server ...
target OS is - centos6
push task  plg  OK
push task  plg  OK
set up task box file - /tmp/sparrowdo-box.json - OK
public@file is uptodate (0.0.2)
public@bash is uptodate (0.1.1)
running task box from /tmp/sparrowdo-box.json ...

/ started
set target content
target created
set target mode to 644
ok      scenario succeeded
ok      output match /target (created|removed)/
ok      output match 'set target content'
STATUS  SUCCEED

/ started
[2016-10-19T10:02:48+00:00] INFO: Forking chef instance to converge...
[2016-10-19T10:02:48+00:00] INFO: *** Chef 11.16.4 ***
[2016-10-19T10:02:48+00:00] INFO: Chef-client pid: 5389
[2016-10-19T10:02:49+00:00] INFO: Setting the run_list to ["recipe[nginx-app]"] from CLI options
[2016-10-19T10:02:49+00:00] INFO: Run List is [recipe[nginx-app]]
[2016-10-19T10:02:49+00:00] INFO: Run List expands to [nginx-app]
[2016-10-19T10:02:49+00:00] INFO: Starting Chef Run for sandbox-generic-ci-i-ef7cbedf
[2016-10-19T10:02:49+00:00] INFO: Running start handlers
[2016-10-19T10:02:49+00:00] INFO: Start handlers complete.
[2016-10-19T10:02:49+00:00] INFO: HTTP Request Returned 404 Not Found:
[2016-10-19T10:02:50+00:00] INFO: Loading cookbooks [[email protected]]
[2016-10-19T10:02:50+00:00] INFO: Processing package[nginx] action install (nginx-app::default line 1)
[2016-10-19T10:02:53+00:00] INFO: Processing package[tomcat7] action install (nginx-app::default line 3)
[2016-10-19T10:02:53+00:00] INFO: Processing service[nginx] action enable (nginx-app::default line 5)
[2016-10-19T10:02:53+00:00] INFO: Processing service[nginx] action start (nginx-app::default line 5)
[2016-10-19T10:02:53+00:00] INFO: Processing service[tomcat7] action enable (nginx-app::default line 9)
[2016-10-19T10:02:53+00:00] INFO: Processing service[tomcat7] action start (nginx-app::default line 9)
[2016-10-19T10:02:53+00:00] INFO: Processing template[/etc/nginx/conf.d/default.conf] action create (nginx-app::default line 13)
[2016-10-19T10:02:53+00:00] INFO: Chef Run complete in 3.604494597 seconds
[2016-10-19T10:02:53+00:00] INFO: Running report handlers
[2016-10-19T10:02:53+00:00] INFO: Report handlers complete
bash-command-done
ok      scenario succeeded
ok      output match 'bash-command-done'
STATUS  SUCCEED

The first run of the script worked successfully, we see that the chef client installed the required packages and configured them, following the logic specified in the cookie.

Change the sparrowdo script a bit, minimizing the output from the client’s chef during the following launches:

task_run %(
    task => "run chef-client",
    plugin => "bash",
    parameters => %(
    command => "chef-client --color --json /tmp/chef.json -l error"
  ),
);

Now it's time to add a couple of checks that our deployment was successful, namely:

• nginx service is visible in the process list
• tomcat service is visible in the process list

This may seem unnecessary because chef guarantees us the processing / output of all errors related to the launch of services, but experience shows that this is not always enough , for example, a service may fall a bit later, after the chef client has successfully completed. And then it never hurts to write a couple more “parallel” tests (which, by the way, can be run outside the context of the client’s chef), especially since it is very simple to do this with sparrowdo. Add a couple of tasks to our sparrowdo script:

task_run  %(
  task => 'check nginx process',
  plugin => 'proc-validate',
  parameters => %(
    pid_file => '/var/run/nginx.pid',
    footprint => 'nginx.*master'
  )
);
task_run  %(
  task => 'check tomcat7 process',
  plugin => 'proc-validate',
  parameters => %(
    pid_file => '/var/run/tomcat7.pid',
    footprint => 'java'
  )
);

Here we used the proc-validate plugin , which checks the existence of a process using a given PID file, and also searches for it by the “trace” in the list of running processes:

  $ sparrowdo --http_proxy=$http_proxy --https_proxy=$https_proxy --host=remote.server  --ssh_user=root  --no_sudo --no_index_update
  running sparrow tasks on remote.server ...
  target OS is - centos6
  push task  plg  OK
  push task  plg  OK
  push task  plg  OK
  push task  plg  OK
  set up task box file - /tmp/sparrowdo-box.json - OK
  public@file is uptodate (0.0.2)
  public@bash is uptodate (0.1.1)
  public@proc-validate is uptodate (0.0.5)
  running task box from /tmp/sparrowdo-box.json ...
  
  / started
  set target content
  target created
  set target mode to 644
  ok      scenario succeeded
  ok      output match /target (created|removed)/
  ok      output match 'set target content'
  STATUS  SUCCEED
  
  / started
  bash-command-done
  ok      scenario succeeded
  ok      output match 'bash-command-done'
  STATUS  SUCCEED
  
  / started
  pid_file - /var/run/nginx.pid
  pid file exists
  pid:4526
  process footprint:  4526 nginx: master process /usr/       31:47
  ok      scenario succeeded
  ok      output match /pid_file - \S+/
  ok      output match 'pid file exists'
  ok      output match /pid:\d+/
  ok      output match /process footprint:/
  ok      'process footprint:  4526 nginx: master process /usr/       31:47' match /nginx.*master/
  STATUS  SUCCEED
  
  / started
  pid_file - /var/run/tomcat7.pid
  pid file exists
  pid:29458
  process footprint: 29458 /usr/lib/jvm/java-1.6.0/bin    22:11:25
  ok      scenario succeeded
  ok      output match /pid_file - \S+/
  ok      output match 'pid file exists'
  ok      output match /pid:\d+/
  ok      output match /process footprint:/
  ok      'process footprint: 29458 /usr/lib/jvm/java-1.6.0/bin    22:11:25' match /java/
  STATUS  SUCCEED

Excellent. We see that our services are really running. Another advantage of this approach is that you can always run such checks on an already deployed server without running the chef client itself.

Ok, let's move on. Check the availability of our services on http . To do this, use the curl utility , here is what you need to add to our sparrowdo script:

task_run  %(
  task => 'install curl',
  plugin => 'package-generic',
  parameters => %(
    list => 'curl',
  )
);
task_run  %(
  task => 'check nginx http port',
  plugin => 'bash',
  parameters => %(
    command => 'sleep 5; curl --retry 2  --noproxy 127.0.0.1  -f -o /dev/null -D - http://127.0.0.1',
  )
);
task_run  %(
  task => 'check tomcat http port',
  plugin => 'bash',
  parameters => %(
    command => 'sleep 5; curl --retry 2  --noproxy 127.0.0.1  -f -o /dev/null -D - http://127.0.0.1:8080',
  )
);

...

Run the script:

$ sparrowdo --http_proxy=$http_proxy --https_proxy=$https_proxy --host=remote.server --ssh_user=root --no_sudo --no_index_update

# часть вывода опущена ...

/ started
package installer
/modules/yum/ started
trying to install curl ...
installer - yum
Installed Packages
curl.x86_64                  7.19.7-37.el6_4                   @base/$releasever
ok      scenario succeeded
ok      output match 'Installed'
STATUS  SUCCEED

/ started
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
102  3698  102  3698    0     0  3218k      0 --:--:-- --:--:-- --:--:--     0
HTTP/1.1 200 OK
Server: nginx/1.0.15
Date: Wed, 19 Oct 2016 11:30:51 GMT
Content-Type: text/html
Content-Length: 3698
Last-Modified: Fri, 26 Apr 2013 20:36:51 GMT
Connection: keep-alive
Accept-Ranges: bytes
bash-command-done
ok      scenario succeeded
ok      output match 'bash-command-done'
STATUS  SUCCEED

/ started
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11197    0 11197    0     0  1195k      0 --:--:-- --:--:-- --:--:-- 1366k
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=ISO-8859-1
Transfer-Encoding: chunked
Date: Wed, 19 Oct 2016 11:30:56 GMT
bash-command-done
ok      scenario succeeded
ok      output match 'bash-command-done'
STATUS  SUCCEED

As we see our services are available on http. I note that we previously installed the curl utility, using the package-generic plugin to install packages.

Well, finally, set the attribute tomcat_redirectto true, forcing nginx to proxy all requests to tomcat and verify that this happens.

Set the attribute:

task_run %(
  task => "set up chef run list an attributes",
  plugin => "file",
  parameters => %(
    target => "/tmp/chef.json",
    content => q:to/JSON/,
    {
      "run_list" : [
        "recipe[nginx-app]",
      ],
        "tomcat_redirect" : true
    }
    JSON
  ),
);

Добавляем параметр expect_stdout в запуск плагина bash, который заставит его проверить STDOUT выполняемой команды по заданному regex:

task_run  %(
  task => 'check nginx http port',
  plugin => 'bash',
  parameters => %(
    command => 'sleep 5; curl --retry 2  --noproxy 127.0.0.1  -f  http://127.0.0.1 | head -n 10',
    expect_stdout => 'Apache Tomcat'
  )
);

Запустим сценарий:

$ sparrowdo --http_proxy=$http_proxy --https_proxy=$https_proxy --host=remote.server  --ssh_user=root  --no_sudo --no_index_update
# часть вывода опущена ...

/ started


    
        
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (23) Failed writing body (146 != 3151)
bash-command-done
ok      scenario succeeded
ok      output match 'bash-command-done'
ok      output match /Apache Tomcat/
STATUS  SUCCEED

/ started
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11197    0 11197    0     0  3364k      0 --:--:-- --:--:-- --:--:--     0
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=ISO-8859-1
Transfer-Encoding: chunked
Date: Wed, 19 Oct 2016 12:03:02 GMT
bash-command-done
ok      scenario succeeded
ok      output match 'bash-command-done'
STATUS  SUCCEED

Заключение

На данном простом примере мы научились как использовать sparrowdo/sparrow в процессах разработки и тестирования chef кукбуков, а именно:

• запускать chef клиента с заданными параметрами и атрибутами на указанных хостах
• проверять существование процессов
• делать простейшие проверки на доступность web сервисов
• устанавливать инструменты, необходимые для тестирования.

На самом деле написать можно было еще много чего. Кейсов для использования sparrowdo и chef действительно большое количество. Просто возьмите готовые плагины тут или же напишите свой, ведь это действительно не сложно и начните использовать sparrowdo в вашей разработке.

Вся фишка системы sparrow, выгодно отличающая ee от других похожих инструментов, таких как serverspec/inspec в том, что функциональность не зашита жестко "в ядре", но пользователь может легко и быстро писать свои плагины, подходящие под его реалии и тут же использовать их посредством sparrowdo, вы даже можете "компилировать" более сложные сценарии, на основе sparrow плагинов с помощью так называемых sparrowdo модулей, которые пишутся на Perl6 ( об этом, если будет интерес, можно будет написать отдельную статью ).

Если данная статья вызвала у вас интерес — пишите, комментируйте, задавайте вопросы и… конструктивно критикуйте :)) буду признателен. Как обычно, единым центром для документации и репозиторием плагинов sparrow является сайт https://sparrowhub.org

Спасибо.

PS В конце статьи, как обычно, простенький вопрос, косвенно относящийся к теме статьи.