Ruby on the rails: production and deployment for dummies

A year ago, I brought my first rail application to an acceptable form. The issue of using ready-made code in production did not interest me before. Why all of a sudden? A simple language, a laconic framework - deploying is clearly no more complicated than overcoming the mental brake after PHP.

The Rails development team recommends using Phusion Passenger , it is something like mod_php - installed, placed files and flew. At the time of studying the issue on the forums there were enough battles about the performance of solutions; Passenger was not their favorite.

I asked the site’s technical director with a million unique employees a day for advice - he sent me to google on Nginx and Unicorn. The production setup instruction found on Habré, dated 2009. Among other things, she was simply overwhelmed by the flaws of the lessons "How to draw an owl."

Some parts of the process are chewed in English in some places, but the monolithic tutorial never caught my eye. The tradition of the rail community is based on the principle of sharing the results and experience of solving problems.

The text is unlikely to be interesting to experienced rail guides, but if they take the time to comment, I will be grateful and make the necessary changes.

- What will we talk about?
- This instruction will help beginners to select and configure hosting, as well as prepare an existing project for an initial deployment and systematic rolling out of updates.

- And in more detail?
- We will use Ubuntu 14.04, RVM, Nginx, Unicorn and Capistrano. The text consists of two chapters: preparing the project and setting up the server. All local manipulations are described in Mac OS X. A modern IDE like RubyMine will not be superfluous to execute procedures , if not, TextMate will do . All described actions are fixed in a  special repository .

Chapter One Hosting

Hosting Choice

A sweet couple of Nginx and Unicorn has a very impressive appetite for RAM, and the rails themselves require the installation of a number of additional software. These limitations clearly indicate the need for VPS. There is an option to use specialized hosting like Heroku , but for beginners it will be useful to do the setup process by hand.

As part of this text, I will use the freshly created Digital Ocean droplet based on Ubuntu 14.04. (There are enough promotional codes for a month — two free use on the Internet, who needs a referral with ten dollars in the account — I’ll give a link.)

System Pack Update

We go into the system under the root and update the packages:

sudo apt-get update
sudo apt-get upgrade

Install Git and NodeJS

sudo apt-get install git-core nodejs

User Creation

I assume that you (like me) have a commercial interest in deployment. We will create a separate user (in other words, a client), in the home directory of which we will deploy the application. In addition to the obvious, this approach provides advantages in the form of a separate RVM (ruby version manager), which will allow using different versions of the interpreter and gems for different clients and applications. We will create a user demoand add him to the group sudo.

sudo adduser demo
sudo adduser demo sudo

Close the session, log in as a user demo.

Cancel password request for sudo

Some deployment procedures will require superuser privileges. In order for the commands to be executed using Capistrano and not cause errors, it is necessary to disable the password request. Edit the file sudoers: sudo nano / etc / sudoers.

# Найдите следующую строку:
%sudo ALL=(ALL:ALL) ALL
# И приведите ее к виду:
%sudo ALL=(ALL) NOPASSWD:ALL

The official Capistrano documentation explains that the native deploy phases do not require sudo; however, when it comes to automating all processes, passwordless sudo is indispensable.

SSH key generation

Two key pairs are required. Using one of them, you (and Capistrano) will authenticate to the server from the local computer. The second pair, you give the server access to the repository (the so-called deployment key). In both cases, leave passphrases blank.

First pair (must be generated locally ):

ssh-keygen -t rsa -b 2048

If you are familiar with the procedure for generating and using keys, then choose the path to the file and durability to your liking; otherwise, leave it by default.

Now you need to copy the contents of the public key (by default ~/.ssh/id_rsa.pubon the local computer) and add it to the file ~/.ssh/authorized_keyson the server. After this simple manipulation using SSH, you can connect to the server without a password. If not, check the rights on the server: 700 for ~/.sshand 600 for ~/.ssh/*.

The second pair (on the server):

ssh-keygen -t rsa -b 2048

Similarly, the content from the server ~/.ssh/id_rsa.pubmust be added to the list of deployment keys (in Github, you can find them in the settings of each repository).

Installing Fresh Nginx

The version of Nginx available in Ubuntu is often older than the official developer repository. I try to use the latest version, but if you have different views, install Nginx yourself and skip this part.

We will add the official Ubuntu repositories to the sudo nano /etc/apt/sources.list system list. Add the lines to the end of the file:

# Nginx official repository
deb http://nginx.org/packages/ubuntu/ trusty nginx
deb-src http://nginx.org/packages/ubuntu/ trusty nginx

Remember that the parameters used are relevant only for Ubuntu 14.04. Look for information on installing on other versions of the OS on the developer's website . To install Nginx from these repositories, you will also need to download and add a key to the system:

wget http://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key

Now you can update the list of available packages and install Nginx:

sudo apt-get update
sudo apt-get install nginx

Delete the default configs from  /etc/nginx/conf.d. (Of course, do not try to do this if you are not working on a “clean” server.) We will create a virtual host for the application in the next chapter.

sudo rm /etc/nginx/conf.d/*
sudo service nginx restart

Install RVM

\curl -sSL https://get.rvm.io | bash -s stable
source ~/.rvm/scripts/rvm

In the case of a clean server, you will also need to install some dependencies:

rvm requirements

It remains to install directly the Ruby version of your application. For example, the new stable version 2.1.3:

rvm install 2.1.3
rvm --default use 2.1.3

You can verify the installation using the ruby ​​-v and rvm info commands

Install application components

Surely, your application will use a number of components (not including gems), such as a database or Imagemagick GPU, you will have to finish installing them yourself. Here it’s worth adding a small remark (if you forget about it, an automatic deployment will fail): rails sometimes require additional packages to work with some components. For example, to use MySQL, you need to install, among other things, a package libmysqlclient-dev.

Chapter Two Preparing the Application

Using git

I assume that you already have a ready-made application. The first rule is that the project must be managed by Git. This is the de facto standard in the rail world (even the .gitignore file at the root of the rail application being created explicitly alludes to this). Moreover, the latest versions of the Capistrano gem, which will be directly responsible for the deployment, natively support only this system.

What is Capistrano

The official definition of Capistrano is: “A remote server automation and deployment tool”. From the user's point of view, Capistrano is a thing that will allow you to execute an arbitrary set of commands on a remote server via SSH. There are other tools for the deployment (for example, Mina), but so far Capistrano is also a certain standard, especially since it allows you to perform parallel deployment of the application directly to a number of servers, including those divided by role.

How Capistrano Works

On the server application structure under the control of Capistrano generally consists of three directories: repo, releasesand  shared. The first one contains a copy of the repository, the second contains releases, the third contains shared files necessary for the application and independent of the release. Also at the root there is a symlink currentthat refers to the version of the current release and the deploy log file.

When you (from your local computer) give the Capistrano command a deploy, an SSH connection to the server is established and a simple algorithm starts. To start, Capistrano checks the remote repository and receives the missing commits. After a new release is created (in the directory releases). The current version of the code is transferred there and a number of tests are performed there.

Simply put, for every new release Capistrano performs the usual commands like bundle install, rake db:migrate, rake assets:precompile, constantly checking for conflicts and errors. Missed a semicolon in  default.scss, didn’t overwrite the current one Gemfile.lock, connected Paperclip, but forgot to install Imagemagic on the server? In all these cases, Capistrano will display an error and terminate the installation without affecting the current working release. If the deployment was successful, but the result did not suit you, using Capistrano can be done rollbackfor the previous release.

Organization of configuration files

Each release for Capistrano is an independent entity. The next time it is deployed, it is completely replaced, so a number of files must be stored outside the application structure (as a rule, content downloaded by users and generated by the application, as well as a number of configs).

We will need a common directory, the necessary files from which we will link to each new release during the deployment; let's call it shared and create locally in the project root mkdir ./shared, previously adding an exception to .gitignore. (Of course, I will not add an exception to the training repository.)

Now, in the shared directory, we will create a future structure in advance. First we need the configand  folders run. In  configput relevant for the battle server database.ymland secrets.yml. (Personally, I prefer to move these two files from config on the local computer, where I can then create links to them.)

mv ./config/database.yml ./shared/config
ln -fs ./shared/config/database.yml ./config/database.yml
mv ./config/secrets.yml ./shared/config
ln -fs ./shared/config/secrets.yml ./config/secrets.yml

Nginx configuration

Here - in  shared/config- we will create a config for Nginx shared/config/nginx.conf. In its basic form, it consists of two small parts: upstream and a typical virtual host. Be especially careful with the paths (in this and all further) in the configuration files. 90% of the errors that occurred during the deployment were associated with them.

upstream unicorn {
    server unix:/home/demo/application/shared/run/unicorn.sock fail_timeout=0; 
}
server { 
    listen 80 default; 
    root /home/demo/application/current/public;
    try_files $uri/index.html $uri.html $uri @app;
    location ^~ /assets/ { 
        expires max;
        add_header Cache-Control public;
    }
    location @app {
        proxy_pass http://unicorn;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;
    }
    error_page 500 502 503 504 /500.html;
    location = /500.html {
        root /home/demo/application/current/public;
    }
}

We use our VPS for a single application, so the host will be used by default. (No need to explain that in a different situation you will have to explicitly specify server_name and take care of the uniqueness of the upstream names?)

Unicorn Configuration

Since we created an upstream in which we indicated the path to the Unicorn socket, let's move on to its config. You must add Unicorn's gem  Gemfile. Do not forget that adding new gems should be accompanied by bundle install and committing to the repository. If you forget about git push, then the old version of the code will be uploaded to the server, which, due to the lack of these gems, will cause errors when deploying.

group :production do
  gem 'unicorn', '~> 4.8.3'
end

shared/configCreate a file in the directory unicorn.rb. For the most part, we should point out the paths to the components of our application. For convenience, this can be done using variables.

# Рабочие директории приложения на сервере
root        = '/home/demo/application'
rails_root  = "#{root}/current"
# Файлы, хранящие идентификаторы запущенных Unicorn-процессов
pidfile     = "#{root}/shared/run/unicorn.pid"
pidfile_old = pidfile + '.oldbin'
pid pidfile
# Главные параметры
worker_processes 1
preload_app true
timeout 30
# Путь к сокету
listen "#{root}/shared/run/unicorn.sock", :backlog => 1024
# Путь к лог-файлам
stderr_path "#{rails_root}/log/unicorn_error.log"
stdout_path "#{rails_root}/log/unicorn.log"
# Установки сборщика мусора
GC.copy_on_write_friendly = true if GC.respond_to?(:copy_on_write_friendly=)
# Блок инструкций, выполняемых до запуска сервера
before_exec do |server|
  ENV["BUNDLE_GEMFILE"] = "#{rails_root}/Gemfile"
end
# Инструкции для управления воркерами и состоянием соединения с БД
before_fork do |server, worker|
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!
  if File.exists?(pidfile_old) && server.pid != pidfile_old
    begin
      Process.kill("QUIT", File.read(pidfile_old).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
    end
  end
end
after_fork do |server, worker|
  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
end

Some system values ​​are also indicated ( full list and documentation ). Pay attention to the number of workers - worker_processes, - each of which consumes a certain amount of memory (how much depends on the application) and on  timeout(usually from 15 to 30 seconds). Many mention about preload_app(the value of falsewhich can reduce the start time of the worker); let's stop for a  truewhile, and then you decide for yourself.

Connect and configure Capistrano

Essential Gems

You  Gemfileneed to add Capistrano and a series of gems that implement its connection with RVM, Bundler and Rails.

group :development do
  gem 'capistrano', '~> 3.2.1'
  gem 'capistrano-rvm', '~> 0.1.1'
  gem 'capistrano-bundler', '~> 1.1.3'
  gem 'capistrano-rails', '~> 1.1.2'
end

It remains to run bundle install, and then initialize Capistrano using cap install. A set of files will be created, a list of which you will see in the console. We will work with three of them: Capfile, config/deploy.rband  config/deploy/production.rb. (In config/deployCapistrano, it creates default files staging.rband  production.rb. We will only configure the battle server using production.rb.)

Update Capfile

All gems associated with Capistrano, the functionality of which we are going to use, must be connected in the root Capfile. Take a look at the default file and uncomment the necessary lines or use the following contents:

require 'capistrano/setup'
require 'capistrano/deploy'
require 'capistrano/rvm'
require 'capistrano/bundler'
require 'capistrano/rails'
Dir.glob('lib/capistrano/tasks/*.rb').each { |r| import r }

Setting up a production server

Open the file config/deploy/production.rband carefully look at its contents. It will help you understand the format and customization options. In general, if you use authorization using an SSH key, you won’t have to write any exotic; just one line:

server '178.62.252.46', user: 'demo', roles: %w{web app}

Deployment Scenario

We got to the most interesting part, the file config/deploy.rb. It is he who describes the parameters, procedures and scenario of the upcoming deployment. I will describe each block of the file, but if you want to look at it in its final form, use the repository.

Required Parameters

First of all, you need to specify the version of Capistrano for which this scenario is intended:

lock '3.2.1'

Capistrano requires a number of required parameters:

# Репозиторий
set :repo_url, 'git@github.com:eboyko/deneb.git'
# Используемое окружение
set :rails_env, 'production'

As well as a parameter deploy_tothat determines the path for deploying the application on the server. We already said above that we mean commercial interest, therefore we will make the script file more universal with the help of variables and set the missing parameter:

# Имя пользователя
set :username, 'demo'
# Имя приложения
set :application, 'application'
# Путь для деплоя
set :deploy_to, "/home/#{fetch(:username)}/#{fetch(:application)}"

We also set the parameter log_level(the default value :debugmakes Capistrano too talkative):

set :log_level, :info

Note Capistrano global variables (sort of shared_path) can be used directly; defined using set - through the method fetch.

Non-volatile data

We have already examined the need for a working release to access data created by previous versions. For example, you store files uploaded by users in  public/upload. To connect them to each new release,  config/deploy.rbyou can set the parameter in :linked_dirs:

set :linked_dirs, %w{public/upload}

At each deployment, it public/uploadwill be replaced by a symlink leading to a directory shared/public/uploadwhere data was collected during the work of previous releases. Similarly, with the help :linked_files, separate files are linked:

set :linked_files, %w{config/secrets.yml config/database.yml}

Note that the addition of  :linked_filesthe specified files ( config/secrets.ymland  config/database.yml) is mandatory. Otherwise, the deployment will fail due to lack of connection to the database.

Procedures

Capistrano allows you to create sets of procedures that, for convenience, can be combined into namespaces. Namespace :deployalready exists, it can only be supplemented; all procedures included in it for the server productioncan be called with the cap production deploy command

Set-up

As a result of the previous steps, we have formed a directory sharedin which, among other things, are configuration files ( shared/config). The logical first step is to upload them to the server. To do this, everything in the same file config/deploy.rb, in the namespace :setup, we will write the procedure:

namespace :setup do
  desc 'Загрузка конфигурационных файлов на удаленный сервер'
  task :upload_config do
    on roles :all do
      execute :mkdir, "-p #{shared_path}"
      ['shared/config', 'shared/run'].each do |f|
        upload!(f, shared_path, recursive: true)
      end
    end
  end
end

As you understand, the procedure involves creating shared_path(because there is no structure on the server yet) and loading the local directory there shared/config. You can execute it with the cap production setup: upload_config command

Nginx Management

I noted above that Capistrano is essentially a way to execute arbitrary commands on a remote server. We downloaded configuration files, including for Nginx. Now we will write several procedures for management: creating a symlink to the config and reload / restart service (they will require rights sudo).

namespace :nginx do
  desc 'Создание симлинка в /etc/nginx/conf.d на nginx.conf приложения'
  task :append_config do
    on roles :all do
      sudo :ln, "-fs #{shared_path}/config/nginx.conf /etc/nginx/conf.d/#{fetch(:application)}.conf"
    end
  end
  desc 'Релоад nginx'
  task :reload do
    on roles :all do
      sudo :service, :nginx, :reload
    end
  end
  desc 'Рестарт nginx'
  task :restart do
    on roles :all do
      sudo :service, :nginx, :restart
    end
  end
  after :append_config, :restart
end

Have you noticed a pleasant trifle? - You can specify the sequence of execution of various procedures both within the same namespace, and between them.

Office Unicorn

Surely there is already a gem that extends Capistrano in the Unicorn control plane, but I was very interested to know how it actually works and works. Therefore, now, similarly to the previous examples, we will write two procedures (start and end) for Unicorn.

set :unicorn_config, "#{shared_path}/config/unicorn.rb"
set :unicorn_pid, "#{shared_path}/run/unicorn.pid"
namespace :application do
  desc 'Запуск Unicorn'
  task :start do
    on roles(:app) do
      execute "cd #{release_path} && ~/.rvm/bin/rvm default do bundle exec unicorn_rails -c #{fetch(:unicorn_config)} -E #{fetch(:rails_env)} -D"
    end
  end
  desc 'Завершение Unicorn'
  task :stop do
    on roles(:app) do
      execute "if [ -f #{fetch(:unicorn_pid)} ] && [ -e /proc/$(cat #{fetch(:unicorn_pid)}) ]; then kill -9 `cat #{fetch(:unicorn_pid)}`; fi"
    end
  end
end

Remember to set the parameter values :unicorn_configand  :unicorn_pid.

Procedures before and after the deployment

After the deployment, we need to delete the oldest releases (by default Capistrano stores the last five), clear the caches and restart Unicorn. The main work block :deploywill look something like this:

namespace :deploy do
  after :finishing, 'application:stop'
  after :finishing, 'application:start'
  after :finishing, :cleanup
end

Putting procedures into separate files

In order not to clutter up the file deploy.rb, the written procedures can (and maybe should) be taken out of it. The  Capfilelast line is responsible for importing such tasks from the directory lib/capistrano/tasks- this is where it is worth moving this part of the logic.

Deployment execution

From this point on, all you need to do to roll out a new version of your application is to commit the changes, do git push and use the cap production deploy command.

By analogy, you can configure the server staging, on which the implementation of cap staging deploy.

Something does not work? - Leave comments, let's complement the article.

Updates expected:

- A brief overview of existing application servers (in the context of choosing Unicorn);
- process management (unicorn worker killer and some manager);
- selection and configuration of an analogue of RVM;
- Install Nginx using PPA;
- edits regarding passwordless sudo;