March 13, 2014 at 16:34

We use nginx, docker, skydns and skydock to update the code on the fly (zero-downtime deployment)

Tools we will use


Docker


Docker is a simple and elegant library for creating lightweight virtual containers isolated from each other, in which any code can be executed. Absolutely resource-demanding, minimal overhead.

Having collected the container once, it can be reused.

A simple example is a Redis database. If we need several Redis servers on the same computer, in the usual approach we will have to change the configuration files in / etc / redis and change the files in /etc/init.d. You can write a bash script, but this does not make the process easier.

In the case of Docker, we can use the following command:

docker run -d --name test-redis-server dockerfile/redis

This command will download the Redis container from the main repository ( index.docker.io ), run it in the background, and give the newly created container the name test-redis-server.

This container can then be started with the command:

docker start test-redis-server

Skydns


Most recently, Docker developers introduced the --link tool when launching a container so that multiple containers can be linked using ENV environment variables. For example, to associate a web application container with containers of Redis, Postgresql, Elasticsearch, etc.

This is convenient, but we need to monitor the correctness of ENV and change the code when changing the conditions for launching containers and the --link parameter.

This is called Service Discovery, there are many solutions for it (you can read more about existing solutions in the Open-Source Service Discovery article ).

One such solution is SkyDNS. A small local DNS and DNS proxy server written in Go (it is important to note that Go is extremely efficient in terms of resource consumption, up to about 6MB of memory on startup. I personally conducted several synthetic tests for myself, the maximum memory consumption with 50 simultaneous simple queries was about 20MB).

SkyDNS allows you to add DNS records using a simple API (in more detail: SkyDNS ), and then make regular queries that return SRV, A or AAAA records.

If the record is not found, then SkyDNS will send a request to google's public dns server (8.8.8.8/8.8.4.4).

SkyDNS uses its own interesting scheme for the domain:

......skydns.local

If environment = production, and service = redis, then you can make a request to redis.production.skydns.local , which will return one or more records (by default, it returns an A record).

Skydock


Skydock is a small program that integrates Docker and SkyDNS.

Skydock uses the SkyDNS domain schema as follows:

...skydns.local

container-name is the name of the Docker image without a repository (e.g. crosbymichael / redis => redis or test / cool-api => cool-api). name is the name of the container assigned to it using the --name parameter.

Skydock automatically detects running containers and adds them to SkyDNS.

Docker assigns a separate IP address for each container, therefore if we start the container as follows:

docker run -d --name redis-test-app dockerfile/redis

We will be able to make a request (later it will be described why we make a request for 172.17.42.1) and get an A record in response, that is, the IP address of the container in which Redis is running.

dig @172.17.42.1 redis-test-app.redis.dev.skydns.local
dev.skydns.local.	27	IN	A	172.17.0.3

The default Redis port is 6379, so we can do something like this in code (example in a pseudo-language):

redisConn = redis.Connect("redis-test-app.redis.dev.skydns.local:6379")

Installation


We will not consider installing Docker, it is very simple and described in detail here: Start using Docker

We need to run 2 containers - one with SkyDNS, the other with Skydock. Unlike the author of Skydock, I recommend starting SkyDNS with the additional parameter -p 172.17.42.1:8080:8080 - this will allow your containers to use the SkyDNS API directly for their own needs.

docker pull crosbymichael/skydns
docker run -d -p 172.17.42.1:53:53/udp -p 172.17.42.1:8080:8080 --name skydns crosbymichael/skydns -nameserver 8.8.8.8:53 -domain skydns.local

IP 172.17.42.1 is the docker0 bridge, used by Docker to configure its own network. We specified the skydns.local domain, although you can do any of your choice, for example: docker, super.local. skydns.local, in my opinion, is more convenient and universal, all the more it is used by default in SkyDNS.

Next, launch Skydock:

docker pull crosbymichael/skydock
docker run -d -v /var/run/docker.sock:/docker.sock --name skydock -link skydns:skydns crosbymichael/skydock -ttl 30 -environment dev -s /docker.sock -domain skydns.local


Here you need to specify TTL (if it is smaller - more dynamic environment, you can make more changes to the architecture faster. If more - less dynamic, more cached environment) and environment - this can be absolutely any line (dev, development, production, stage, qa )

If everything went well, then all the necessary components are running.

The only thing to do is when you start the container with your application, you must specify the primary DNS server:

docker run -d --dns 172.17.42.1 test/cool-api

Our container with cool-api will be available in the local DNS at: cool-api.dev.skydns.local - this will list IP addresses of all containers with the name cool-api. We will use this property to configure nginx.

If you specify --name api1 when starting the container, then it will be available by api1.cool-api.dev.skydns.local - just 1 container with the name api1.

Inside the container, you can now specify the domain directly, since it will use the local DNS: redis.dev.skydns.local - will return A records of all containers running Redis. Naturally, only 1 address will be selected to which the Redis client will connect.

nginx


Using this magic of local DNS, we can do for example:

  • Load balancing - web server or database connections
  • Simple query routing by criteria. For example, creating an image test / cool-api-v1 - we can send requests to the API v1 to one container ( cool-api-v1.dev.skydns.local ), and use cool-api.dev.skydns.local as the latest version. At the same time with automatic balancing
  • Update code on the fly - what we need

Add the following configuration file to nginx:

server {
        listen 80;
        server_name super-cool-domain.com;
        # говорим nginx использовать SkyDNS
        resolver 172.17.42.1 valid=5s;
        resolver_timeout 5s;
        # Это нам необходимо сделать для того, чтобы nginx использовал локальный DNS. По-другому nginx не понимает
        set $dns cool-api.production.skydns.local;
        location /api {
            proxy_pass http://$dns:8080;
        }
        # Чисто для примера
        location / {
            try_files $uri /index.html;
        }
    }

We simply specify that nginx use the local DNS server, then specify where to go - the $ dns variable and do the good old proxy_pass to the port that your application uses.

Now, when starting a new container, Skydock will add it to SkyDNS, nginx will proxy requests to this container. When the container stops, Skydock will remove its address from SkyDNS.

By launching 2-3 containers in this way, we can balance the load between them.

Let's start some additional containers, nginx will automatically begin to proxy requests to them. After that, we can stop the work of the old ones - thereby making the update on the fly.

This will not require reconfiguration. Everything works by default. Essentially, all you need to do to change the code is:

# Скачиваем новую версию контейнера с обновленным кодом веб приложения
docker pull test/cool-api
# Сохраняем ID текущих контейнеров
OLDPORTS=( `docker ps | grep cool-api | awk '{print $1}'` )
# Запускаем новый контейнер
docker run -d test/cool-api
# Останавливаем старые
for i in ${OLDPORTS[@]}
do
	echo "removing old container $i"
	docker kill $i
done

Conclusion


The author of Skydock wants to develop the idea and make support for several servers in the data center. At the moment, this is not implemented, although SkyDNS itself is already used in production environments on many servers.

The article also does not touch on the topic of docker repositories. There is an open source solution that allows you to create a private repository of images.

The solution has almost no effect on performance; Docker and SkyDNS are very well designed and use the efficient and fast Go language.
At first glance, it seems that this is some complication of the process, but the result is a very flexible solution that does not need to be additionally configured after the initial setup.
Tags:

Also popular now: