Second-order virtualization

UPDATE (2016-01-28) : now there is a Docker for this .

What to do when you need a bunch of small and cheap servers to test different versions of different sites? You can buy a Dedik and put OpenVZ on it. Although, OpenVZ will be rather small - there is a lot of memory. Better put XEN. Or KVM. Or even VMWare.
And let's start all this admin? - Of course not.

Virtualization my love

I'll start from afar, the first post is the same. More than six years ago, I still changed one hefty and expensive iron server to several small virtual ones. At that time, the masterhost was still a cake, and he had virtual machines only under Virtuozzo (however, as now). So we met. As time passed, the technology of containers fascinated me with its simplicity: a file system in a simple folder, full backups with one click, stable kernels from the host and, most importantly, no troubles with iron. The dream come true of a man who never assembled a server.

But when it becomes necessary to go beyond the restrictions of two or three tariffs convenient for the hoster, disappointments begin. In my case, I needed a lot of small servers created on the fly, which run different versions of the site for each branch in the repository. It takes a little memory, a little disk too, the processor and network are used occasionally. And most importantly, the server must be able to create from a script. And, best of all, fast.

And there are difficulties with this. Firstly, serious guys with their APIs (Linode, Amazon) use XEN from 20 bucks per server with a bunch of superfluous. This is not an option. Secondly, cheap hosting services constantly lag and fall off, even if you learn how to automatically create servers with them. Also not an option. And all this is logical, because expensive hosters are therefore expensive, because everyone knows how and works stably.

But what if you get cheap containers on expensive hosting? Starting with ubuntu 12.04, this is easier than steaming turnip: using Linux Containers, who have been living in the vanilla kernel for several years now. In the case of the server from Linode, these are a couple of apt-gets, a small patch, a reboot and 300 MB of space.

Vanilla core

Let's start:
sudo apt-get update && sudo apt-get upgrade

Let's put the LXC scripts:
sudo apt-get install lxc

And check the Linux Containers support in the kernel:
sudo lxc-checkconfig

Today’s default core at the linode supports containers very partially. Put the vanilla kernel from ubuntu:
sudo apt-get install linux-virtual

virtualbecause you need XEN support.

Further we move according to a piece of the instruction for PV-GRUB :
sudo apt-get install grub-legacy-ec2

When the hornbeam installer asked the question "where to put", I chose xvda.

Next, run `/ boot / grub / menu.lst` (near the end of the file):
- # defoptions=console=hvc0
+ # defoptions=console=hvc0 rootflags=nobarrier

and update the hornbeam config:
sudo update-grub-legacy-ec2

Everything, the container is ready to load the vanilla kernel. It remains only to ask our linode to boot in accordance with the hornbeam config. To do this, go to the virtual server profile editor, look at the kernel bit size (for me Latest 32 bit) and select the appropriate one pv-grub-*(for me pv-grub-x86_32) there. Next, save the profile and reboot the server.


Now you can check the LXC support in the kernel:
sudo lxc-checkconfig

and enjoy all the green items.

A new network interface will appear in the system:
ifconfig | grep lxc

which will combine all containers and release them on the Internet.

Finally, create the first container:
sudo lxc-create -t ubuntu -n demo1

The first time this process will drag on for about five minutes, as it lxc-createwill collect a brand new ubuntu in a folder /var/cache/lxc/, and then copy it to a folder /var/lib/lxc/demo1/rootfs/, where our new container will live.

You can start the container and get into its console like this (login and password ubuntu):
sudo lxc-start -n demo1

Go out, offer by clicking Ctrl+A Q. You lxc-startcannot get out of this, but you can simply close the terminal window. In the future, you can open the container console like this (login and password are the same):
sudo lxc-console -n demo1

It remains only to configure the launch of the container at startup:
sudo ln -s /var/lib/lxc/demo1/config /etc/lxc/auto/demo1.conf

Window to container

Now you need to somehow get to the site inside the container. Since he does not have a public IP, he will have to be smart.
You can pick it up iptables, since the containers are still on the same machine. Or disable network virtualization altogether, then all containers will have a common network interface with the virtual machine. But today, I am not ready to smoke such strong mana. Better, let's do the same as with any other private cluster: we will proxy HTTP requests to internal nodes. Today I have experience proxying only using nginx, and I will describe it. But if your site uses web sockets, then you need to proxy using HAproxy , since nginx is going to find their support only at the beginning of 2013.

So, the config for nginx:
user  nobody nogroup;
worker_processes  2;
events { worker_connections  1024; }
http {
  keepalive_timeout 90;
  access_log off;
  server {
    location / {
      proxy_redirect  http://localhost/  http://$host:$server_port/;
      proxy_buffering  off;

Everything is simple: nginx receives a request to port 80 of the virtual machine and transfers it to port 80 of the container with the address The response is not buffered, otherwise nginx likes to add pictures first to disk, and then to the network. The section serverwill need to be repeated for each site / container pair, changing server_nameand proxy_passin accordance with your situation. I am sure everyone can write a script for this.


On the one hand, what is there to talk about? This is a server for demos and development surveys. On the other hand, containers are good because they do not add complex abstractions. The processor receives everything in its original form, memory, too. Probably a little spent on the isolation logic of the file system, and even then, only when opening the file. Yes, the network is yes, virtual, but only at the IP level, no one emulates a network card. In the comments inkvizitor68sl names 1-2% of the losses. I believe that for an ordinary site there will be even less. Of course, if you enable quotas, much will change. But accounting and limiting resource consumption are always worth something. Because my containers are not limited in anything, they are simply isolated, so that it would be more convenient to deploy a bunch of small sites from a script. That's all.

Deep conclusions


Oh yes. Solaris and Frya, a hundred years old already, know how to do this out of the box. And the same thing can be done on OpenVZ (even inside LXC).

Also popular now: