Multi-hosting django applications using nginx + uwsgi + virtualenv

Task: deploy several django projects using different versions of django and different versions of python on the same server.

Instructions are provided for Ubuntu 12.04.

Training


To start, put the versions of python that interest us.

Necessary packages for compilation:
sudo apt-get install zlib1g zlib1g-dev zlibc libssl-dev

Put python, I set 2.7.4 and 3.3.1
wget http://python.org/ftp/python/2.7.4/Python-2.7.4.tar.bz2
tar -xf Python-2.7.4.tar.bz2
cd Python-2.7.4
./configure --prefix=/opt/python2.7/ --enable-unicode=ucs4
make && make install

wget http://python.org/ftp/python/3.3.1/Python-3.3.1.tar.bz2
tar -xf Python-3.3.1.tar.bz2
cd Python-3.3.1
./configure --prefix=/opt/python3.3/
make && make install

Let's create directories for our project configs.
  • /home/hosting/.nginx/ - there will be nginx configs
  • /home/hosting/.uwsgi/ - there will be uwsgi configs
  • /home/hosting/.virtualenvs/ - here will be located the virtual environments of the projects
  • / home / hosting / project1 / - files of the first django project
  • / home / hosting / project2 / - files of the second django project

Nginx installation


apt-get install nginx-full

Why is nginx-full and not nginx? Nginx-full already includes a module for working with uwsgi.

You need to tell nginx where to load the virtual host configs from.
Open /etc/nginx/nginx.conf.
After include /etc/nginx/sites-enabled/*; adding the line include /home/hosting/.nginx/*.conf;
Now you need to create nginx-configs of virtual hosts.
  • /home/hosting/.nginx/project1.conf
  • /home/hosting/.nginx/project2.conf

Config example:
Hidden text
server {
    server_name project1.com;
    access_log /var/log/project1.access.log;
    error_log /var/log/project1.error.log;
    location / {
        uwsgi_pass unix:/tmp/project1.sock;
        include /etc/nginx/uwsgi_params;
    }
    location /static/ {
        alias /home/hosting/project1/static/;
    }
    location /media/ {
        alias /home/hosting/project1/media/;
    }
}

You need to give permissions to the /home/hosting/.nginx directory to www-data user (or to the user nginx is running under).
chown -R www-data:www-data /home/hosting/.nginx/

We start nginx
service nginx start

Install virtualenvwrapper


virtualenvwrapper - a handy wrapper around virtualenv.
We put pip if it is not worth it:
sudo apt-get install python-pip

We put virtualenvwrapper:
pip install virtualenvwrapper

In ~ / .bashrc add:
export WORKON_HOME=/home/hosting/.virtualenvs/
source /usr/local/bin/virtualenvwrapper.sh

Relocate to the console so that .bashrc boots. We should now have the mkvirtualenv command available in the console.

We place the project files in the directories:
  • / home / hosting / project1 /
  • / home / hosting / project2 /

For each project, create a virtual environment. Let's say project1 will run on python 2.7 and project2 on 3.3.
mkvirtualenv project1 -p /opt/python2.7/bin/python
deactivate
mkvirtualenv project2 -p /opt/python3.3/bin/python3
deactivate

For each project, we put dependencies in a virtual environment. (In my case, the dependencies are written in the requirements.txt file in the root of each project)
workon project1
cd /home/hosting/project1
pip install -r requirements.txt
workon project2
cd /home/hosting/project2
pip install -r requirements.txt

Configuring uwsgi.


We will configure it in the emperor mode (--emperor), as This mode is specifically designed for multi-hosting.

In emperor mode, uwsgi will automatically load the configs from the specified directory, which is convenient, that is, we run uwsgi once, and then he creates the processes for all applications in the configs.

By default, uwsgi executes the project code with the python that is in the current environment, so we will need to run uwsgi from virtualenv.
Since we have several different versions of python, the uwsgi launched, for example, from under python 2.7, will not be able to serve the django application with the environment from python 3.3.
So we will create an emperor for each version of python, and we will group application configs according to the interpreter version.

We create virtual environments for emperors.
mkvirtualenv python27 -p /opt/python2.7/bin/python
deactivate
mkvirtualenv python33 -p /opt/python3.3/bin/python3
deactivate

Now you need to put uwsgi in each virtualenv and configure it.
workon python27
pip install uwsgi
workon python33
pip install uwsgi

Create config directories for each uwsgi emperor.
mkdir /home/hosting/.uwsgi/python27
mkdir /home/hosting/.uwsgi/python33

Create uwsgi-configs for each project.
/home/hosting/.uwsgi/python27/project1.ini
Hidden text
[uwsgi]
protocol = wsgi
master = true
processes = 1 # по количеству ядер
socket = /tmp/project1.sock
# Докидываем в pythonpath библиотеки из виртуаленва, т.к uwsgi в динамическом режиме не умеет искать библиотеки в virtualenv
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
pythonpath = /home/hosting/.virtualenvs/project1/lib/python27.zip
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/plat-linux2
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-tk
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-old
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-dynload
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/site-packages
 # почему-то в virtualenv не все нужные файлы есть, поэтому пришлось добавить эту строчку
pythonpath = /opt/python2.7/lib/python2.7
chdir = /home/hosting/project1
virtualenv = /home/hosting/.virtualenvs/project1
env = DJANGO_SETTINGS_MODULE=settings
module = django.core.handlers.wsgi:WSGIHandler()
no-site = true
vhost = true
chmod-socket = 666

The second project config is similar. The file name must end with .ini, otherwise uwsgi will not pick up this config.

Now you need to register uwsgi as a service in the system. I used upstart, it is in the ubunt from the box.
Let's create two config files:
/etc/init/uwsgi27.conf
description "uWSGI Emperor (python 2.7)"
start on runlevel [2345]
stop on runlevel [06]
exec /home/hosting/.virtualenvs/python27/bin/uwsgi --master --emperor /home/hosting/.uwsgi/python27 --logto /var/log/uwsgi27.emperor.log

/etc/init/uwsgi33.conf
description "uWSGI Emperor (python 3.3)"
start on runlevel [2345]
stop on runlevel [06]
exec /home/hosting/.virtualenvs/python33/bin/uwsgi --master --emperor /home/hosting/.uwsgi/python33/ --logto /var/log/uwsgi33.emperor.log

Users and Security


From root, we will only run “imperial” processes, and the projects themselves will be under their own users.

Create a user for each of the projects.
adduser --no-create-home --disabled-login --disabled-password www-project1
adduser --no-create-home --disabled-login --disabled-password www-project2

Add uid gid parameters to each of uwsgi ini-configs
uid = www-project1 # пользователь
gid = www-project1 # группа

Set the correct access rights
Hidden text
chown -R www-data:www-data /home/hosting/.nginx
chmod -R 770 /home/hosting/.nginx
chown -R root:root /home/hosting/.uwsgi
chmod -R 770 /home/hosting/.uwsgi
chown -R root:root /home/hosting/.virtualenvs/python27 /home/hosting/.virtualenvs/python33
chmod -R 775 /home/hosting/.virtualenvs
chown -R www-project1:www-project1 /home/hosting/project1 /home/hosting/.virtualenvs/project1
chown -R www-project2:www-project2 /home/hosting/project2 /home/hosting/.virtualenvs/project2

Launch uwsgi
service uwsgi27 start
service uwsgi33 start

We check - everything should work.
If something does not work, look at the nginx logs specified in the project config and the logs of the uwsgi emperor.
A sign that uwsgi has successfully deployed the application is the presence of a line WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x135e280 pid: 21737 (default app)in the uwsgi log.
To add a new application, you need to create a config in .nginx and .uwsgi and restart nginx. uwsgi itself will pick up a new config.

References


projects.unbit.it/uwsgi/wiki/MultiPython
projects.unbit.it/uwsgi/wiki/DynamicVirtualenv
auphonic.com/blog/2011/06/18/django-deployment-nginx-uwsgi-virtualenv-and-fabric
eshlox.net / en / 2012/09/11 / nginx-uwsgi-virtualenv-and-django-ubuntu-1204
uwsgi-docs.readthedocs.org/en/latest/Emperor.html

PS


The method described in the article is not very beautiful in terms of architecture; Initially, I was hoping to get by with one uwsgi emperor and resolve the interpreter version with the plugin parameter in the application config. But I could not build a uwsgi-plugin for python, so I had to do it differently.

Also popular now: