Multi-hosting django applications using nginx + uwsgi + virtualenv
- From the sandbox
- Tutorial
Task: deploy several django projects using different versions of django and different versions of python on the same server.
Instructions are provided for Ubuntu 12.04.
To start, put the versions of python that interest us.
Necessary packages for compilation:
Put python, I set 2.7.4 and 3.3.1
Let's create directories for our project configs.
Why is nginx-full and not nginx? Nginx-full already includes a module for working with uwsgi.
You need to tell nginx where to load the virtual host configs from.
Open /etc/nginx/nginx.conf.
After
Now you need to create nginx-configs of virtual hosts.
Config example:
You need to give permissions to the /home/hosting/.nginx directory to www-data user (or to the user nginx is running under).
We start nginx
virtualenvwrapper - a handy wrapper around virtualenv.
We put pip if it is not worth it:
We put virtualenvwrapper:
In ~ / .bashrc add:
Relocate to the console so that .bashrc boots. We should now have the mkvirtualenv command available in the console.
We place the project files in the directories:
For each project, create a virtual environment. Let's say project1 will run on python 2.7 and project2 on 3.3.
For each project, we put dependencies in a virtual environment. (In my case, the dependencies are written in the requirements.txt file in the root of each project)
We will configure it in the emperor mode (--emperor), as This mode is specifically designed for multi-hosting.
In emperor mode, uwsgi will automatically load the configs from the specified directory, which is convenient, that is, we run uwsgi once, and then he creates the processes for all applications in the configs.
By default, uwsgi executes the project code with the python that is in the current environment, so we will need to run uwsgi from virtualenv.
Since we have several different versions of python, the uwsgi launched, for example, from under python 2.7, will not be able to serve the django application with the environment from python 3.3.
So we will create an emperor for each version of python, and we will group application configs according to the interpreter version.
We create virtual environments for emperors.
Now you need to put uwsgi in each virtualenv and configure it.
Create config directories for each uwsgi emperor.
Create uwsgi-configs for each project.
/home/hosting/.uwsgi/python27/project1.ini
The second project config is similar. The file name must end with .ini, otherwise uwsgi will not pick up this config.
Now you need to register uwsgi as a service in the system. I used upstart, it is in the ubunt from the box.
Let's create two config files:
/etc/init/uwsgi27.conf
/etc/init/uwsgi33.conf
From root, we will only run “imperial” processes, and the projects themselves will be under their own users.
Create a user for each of the projects.
Add uid gid parameters to each of uwsgi ini-configs
Set the correct access rights
Launch uwsgi
We check - everything should work.
If something does not work, look at the nginx logs specified in the project config and the logs of the uwsgi emperor.
A sign that uwsgi has successfully deployed the application is the presence of a line
To add a new application, you need to create a config in .nginx and .uwsgi and restart nginx. uwsgi itself will pick up a new config.
projects.unbit.it/uwsgi/wiki/MultiPython
projects.unbit.it/uwsgi/wiki/DynamicVirtualenv
auphonic.com/blog/2011/06/18/django-deployment-nginx-uwsgi-virtualenv-and-fabric
eshlox.net / en / 2012/09/11 / nginx-uwsgi-virtualenv-and-django-ubuntu-1204
uwsgi-docs.readthedocs.org/en/latest/Emperor.html
The method described in the article is not very beautiful in terms of architecture; Initially, I was hoping to get by with one uwsgi emperor and resolve the interpreter version with the plugin parameter in the application config. But I could not build a uwsgi-plugin for python, so I had to do it differently.
Instructions are provided for Ubuntu 12.04.
Training
To start, put the versions of python that interest us.
Necessary packages for compilation:
sudo apt-get install zlib1g zlib1g-dev zlibc libssl-dev
Put python, I set 2.7.4 and 3.3.1
wget http://python.org/ftp/python/2.7.4/Python-2.7.4.tar.bz2
tar -xf Python-2.7.4.tar.bz2
cd Python-2.7.4
./configure --prefix=/opt/python2.7/ --enable-unicode=ucs4
make && make install
wget http://python.org/ftp/python/3.3.1/Python-3.3.1.tar.bz2
tar -xf Python-3.3.1.tar.bz2
cd Python-3.3.1
./configure --prefix=/opt/python3.3/
make && make install
Let's create directories for our project configs.
- /home/hosting/.nginx/ - there will be nginx configs
- /home/hosting/.uwsgi/ - there will be uwsgi configs
- /home/hosting/.virtualenvs/ - here will be located the virtual environments of the projects
- / home / hosting / project1 / - files of the first django project
- / home / hosting / project2 / - files of the second django project
Nginx installation
apt-get install nginx-full
Why is nginx-full and not nginx? Nginx-full already includes a module for working with uwsgi.
You need to tell nginx where to load the virtual host configs from.
Open /etc/nginx/nginx.conf.
After
include /etc/nginx/sites-enabled/*;
adding the line include /home/hosting/.nginx/*.conf;
Now you need to create nginx-configs of virtual hosts.
- /home/hosting/.nginx/project1.conf
- /home/hosting/.nginx/project2.conf
Config example:
Hidden text
server {
server_name project1.com;
access_log /var/log/project1.access.log;
error_log /var/log/project1.error.log;
location / {
uwsgi_pass unix:/tmp/project1.sock;
include /etc/nginx/uwsgi_params;
}
location /static/ {
alias /home/hosting/project1/static/;
}
location /media/ {
alias /home/hosting/project1/media/;
}
}
You need to give permissions to the /home/hosting/.nginx directory to www-data user (or to the user nginx is running under).
chown -R www-data:www-data /home/hosting/.nginx/
We start nginx
service nginx start
Install virtualenvwrapper
virtualenvwrapper - a handy wrapper around virtualenv.
We put pip if it is not worth it:
sudo apt-get install python-pip
We put virtualenvwrapper:
pip install virtualenvwrapper
In ~ / .bashrc add:
export WORKON_HOME=/home/hosting/.virtualenvs/
source /usr/local/bin/virtualenvwrapper.sh
Relocate to the console so that .bashrc boots. We should now have the mkvirtualenv command available in the console.
We place the project files in the directories:
- / home / hosting / project1 /
- / home / hosting / project2 /
For each project, create a virtual environment. Let's say project1 will run on python 2.7 and project2 on 3.3.
mkvirtualenv project1 -p /opt/python2.7/bin/python
deactivate
mkvirtualenv project2 -p /opt/python3.3/bin/python3
deactivate
For each project, we put dependencies in a virtual environment. (In my case, the dependencies are written in the requirements.txt file in the root of each project)
workon project1
cd /home/hosting/project1
pip install -r requirements.txt
workon project2
cd /home/hosting/project2
pip install -r requirements.txt
Configuring uwsgi.
We will configure it in the emperor mode (--emperor), as This mode is specifically designed for multi-hosting.
In emperor mode, uwsgi will automatically load the configs from the specified directory, which is convenient, that is, we run uwsgi once, and then he creates the processes for all applications in the configs.
By default, uwsgi executes the project code with the python that is in the current environment, so we will need to run uwsgi from virtualenv.
Since we have several different versions of python, the uwsgi launched, for example, from under python 2.7, will not be able to serve the django application with the environment from python 3.3.
So we will create an emperor for each version of python, and we will group application configs according to the interpreter version.
We create virtual environments for emperors.
mkvirtualenv python27 -p /opt/python2.7/bin/python
deactivate
mkvirtualenv python33 -p /opt/python3.3/bin/python3
deactivate
Now you need to put uwsgi in each virtualenv and configure it.
workon python27
pip install uwsgi
workon python33
pip install uwsgi
Create config directories for each uwsgi emperor.
mkdir /home/hosting/.uwsgi/python27
mkdir /home/hosting/.uwsgi/python33
Create uwsgi-configs for each project.
/home/hosting/.uwsgi/python27/project1.ini
Hidden text
[uwsgi]
protocol = wsgi
master = true
processes = 1 # по количеству ядер
socket = /tmp/project1.sock
# Докидываем в pythonpath библиотеки из виртуаленва, т.к uwsgi в динамическом режиме не умеет искать библиотеки в virtualenv
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg
pythonpath = /home/hosting/.virtualenvs/project1/lib/python27.zip
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/plat-linux2
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-tk
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-old
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/lib-dynload
pythonpath = /home/hosting/.virtualenvs/project1/lib/python2.7/site-packages
# почему-то в virtualenv не все нужные файлы есть, поэтому пришлось добавить эту строчку
pythonpath = /opt/python2.7/lib/python2.7
chdir = /home/hosting/project1
virtualenv = /home/hosting/.virtualenvs/project1
env = DJANGO_SETTINGS_MODULE=settings
module = django.core.handlers.wsgi:WSGIHandler()
no-site = true
vhost = true
chmod-socket = 666
The second project config is similar. The file name must end with .ini, otherwise uwsgi will not pick up this config.
Now you need to register uwsgi as a service in the system. I used upstart, it is in the ubunt from the box.
Let's create two config files:
/etc/init/uwsgi27.conf
description "uWSGI Emperor (python 2.7)"
start on runlevel [2345]
stop on runlevel [06]
exec /home/hosting/.virtualenvs/python27/bin/uwsgi --master --emperor /home/hosting/.uwsgi/python27 --logto /var/log/uwsgi27.emperor.log
/etc/init/uwsgi33.conf
description "uWSGI Emperor (python 3.3)"
start on runlevel [2345]
stop on runlevel [06]
exec /home/hosting/.virtualenvs/python33/bin/uwsgi --master --emperor /home/hosting/.uwsgi/python33/ --logto /var/log/uwsgi33.emperor.log
Users and Security
From root, we will only run “imperial” processes, and the projects themselves will be under their own users.
Create a user for each of the projects.
adduser --no-create-home --disabled-login --disabled-password www-project1
adduser --no-create-home --disabled-login --disabled-password www-project2
Add uid gid parameters to each of uwsgi ini-configs
uid = www-project1 # пользователь
gid = www-project1 # группа
Set the correct access rights
Hidden text
chown -R www-data:www-data /home/hosting/.nginx
chmod -R 770 /home/hosting/.nginx
chown -R root:root /home/hosting/.uwsgi
chmod -R 770 /home/hosting/.uwsgi
chown -R root:root /home/hosting/.virtualenvs/python27 /home/hosting/.virtualenvs/python33
chmod -R 775 /home/hosting/.virtualenvs
chown -R www-project1:www-project1 /home/hosting/project1 /home/hosting/.virtualenvs/project1
chown -R www-project2:www-project2 /home/hosting/project2 /home/hosting/.virtualenvs/project2
Launch uwsgi
service uwsgi27 start
service uwsgi33 start
We check - everything should work.
If something does not work, look at the nginx logs specified in the project config and the logs of the uwsgi emperor.
A sign that uwsgi has successfully deployed the application is the presence of a line
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x135e280 pid: 21737 (default app)
in the uwsgi log. To add a new application, you need to create a config in .nginx and .uwsgi and restart nginx. uwsgi itself will pick up a new config.
References
projects.unbit.it/uwsgi/wiki/MultiPython
projects.unbit.it/uwsgi/wiki/DynamicVirtualenv
auphonic.com/blog/2011/06/18/django-deployment-nginx-uwsgi-virtualenv-and-fabric
eshlox.net / en / 2012/09/11 / nginx-uwsgi-virtualenv-and-django-ubuntu-1204
uwsgi-docs.readthedocs.org/en/latest/Emperor.html
PS
The method described in the article is not very beautiful in terms of architecture; Initially, I was hoping to get by with one uwsgi emperor and resolve the interpreter version with the plugin parameter in the application config. But I could not build a uwsgi-plugin for python, so I had to do it differently.