Fast and automatic launch of Django on IIS 7.x in the production environment (+ performance tests)

Django is a popular Python web application framework. Its popularity is constantly growing due to the availability of rapid development tools, an integrated administrative interface and high speed. There is a simple and reliable way to deploy and run django applications on the IIS web server using the Web Platform Installer and the Helicon Zoo package repository.

Helicon Zoo is a repository of popular web freemovers and applications for Microsoft IIS. It uses Microsoft Web Platform Installer (WebPI) technology to deploy applications. With their help, various dependencies are processed and the installation process of the necessary components occurs, such as Python, Django, various database drivers and modules. Well, the Helicon Zoo module itself, which glues the whole thing with MS IIS 7.

How to use

In web development, there are two fairly independent environments - development and production. Helicon Zoo can be used both in production and on the developer's machine, or both there and there at the same time. In any case, the sequence of actions is approximately the following:

First you need to download and install the Web Platform Installer from the Microsoft website - ( http://www.microsoft.com/web/downloads/platform.aspx ). WebPI already contains a large number of such frameworks and applications for IIS, such as PHP, ASP.NET, Wordpress, Drupal, phpBB. To connect Helicon Zoo you need to add a new feed to WebPI:

• Launch WebPI and click Option
  
  
• In the field “Display additional scenarios” insert http://www.helicontech.com/zoo/feed and click Add feed
  
  
  

Now, if you select Applications, Tools in the WebPI interface, then at the end of the list you can see new applications: Blank Django Project, Blank Rails Project, Blank Perl Project, Blank Mojolicious Project:



Blank Django Project is a simple “Hello, World!” - An application that is used to install all the dependencies necessary to run a Django application and, in fact, an empty Django project. To install, click “Add” and “Install”, a form with a list of dependencies will appear:



To accept the licenses and start the download and installation, click “I Accept”. After installing and downloading all the dependencies, the installed project can be launched by clicking on “Launch application in browser”:



Such a window means that Python, Django and everything you need to run the applications themselves have just been installed on your computer. Now, if this is a development machine, you can start development with this empty “Hello Worls” application. After you do something with this application, it can be downloaded to the production server with all the files, directories, necessarily with the web.config file, and if Blank Django Project was previously installed on the server (no matter what directory, you only need it dependencies), the application should work there. Every time you upload your changes to the server, you need to restart the Application Pool. This is a feature of Django (and Rails); for other frameworks, a restart may not be necessary. Upon restart, the data will be automatically migrated -manage.py syncdb . This allows you to use the solution on shared hosting where the user does not have access to the console.

Under the hood

The core of Helicon Zoo is the native IIS module, in fact, playing the role of a bridge between the IIS web server and frameworks based on Ruby, Python, Perl, etc. The module works using the FastCGI protocol, which has already established itself as a reliable and fast method for interacting web applications with web server. The interaction between them occurs asynchronously using the I / O Completion Port technology. As a transport, either named pipes or tcp sockets are used. Supported are IIS 7, IIS 7.5, and IIS Express.

The basic configuration of the Helicon Zoo module is located in the <heliconZooServer> section of the applicationHost.config file. This section describes all FastCGI engines with which they will work through Zoo. Here is an example of a description of the engine for running python wsgi applications (Django applications in particular), zoofcgi.py is a python worker who implements transport on named pipes:

<enginename="python.2.7.pipe"fullPath="c:\python27\python.exe"arguments="-O %SystemDrive%\Zoo\Workers\python\zoofcgi.py"transport="pipe" />

Web.config

Web applications running through Zoo are configured through the web.config file. Here is an example of such a file for configuring a Django application to work in a 32-bit pool:

<?xml version="1.0" encoding="UTF-8"?><configuration><system.webServer><heliconZoo><applicationname="django.project.x86" ><environmentVariables><addname="PYTHONPATH"value="%APPL_PHYSICAL_PATH%;%PYTHONPATH%" /><addname="DJANGO_SETTINGS_MODULE"value="settings" /><addname="DEPLOY_FILE"value="deploy.py" /><addname="DEPLOY_LOG"value="log\deploy.log" /></environmentVariables></application></heliconZoo><handlers><addname="django.project.x86"scriptProcessor="python.2.7.pipe"path="*"verb="*"modules="HeliconZoo_x86"preCondition="bitness32"resourceType="Unspecified"requireAccess="Script" /></handlers></system.webServer></configuration>

Here are the important points:

• scriptProcess = "python.2.7.pipe" - a link to the FastCGI engine defined in applicationHost.config;
• PYTHONPATH environment variable - the path by which python searches for its modules; in our case, we add the path to our django application in PYTHONPATH;
• environment variable DJANGO_SETTINGS_MODULE - python path to the configuration file settings.py of the Django application, for example, 'mysite.settings'
• the environment variable 'DEPLOY_FILE' is the path to the file inside the application that will be launched before the first request to the application is executed after the restart of the IIS application. In this file, you need to execute, for example, the commands for synchronizing the database: 'manage.py syncdb –noinput';
• environment variable 'DEPLOY_LOG' - the path inside the application to the file into which the output of the DEPLOY_FILE command will be written for debugging purposes. The directory in which this log file is located must have write permissions for the user who runs the IIS application (usually the IIS_IUSRS group).

Static content

The Django framework is not suitable for the fast and safe processing of static files (images, scripts, style files). The web server should handle the statics directly. To do this, add all the static files to a directory and turn off the request handler in the django application in it:

<?xml version="1.0" encoding="UTF-8"?><configuration><system.webServer><handlers><removename="django.project.x86" /></handlers></system.webServer></configuration>

Sample Django Project on IIS 7

Suppose our django project consists of three applications:

• site - the site itself with templates, statics, urls.py and settings.py
• blog
• store

Static files located in site / media will be accessible through the virtual directory / media / (i.e. MEDIA_URL = '/ media /'). After installing Blank Django Project in the root of the site and copying our django project, we get the following directory structure:



/ media / is a virtual directory that points to site / media; Zoo module is disabled in it as described above. In the root web.config, PYTHONPATH points to the root of the site and DJANGO_SETTINGS_MODULE is set to 'site.settings':

... 
<heliconZoo><applicationname="django.project.x86" ><environmentVariables><addname="PYTHONPATH"value="%APPL_PHYSICAL_PATH%" /><addname="DJANGO_SETTINGS_MODULE"value="site.settings" /></environmentVariables>

Performance

And finally, the most interesting thing is performance testing. Extensive testing will be later, now so far in haste, but these results can be judged. The test machine as a server is Core 2 Quad 2.4 Ghz, 8 Gb RAM, a gigabit network. To generate the load, a more powerful computer with Apache Benchmark was used. For testing Apache and Nginx, Ubuntu 11.04 Server x64 was used. IIS 7 tests ran on Windows Server 2008 R2. No virtualoks - honest iron. The most advanced uwsgi, as well as wsgi and fast_cgi for comparison, were used as transport on Nginx. Even on IIS 7 compared with PyISAPIe.

Two small scripts were written as test pages. The first displays the current time with high resolution - this is done so that there is confidence that the pages are not taken from the cache. The second does the same, but previously stores the result in the database. All this through a template to use the real infrastructure of Django, the database is MySQL. Settings are everywhere by default, as the goal is to test exactly the most typical configurations. So, the results (in queries per second):



Here everything is more or less clear and stable. Uwsgi is ahead, apparently due to closer integration with Django code - you will need to smoke it ... PyISAPIe is far behind, which is understandable.



But with the database, the result is not so stable. Why Nginx + fcgi + MySQL showed such strange 175 queries per second is not clear. The result of MySQL on Windows is also depressing, although on shared hosting the problem may not be as critical. The fact is that performance drops mainly due to some kind of internal locks in MySQL; the server itself is practically not loaded while it generates these 104 queries. It can be assumed that with an increase in the number of sites on the server, and accordingly, an increase in the number of databases, if they do not block among themselves, the total server capacity will be sufficient.

So we decided to add MS SQL Express to the tests. Its result is also understandable - everything rests on Python itself and the database driver for it, but overall it looks pretty decent. PyISAPIe with MS SQL Express did not work.

I would like to separately note the ability of IIS 7 to hold a large number of connections. So IIS 7 + Helicon Zoo easily withstood thousands of simultaneous connections (we just had nothing more to generate), while Ubuntu, with default settings, quickly passed with an increase in the number of connections. And Apache also turned out to be very gluttonous for memory. So with the increase in the number of connections Apache consumed about 3 GB of memory in 20 seconds of the test - they did not check further.

findings

The presented solution shows stable operation and decent performance. It is quite ready for use both as a development environment and in production. Especially important is the ability to use Helicon Zoo with various Windows hosting services in order to provide Django services to its customers. Hopefully, with the growing number of Django servers on Windows, its developers will pay more attention to debugging and optimizing their code for the Windows platform. And the army of existing Windows developers can be a good help for current open-source projects.