Monitoring Django Project Statistics Using Pinba on Debian GNU / Linux

In this topic, we will talk about step-by-step setup and configuration of the project and the server using the pinba-engine + pinboard + django-pinba bundle to collect and visualize project statistics on Django.

All projects are free and their source code is available on GitHub under the GPL or MIT licenses.

Why pinba?

The team has projects that were written in php. Project attendance is quite high (over 200k). There are so many bottlenecks. Often, after any updates and innovations, we received huge brakes, which we did not immediately learn about. What was needed was a solution that turned to the customer and the programmer. The uploader really does not like zabbix and munin. There was a solution. It was a pinboard and pinba . The customer liked it. He asked to screw pinba to all projects that were spinning on django.

Go

The setup consists of several parts:
1. Installing pinba-engine-mysql as a server for monitoring (on Debian GNU / Linux)
2. Installing pinboard to view the current status of the project
3. Installing the django-pinba battery to send statistics

1. Installing pinba-engine-mysql as a monitoring server (on Debian GNU / Linux)

1.1. We connect an additional repository, from the author of the pinba project:

# echo "deb http://php53.dotdeb.org stable all" >> /etc/apt/sources.list && echo "deb-src http://php53.dotdeb.org stable all" >> /etc/apt/sources.list
# gpg --keyserver keys.gnupg.net --recv-key 89DF5277 && gpg -a --export 89DF5277 | sudo apt-key add -
# apt-get update && apt-get upgrade -y

1.2. Install MySQL and Pinba-Engine:

# apt-get install dialog mysql-server -y
# apt-get install pinba-engine-mysql-5.5 -y

1.3. Check if pinba-engine-mysql is listening on the port we need:

# netstat -uln|grep :30002

2. Installing pinboard, to view the current status of the project

2.1. Install all the necessary dependencies:

# apt-get install git-core nginx php5-fpm curl php-apc php5-cli php5-mysqlnd vim -y

2.2. We clone the project and pull up the necessary packages:

# cd /var/www/ && git clone git://github.com/intaro/pinboard.git && cd ./pinboard
# git checkout v1.0
# curl -sS https://getcomposer.org/installer | php && php composer.phar install
# cp config/parameters.yml.dist config/parameters.yml

2.3. Configure database connection settings:

# vim config/parameters.yml

2.4. We migrate and register tasks in the crown:

# ./console migrations:migrate
# ./console register-crontab

2.5. Configure the nginx virtual host:

# touch /etc/nginx/sites-available/pinba.conf && ln -s /etc/nginx/sites-available/pinba.conf /etc/nginx/sites-enabled/ && vim /etc/nginx/sites-enabled/pinba.conf

2.6. Insert the virtual host configuration and specify the necessary server_name:

server {
    listen 80;
    server_name pinboard.example.com;
    root /var/www/pinboard/web;
    location = / {
        try_files @site @site;
    }
    location / {
        try_files $uri $uri/ @site;
    }
    location ~ \.php$ {
       return 404;
    }
    location @site {
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        include fastcgi_params;
        fastcgi_param  SCRIPT_FILENAME $document_root/index.php;
        fastcgi_param HTTPS $https if_not_empty;
    }
    location ~ /\.(ht|svn|git) {
        deny  all;
    }
}

2.7. Rebooting the nginx daemon:

# /etc/init.d/nginx restart

2.8. Add the time zone configuration in php.ini:

# sed -i 's/;date.timezone =/date.timezone = Europe\/Moscow/g' /etc/php5/cli/php.ini /etc/php5/fpm/php.ini

2.9. Check the operation of the script specified in the crown:

# /var/www/pinboard/console aggregate

If the script did the job without errors, then everything works correctly. This completes the pinboard setup.

3. Installing a django-pinba battery to send statistics

3.1. Install the battery:

$ pip install django-pinba

3.2. Open the settings.py settings file and add django-pinba to MIDDLEWARE_CLASSES:

MIDDLEWARE_CLASSES = (
    'pinba.middleware.PinbaMiddleware',
    ...
)

3.3. Specify the settings of our server with pinba-engine:

PINBA_SERVER = '192.168.55.11'
PINBA_PORT = 30002
PINBA_ENABLED = True

It is advisable not to keep statistics on the same server as the project.

Conclusion

Now you can open our site in a browser window, wait for a while, which was indicated in crontab for pinboard, and enjoy the beautiful graphics. The load for the test can be created using the ab or wget utility.

The interface is as follows: