Deploy Mercurial Repositories via FastCGI Using Nginx on FreeBSD

    I have succumbed to the fashion and exciting perspectives of DVCS recently. This pushed me off the beaten track of Subversion + Trac and forced me to look for new schemes for storing source codes in different companies. And provide them convenient access to developers, customers and other interested individuals.

    It so happened that I specialize in FreeBSD and am not so good at Linux. And I prefer where you can use Nginx instead of Apache httpd. Therefore, I decided to create for myself a unified architecture that will allow you to store an unlimited number of repositories and to differentiate access for them to different groups of people on this platform.

    Of course, Bitbucket is our everything. But any developer has closed projects that I would not want to upload to public. You can, of course, pay $ 50 per month for the opportunity to host 25 projects on bitbucket. I personally think that it is better to spend this money on a dedicated server and raise as many projects as you like. It will not be so convenient, but its own and with the possibility of tuning, backup and other goodies.


    Formulation of the problem


    You must have a repository of Mercurial repositories with the ability to access them via the web. Access rights must be specified read / write for specific users.

    Specific points

    • Repositories should be stored in the directory / usr / home / www / repos /
    • Mercurial settings are stored in / usr / home / www / repos / cgi /
    • Hosting takes place on a local server. Therefore, access is only via http. Outside, access via https will be served by another server, also based on nginx, and transferred to the local port on port 80.
    • Let's say the server is called dev.example.com. The list of repositories should be available at
      http://dev.example.com/hg/

    Installing prerequisites


    Details on how to deploy multiple repositories can be found on the official page . To do this, use the cgi hgwebdir.cgi script. We, of course, will use its FastCGI version of hgwebdir.fcgi.

    Mercurial does not include a self-contained FastCGI server that can be run on the desired port. It is assumed that hgwebdir.fcgi will be started by apache httpd or lighttpd. To solve this problem, we will use the fastcgi wrapper spawn-fcgi.

    So, first of all, install the following ports
    • www / spawn-fcgi
    • devel / mercurial
    • www / nginx
    • www / py-flup

    Customize hgwebdir


    Copy the default script to our settings directory. Edit our hgwebdir.fcgi. It looks like this for me. Next, in the configuration file /usr/home/www/cgi/hgweb.config, we specify the paths to the collections and the address setting. The baseurl parameter says that all links that hgwebdir will generate will contain / hg at the beginning of the path ( remember, we have the address where the repositories dev.example.com/hg/ should be available)

    sudo -u www mkdir /usr/home/www/cgi/
    cp /usr/local/share/mercurial/www/hgwebdir.fcgi /usr/home/www/cgi/




    #!/usr/bin/env python
    from mercurial import demandimport; demandimport.enable()
    from mercurial.hgweb.hgwebdir_mod import hgwebdir
    from flup.server.fcgi import WSGIServer
    WSGIServer(hgwebdir('/usr/home/www/cgi/hgweb.config')).run()




    [collections]
    /usr/home/www/repos = /usr/home/www/repos
    [web]
    baseurl = /hg




    Configure spawn-fcgi


    In the FreeBSD configuration file /etc/rc.conf, add the following lines. You can specify the script itself as spawn_fcgi_app, and not the path to the python interpreter. But then the service will not stop correctly, because system scripts will look for a process called hgwebdir.fcgi to stop, but in reality the process is called python, because hgwebdir.fcgi is a program that runs using the interpreter. Now you can run the daemon

    # Mercurial
    spawn_fcgi_enable="YES"
    spawn_fcgi_app="/usr/local/bin/python"
    spawn_fcgi_app_args="/usr/home/www/cgi/hgwebdir.fcgi"
    spawn_fcgi_pidfile="/var/run/hg.pid"
    spawn_fcgi_bindsocket="/var/run/hg.socket"






    sudo /usr/local/etc/rc.d/spawn-fcgi start


    Configure nginx



    In the configuration file /usr/local/etc/nginx/nginx.conf, create a new server section with the following contents

        server {
            listen 80;
            server_name dev.example.com;
            set_real_ip_from 172.16.224.1;
            real_ip_header X-Forwarded-For;
            location / {
                root / usr / local / www / nginx;
                index index.html index.htm;
            }
            location / hg {
                if ($ uri ~ /hg(/.*)$) {
                    set $ path $ 1;
                }
                client_max_body_size 100M;
                fastcgi_pass unix: /var/run/hg.socket;
                include fastcgi_params;
                fastcgi_param PATH_INFO $ path;
                fastcgi_param REMOTE_USER $ remote_user;
                auth_basic "Mercurial repositories";
                auth_basic_user_file hg_htpasswd;
            }
        }
    


    At this point, be extremely careful. Nginx by default does not set the variables that are necessary for correct operation with mercurial. Therefore, I will try to explain the meaning of each line. They were given to me by studying the sources of mercurial :)

    set_real_ip_from , real_ip_header - as I mentioned above, external https access is served by a separate nginx, standing on a server that looks to the world. So that on our local server the real addresses of the clients get to the logs, and not the address of this external server that proxies requests, and these directives were added.

    if block and PATH_INFO variable- to determine the repository that is being accessed, hgwebdir uses the PATH_INFO variable. We need to cut off the beginning of the path / hg, because otherwise, hgwebdir will think that you are trying to access the repository under the name "hg". Here you need to be careful and use nginx's $ uri variable rather than $ request_uri to calculate PATH_INFO, as some manuals on the Internet mention. Because $ request_uri also contains $ args in addition to the path. And if you pass it to PATH_INFO, then you will receive 404 Not Found when trying to work with the repository.

    client_max_body_size- by default, the maximum length of a request to nginx is 1M. Naturally, if you try to commit changes by more than 1M, or make the initial push of a large repository, the request will fail. Therefore, this size is increased to 100M. I think in most cases this is enough.

    include fastcgi_params - we connect other standard variables of type QUERY_STRING, etc.

    REMOTE_USER - in this variable hgwebdir expects to get the username.

    Well, the last two lines - setting http basic auth authorization.

    That's all, now you can fix other parameters not related to hg in nginx.conf (for example, where to put the logs) and start the web server.

    Add nginx_enable="YES"to /etc/rc.conf and dosudo /usr/local/etc/rc.d/nginx start

    Repository Creation Example



    Now we are ready to create the first repository. Everything is done from the user www, because it is under him that nginx and hgwebdir work, and he needs write permissions to this directory. Let's make a test repository. We give read rights to user user1 and read / write permissions to user2. We edit test / .hg / hgrc so that it looks like this. Done, you can check access.

    # Идем в директорию с репозиториями
    cd /usr/home/www/repos
    # Создаем новый репозиторий
    sudo -u www hg init test
    # Добавляем пользователей в nginx
    # -c - создать файл
    # htpasswd - из пакета www/apache22
    sudo htpasswd -b -c /usr/local/etc/nginx/hg_htpasswd user1 pass1
    sudo htpasswd -b /usr/local/etc/nginx/hg_htpasswd user2 pass2




    [web]
    # Нам не нужен ssl, поскольку о нем заботится nginx
    push_ssl = False
    allow_push = user2
    allow_read = user1,user2




    Instead of a conclusion


    In addition to general migration from subversion to mercurial, I now look intently at redmine instead of trac. I liked the idea that users can register and create projects themselves. Unfortunately, I did not find a ready-made solution on how to automatically create an hg repository for the created project. Moreover, with the cloning of access rights as described in this article. If someone knows - I will be grateful for the reference. In any case, I'm going to solve this problem in the near future either by writing a plugin for redmine, or in a more different / correct way.

    If you liked this article and would like to see a sequel, leave feedback. I will then share my new knowledge of integrating our new environment with redmine.

    Also popular now: