Python packages and their use

Good day to all!

The topic of organizing work with external packages using submodules or trees in Git has already been raised on Habré . This seemed like a good solution, but in fact resulted in inconvenience and confusion. Then I decided to port everything to python packages. What I will share now with the example of a Django application.

Instead of the foreword

When you do many different projects, your modules, extensions, utilities accumulate. Between them, dependencies are established. And the question arises in the convenient organization of infrastructure for the development and deployment of projects. To make it more clear, I’ll tell you about Django projects.

Create a Django application and extract it into a python package. The application will display news. In the administrative part of the site, we will need our own experience in usability. For this, we specify the app-admintools package in the dependencies .

Training

Create the app-news repository in Gitosis:

gitosis-admin / gitosis.conf

[group apps]
writable = app-news
members = @developers @hosts

send the changes:

git pull

initiate:

git clone git@git.home.com:app-news

The structure of our package

+-app-news/
  +-home/
    +-news/
      +-locale/
      +-static/
      +-templates/
      +-__init__.py
      +-models.py
      +-urls.py
      +-views.py
    +-__init__.py
  +-MANIFEST.in
  +-setup.py

setup.py

from setuptools import setup, find_packages
setup(
    name='home.news',
    version='0.1',
    description='News for Django',
    author='Elias',
    namespace_packages=['home'],                                                                      # line 8
    packages=find_packages(),
    platforms='any',
    zip_safe=False,
    include_package_data=True,
    dependency_links=['git+ssh://git@git.home.com/app-admintools@v0.1#egg=admintools-0.1'],           # line 13
    install_requires=['admintools==0.1'],                                                             # line 14
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Environment :: Web Environment',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: BSD License',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Topic :: Internet :: WWW/HTTP :: Dynamic Content',
        'Framework :: Django',
    ],
)

Here, we declared the use of the home namespace (p. 8) and specified dependencies for the home.news package in our repository (p. 13.14).

MANIFEST.in

recursive-include home/*/locale *
recursive-include home/*/static *
recursive-include home/*/templates *

The manifest defines the additional data that the package needs.

home / init .py

import pkg_resources
pkg_resources.declare_namespace(__name__)

Register the namespace.

Create a remote branch v0.1 from master:

git push origin master:v0.1

All! The package is ready to install:

pip install git+ssh://git@git.home.com/app-news@v0.1

or in edit mode:

pip install -e ./app-news

or through setuptools:

python setup.py develop

but in this case, you can’t download dependencies in the form of links to repositories, since setuptools does not support this format

Package Connection

I described one method above, in the form of setup.py dependencies.

It is also convenient to store dependencies in a separate requirements.txt file (for example, for a site):

git+ssh://git@git.home.com/app-news@v0.1

and put through pip:

pip install -r requirements.txt

Conclusion

The idea of ​​shifting dependency management to Git was inconvenient (used submodules). First of all, this manifested itself when it was necessary to write code and debug it in two projects at once, with common components that needed to be synchronized. Usually through a commit. Another problem occurred when at the same time two developers update the package and a link to it. As a result, non-trivial migration. And so on ... But now you can put the package in edit mode and edit the code without synchronization. At the same time, all the charms of accessing packages by ssh keys remained.
I hope the article will help you simplify the development and become a little happier :)

Thank you, good luck!