Using the local package directory in Python now

In Python 3.8, it is proposed to add an alternative to virtual environments - a local directory with PEP 582 packages Python local packages directory .

This PEP suggests adding a mechanism for automatic directory detection to Python __pypackages__and using it when importing as a source of installed packages. The directory __pypackages__will have a higher priority during import than global or user directories with packages. This will prevent the creation, activation or deactivation of virtual environments.

This is what the package structure will look like in Python 3.8 using __pypackages__:

foo
    __pypackages__
        3.8
            lib
                bottle
    myscript.py

In this article, I’ll show you how to use the local directory with packages without waiting for Python 3.8 to exit.

This article describes a basic example tested in Linux, Python 3.5. For other platforms, you may need to make changes.

Installing packages in a local directory

Installation is almost the same as installing packages using pip, except for an additional option --target. In it we indicate the full or relative path to the directory with local packages.

pip3 install --target="$PWD/__pypackages__/3.5/lib/" bar

$ PWD is a variable with the current working directory.

The following directory tree will be created:

foo
    __pypackages__
        3.5
            lib
                bar
    myscript.py

The Python version and subdirectories must be specified manually.

There may also be problems if you need to install packages with binary code and for different architectures. I didn’t have such packages, but as a solution, you can add more architecture to the directory structure.

pip3 install --ignore-installed --install-option="--prefix=$PWD/__pypackages__" --install-option="--no-compile" bar

foo
    __pypackages__
        lib
            python3.5
                site-packages
                    bar
    myscript.py

Using the local directory with packages

After installing the packages, it remains to tell the interpreter where to look for dependencies.

To do this, you need to list the sys.pathpath to the local directory with packages. It is enough to add the path to the main (first loaded) module, adding to the rest is not necessary. After that, you can import locally installed packages.

import os
import sys
_PATH = '/__pypackages__/3.5/lib/'
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + _PATH)
import bar

The only condition is that the main module must be at the same level of nesting as the directory __pypackages__.

Another way to tell Python where to look for packages is to set an environment variable before running the script.

PYTHONPATH="$PWD/__pypackages__/3.5/lib/:$PYTHONPATH" python3 ./myscript.py

In such a simple way, you can achieve functionality similar to PEP 582 right now.