Using the local package directory in Python now
In Python 3.8, it is proposed to add an alternative to virtual environments - a local directory with PEP 582 packages Python local packages directory .
This PEP suggests adding a mechanism for automatic directory detection to Python __pypackages__
and using it when importing as a source of installed packages. The directory __pypackages__
will have a higher priority during import than global or user directories with packages. This will prevent the creation, activation or deactivation of virtual environments.
This is what the package structure will look like in Python 3.8 using __pypackages__
:
foo
__pypackages__
3.8
lib
bottle
myscript.py
In this article, I’ll show you how to use the local directory with packages without waiting for Python 3.8 to exit.
This article describes a basic example tested in Linux, Python 3.5. For other platforms, you may need to make changes.
Installing packages in a local directory
Installation is almost the same as installing packages using pip, except for an additional option --target
. In it we indicate the full or relative path to the directory with local packages.
pip3 install --target="$PWD/__pypackages__/3.5/lib/" bar
$ PWD is a variable with the current working directory.
The following directory tree will be created:
foo
__pypackages__
3.5
lib
bar
myscript.py
The Python version and subdirectories must be specified manually.
There may also be problems if you need to install packages with binary code and for different architectures. I didn’t have such packages, but as a solution, you can add more architecture to the directory structure.
There is another way to install packages in a specific directory:
pip3 install --ignore-installed --install-option="--prefix=$PWD/__pypackages__" --install-option="--no-compile" bar
But you must specify the full path to the installation location and the directory tree will be different from that proposed in PEP 582:
foo
__pypackages__
lib
python3.5
site-packages
bar
myscript.py
Using the local directory with packages
After installing the packages, it remains to tell the interpreter where to look for dependencies.
To do this, you need to list the sys.path
path to the local directory with packages. It is enough to add the path to the main (first loaded) module, adding to the rest is not necessary. After that, you can import locally installed packages.
import os
import sys
_PATH = '/__pypackages__/3.5/lib/'
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + _PATH)
import bar
The only condition is that the main module must be at the same level of nesting as the directory __pypackages__
.
Another way to tell Python where to look for packages is to set an environment variable before running the script.
PYTHONPATH="$PWD/__pypackages__/3.5/lib/:$PYTHONPATH" python3 ./myscript.py
In such a simple way, you can achieve functionality similar to PEP 582 right now.