Python module install location

Installing Packages¶

This section covers the basics of how to install Python packages .

It’s important to note that the term “package” in this context is being used to describe a bundle of software to be installed (i.e. as a synonym for a distribution ). It does not to refer to the kind of package that you import in your Python source code (i.e. a container of modules). It is common in the Python community to refer to a distribution using the term “package”. Using the term “distribution” is often not preferred, because it can easily be confused with a Linux distribution, or another larger software distribution like Python itself.

Requirements for Installing Packages¶

This section describes the steps to follow before installing other Python packages.

Ensure you can run Python from the command line¶

Before you go any further, make sure you have Python and that the expected version is available from your command line. You can check this by running:

You should get some output like Python 3.6.3 . If you do not have Python, please install the latest 3.x version from python.org or refer to the Installing Python section of the Hitchhiker’s Guide to Python.

If you’re a newcomer and you get an error like this:

>>> python3 --version Traceback (most recent call last): File "", line 1, in NameError: name 'python3' is not defined 

It’s because this command and other suggested commands in this tutorial are intended to be run in a shell (also called a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating system’s shell and interacting with Python.

If you’re using an enhanced shell like IPython or the Jupyter notebook, you can run system commands like those in this tutorial by prefacing them with a ! character:

In [1]: import sys ! --version Python 3.6.3

It’s recommended to write rather than plain python in order to ensure that commands are run in the Python installation matching the currently running notebook (which may not be the same Python installation that the python command refers to).

Due to the way most Linux distributions are handling the Python 3 migration, Linux users using the system Python without creating a virtual environment first should replace the python command in this tutorial with python3 and the python -m pip command with python3 -m pip —user . Do not run any of the commands in this tutorial with sudo : if you get a permissions error, come back to the section on creating virtual environments, set one up, and then continue with the tutorial as written.

Ensure you can run pip from the command line¶

Additionally, you’ll need to make sure you have pip available. You can check this by running:

If you installed Python from source, with an installer from python.org, or via Homebrew you should already have pip. If you’re on Linux and installed using your OS package manager, you may have to install pip separately, see Installing pip/setuptools/wheel with Linux Package Managers .

If pip isn’t already installed, then first try to bootstrap it from the standard library:

python3 -m ensurepip --default-pip

py -m ensurepip --default-pip

If that still doesn’t allow you to run python -m pip :

• Securely Download get-pip.py1
• Run python get-pip.py . 2 This will install or upgrade pip. Additionally, it will install setuptools and wheel if they’re not installed already.

Warning Be cautious if you’re using a Python install that’s managed by your operating system or another package manager. get-pip.py does not coordinate with those tools, and may leave your system in an inconsistent state. You can use python get-pip.py —prefix=/usr/local/ to install in /usr/local which is designed for locally-installed software.

Ensure pip, setuptools, and wheel are up to date¶

While pip alone is sufficient to install from pre-built binary archives, up to date copies of the setuptools and wheel projects are useful to ensure you can also install from source archives:

python3 -m pip install --upgrade pip setuptools wheel

py -m pip install --upgrade pip setuptools wheel

Optionally, create a virtual environment¶

See section below for details, but here’s the basic venv 3 command to use on a typical Linux system:

python3 -m venv tutorial_env source tutorial_env/bin/activate

py -m venv tutorial_env tutorial_env\Scripts\activate

This will create a new virtual environment in the tutorial_env subdirectory, and configure the current shell to use it as the default python environment.

Creating Virtual Environments¶

Python “Virtual Environments” allow Python packages to be installed in an isolated location for a particular application, rather than being installed globally. If you are looking to safely install global command line tools, see Installing stand alone command line tools .

Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python3.6/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.

Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.

Also, what if you can’t install packages into the global site-packages directory? For instance, on a shared host.

In all these cases, virtual environments can help you. They have their own installation directories and they don’t share libraries with other virtual environments.

Currently, there are two common tools for creating Python virtual environments:

• venv is available by default in Python 3.3 and later, and installs pip and setuptools into created virtual environments in Python 3.4 and later.
• virtualenv needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and pip , setuptools and wheel are always installed into created virtual environments by default (regardless of Python version).

The basic usage is like so:

python3 -m venv source /bin/activate

Источник

How to import local modules with Python

Importing files for local development in Python can be cumbersome. In this article, I summarize some possibilities for the Python developer.

TL; DR: I recommend using python -m to run a Python file, in order to add the current working directory to sys.path and enable relative imports.

Definitions

Script: Python file meant to be run with the command line interface.

Module: Python file meant to be imported.

Package: directory containing modules/packages.

Example

Consider the following project structure:

project ├── e.py └── src ├── a │ └── b.py └── c └── d.py 

Say that in b.py , we want to use d.py :

Let’s try to run python from the project directory:

~/Documents/code/project$ python3 src/a/b.py Traceback (most recent call last): File "/home/qfortier/Documents/code/project/src/a/b.py", line 4, in import src.c.d ModuleNotFoundError: No module named 'src' 

What happened? When Python tries to import a module, it looks at the paths in sys.path (including, but not limited to, PYTHONPATH):

~/Documents/code/project$ python3 src/a/b.py ['/home/qfortier/Documents/code/project/src/a', '/usr/lib/python38.zip', '/usr/lib/python3.8', . ] 

We see that the directory containing b.py (the script we run) is added to sys.path . But d.py is not reachable from the directory a , hence the above error.

1st solution: add root to sys.path

We can add the path to the root of the project:

~/Documents/code/project$ python3 src/a/b.py ['/home/qfortier/Documents/code/project/src/a', . '.'] 

The added path ‘.’ refers to the current working directory (from which Python was run) ~/Documents/code/project. Python can indeed import c.d from this directory.

This solution works regardless of the directory used to run Python:

~/Documents/code/project$ cd ../.. ~/Documents$ python3 code/project/src/a/b.py ['/home/qfortier/Documents/code/project/src/a', . 'code/project'] 

It should also work on a different computer, with a different filesystem or OS, thanks to Pathlib.
However, modifying sys.path at the beginning of every file is tedious and hard to maintain. A better solution would be to use a context.py file modifying sys.path and imported by every other file, but this is still unsatisfying.
Another possibility is to add the root directory to PYTHONPATH (before running the script). This is done automatically by poetry , for example.

Relative import

Relative imports were introduced in PEP 328 as a way to improve maintainability and avoid very long import statements. They must use the from .module import something syntax:

~/Documents/code/project$ python3 src/a/b.py Traceback (most recent call last): File "src/a/b.py", line 4, in from ..c import d ImportError: attempted relative import with no known parent package 

This is not working! Indeed, relative imports rely on the __name__ or __package__ variable (which is __main__ and None for a script, respectively).
If we import b.py from another file, everything is fine:

~/Documents/code/project$ python3 e.py Name: src.a.b Package: src.a 

Note: to go up several directories in relative imports, use additional dots: from . c import d would go 2 directories up, for example.

2nd solution: run as a module

It is unfortunate that scripts can’t use relative imports. PEP 338 overcomes this limitation by adding the -m option. With this option, a script is run as if it was imported as a module.

~/Documents/code/project$ python3 -m src.a.b # note that we use . and not / here Name: __main__ Package: src.a 

python3 -m src.a.b acts as if a Python file containing import src.a.b was run in the current directory, with two notable differences:

• the __package__ variable is filled, enabling relative imports (see PEP 366)
• the directory from which Python was run (here: ~/Documents/code/project) is added to sys.path

Since I don’t see any drawback to this, I recommend always using python3 -m to run a script.

Run as a module on Visual Code

The default launch configuration in Visual Code runs Python files as scripts (without -m ). This GitHub issue explains how to add -m automatically:

• add the “Command Variable” extension to Visual Code
• set the following Python launch configuration in your settings.json:

3rd solution : modify PYTHONPATH

With Visual Code, you can automatically add the root directory to your project by adding a line to the launch configuration :

Alternatively, you can add a .env file in the root directory :

4rd solution (outdated): install in editable mode

pip install -e . installs a package in editable mode, which can then be imported anywhere on the computer. In practice, this is essentially a symbolic link to your package. Therefore, any modification of the package is reflected on the code importing it. It requires a setup.py at the root of your package.

Note: according to PEP 517, this is no longer recommended: Python packages should rely on a toml file (see Poetry) and not on setup.py anymore.

References

Updated: May 9, 2021

Источник