modulefinder — Find modules used by a script¶
This module provides a ModuleFinder class that can be used to determine the set of modules imported by a script. modulefinder.py can also be run as a script, giving the filename of a Python script as its argument, after which a report of the imported modules will be printed.
modulefinder. AddPackagePath ( pkg_name , path ) ¶
Record that the package named pkg_name can be found in the specified path.
modulefinder. ReplacePackage ( oldname , newname ) ¶
Allows specifying that the module named oldname is in fact the package named newname.
class modulefinder. ModuleFinder ( path = None , debug = 0 , excludes = [] , replace_paths = [] ) ¶
This class provides run_script() and report() methods to determine the set of modules imported by a script. path can be a list of directories to search for modules; if not specified, sys.path is used. debug sets the debugging level; higher values make the class print debugging messages about what it’s doing. excludes is a list of module names to exclude from the analysis. replace_paths is a list of (oldpath, newpath) tuples that will be replaced in module paths.
Print a report to standard output that lists the modules imported by the script and their paths, as well as modules that are missing or seem to be missing.
Analyze the contents of the pathname file, which must contain Python code.
A dictionary mapping module names to modules. See Example usage of ModuleFinder .
Example usage of ModuleFinder ¶
The script that is going to get analyzed later on (bacon.py):
import re, itertools try: import baconhameggs except ImportError: pass try: import guido.python.ham except ImportError: pass
The script that will output the report of bacon.py:
from modulefinder import ModuleFinder finder = ModuleFinder() finder.run_script('bacon.py') print('Loaded modules:') for name, mod in finder.modules.items(): print('%s: ' % name, end='') print(','.join(list(mod.globalnames.keys())[:3])) print('-'*50) print('Modules not imported:') print('\n'.join(finder.badmodules.keys()))
Sample output (may vary depending on the architecture):
Loaded modules: _types: copyreg: _inverted_registry,_slotnames,__all__ re._compiler: isstring,_sre,_optimize_unicode _sre: re._constants: REPEAT_ONE,makedict,AT_END_LINE sys: re: __module__,finditer,_expand itertools: __main__: re,itertools,baconhameggs re._parser: _PATTERNENDERS,SRE_FLAG_UNICODE array: types: __module__,IntType,TypeType --------------------------------------------------- Modules not imported: guido.python.ham baconhameggs
Where does Python look for modules?¶
Let’s say we have written a Python module and saved it as a_module.py , in a directory called code .
We also have a script called a_script.py in a directory called scripts .
We want to be able to import the code in a_module.py to use in a_script.py . So, we want to be able to put his line in a_script.py :
The module and script might look like this:
def func(): print("Running useful function") import a_module a_module.func()
At the moment, a_script.py will fail with:
$ python3 scripts/a_script.py Traceback (most recent call last): File "scripts/a_script.py", line 1, in import a_module ModuleNotFoundError: No module named 'a_module'
When Python hits the line import a_module , it tries to find a package or a module called a_module . A package is a directory containing modules, but we will only consider modules for now. A module is a file with a matching extension, such as .py . So, Python is looking for a file a_module.py , and not finding it.
You will see the same effect at the interactive Python console, or in IPython:
>>> import a_module Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'a_module'
Python looks for modules in “sys.path”¶
Python has a simple algorithm for finding a module with a given name, such as a_module . It looks for a file called a_module.py in the directories listed in the variable sys.path .
>>> import sys >>> type(sys.path) >>> for path in sys.path: . print(path) . /Users/brettmz-admin/dev_trees/psych-214-fall-2016/sphinxext /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python37.zip /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7 /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload /Users/brettmz-admin/Library/Python/3.7/lib/python/site-packages /Users/brettmz-admin/dev_trees/grin /Users/brettmz-admin/dev_trees/rmdex /usr/local/lib/python3.7/site-packages
The a_module.py file is in the code directory, and this directory is not in the sys.path list.
Because sys.path is just a Python list, like any other, we can make the import work by appending the code directory to the list.
>>> import sys >>> sys.path.append('code') >>> # Now the import will work >>> import a_module
There are various ways of making sure a directory is always on the Python sys.path list when you run Python, including:
- put the directory into the contents of the PYTHONPATH environment variable – Using PYTHONPATH
- make the module part of an installable package, and install it – see: making a Python package.
As a crude hack, you can also put your code directory on the Python sys.path at the top of the files that need it:
import sys sys.path.append('code') import a_module a_module.func() $ python3 scripts/a_script_with_hack.py Running useful function The simple append above will only work when running the script from a directory containing the code subdirectory. For example:
$ mkdir another_dir $ cd another_dir $ python3 ../scripts/a_script_with_hack.py Traceback (most recent call last): File "../scripts/a_script_with_hack.py", line 4, in import a_module ModuleNotFoundError: No module named 'a_module'
This is because the directory code that we specified is a relative path, and therefore Python looks for the code directory in the current working directory.
To make the hack work when running the code from any directory, you could use some path manipulation on the The “__file__” variable :
from os.path import dirname, abspath, join import sys # Find code directory relative to our directory THIS_DIR = dirname(__file__) CODE_DIR = abspath(join(THIS_DIR, '..', 'code')) sys.path.append(CODE_DIR) import a_module a_module.func()
Now the module import does work from another_dir :
$ python3 ../scripts/a_script_with_better_hack.py Running useful function