Python which modules are available

How to get a list of python modules (with version for each module) and create a requirements.txt file ?

Examples of commands I used to get a list of python modules with corresponding versions. If you have better options to do that, your are welcome to add a comment or edit the note.

Get a list of python modules available in my current environment

To get a list of python modules available, a solution is to use freeze:

absl-py==0.9.0 alabaster==0.7.10 anaconda-client==1.6.9 anaconda-navigator==1.7.0 anaconda-project==0.8.2 appnope==0.1.0 appscript==1.0.1 asn1crypto==0.24.0 astor==0.8.1 astroid==1.6.1 astropy==2.0.3 attrs==17.4.0 Babel==2.5.3 backports.shutil-get-terminal-size==1.0.0 beautifulsoup4==4.6.0 bitarray==0.8.1 bkcharts==0.2 blaze==0.11.3 bleach==2.1.2 bokeh==0.12.13 boto==2.48.0 Bottleneck==1.2.1 cachetools==3.1.1 certifi==2018.1.18 cffi==1.11.4 chardet==3.0.4 click==6.7 cloudpickle==0.5.2 clyent==1.2.2 colorama==0.3.9 conda==4.8.3 conda-build==3.4.1 conda-package-handling==1.7.0+0.g7c4a471.dirty conda-verify==2.0.0 contextlib2==0.5.5 cryptography==2.1.4 cycler==0.10.0 Cython==0.27.3 cytoolz==0.9.0 dask==0.16.1 datashape==0.5.4 decorator==4.2.1 distributed==1.20.2 docutils==0.14 entrypoints==0.2.3 et-xmlfile==1.0.1 fastcache==1.0.2 filelock==2.0.13 Flask==0.12.2 Flask-Cors==3.0.3 geoip2==2.9.0 gevent==1.2.2 glob2==0.6 gmpy2==2.0.8 google-api-python-client==1.7.11 google-auth==1.6.3 google-auth-httplib2==0.0.3 greenlet==0.4.12 h5py==2.7.1 heapdict==1.0.0 html5lib==1.0.1 httplib2==0.14.0 idna==2.6 imageio==2.2.0 imagesize==0.7.1 ipykernel==4.8.0 ipython==6.2.1 ipython-genutils==0.2.0 ipywidgets==7.1.1 isort==4.2.15 itsdangerous==0.24 jdcal==1.3 jedi==0.11.1 Jinja2==2.10 jsonschema==2.6.0 jupyter==1.0.0 jupyter-client==5.2.2 jupyter-console==5.2.0 jupyter-core==4.4.0 jupyterlab==0.31.5 jupyterlab-launcher==0.10.2 lazy-object-proxy==1.3.1 llvmlite==0.21.0 locket==0.2.0 lxml==4.1.1 MarkupSafe==1.0 matplotlib==2.1.2 maxminddb==1.4.1 mccabe==0.6.1 mistune==0.8.3 mpmath==1.0.0 msgpack-python==0.5.1 multipledispatch==0.4.9 navigator-updater==0.1.0 nbconvert==5.3.1 nbformat==4.4.0 networkx==2.1 nltk==3.2.5 nose==1.3.7 notebook==5.4.0 numba==0.36.2 numexpr==2.6.4 numpy==1.14.0 numpydoc==0.7.0 oauth2client==4.1.3 odo==0.5.1 olefile==0.45.1 openpyxl==2.4.10 packaging==16.8 pandas==0.22.0 pandocfilters==1.4.2 parso==0.1.1 partd==0.3.8 path.py==10.5 pathlib2==2.3.0 patsy==0.5.0 pdf2image @ file:///home/conda/feedstock_root/build_artifacts/pdf2image_1588278609827/work pep8==1.7.1 pexpect==4.3.1 pickleshare==0.7.4 Pillow==5.0.0 pkginfo==1.4.1 pluggy==0.6.0 ply==3.10 prompt-toolkit==1.0.15 protobuf==3.12.4 psutil==5.4.3 ptyprocess==0.5.2 py==1.5.2 pyasn1==0.4.7 pyasn1-modules==0.2.6 pycodestyle==2.3.1 pycosat==0.6.3 pycparser==2.18 pycrypto==2.6.1 pycurl==7.43.0.1 pyflakes==1.6.0 Pygments==2.2.0 pylint==1.8.2 pyodbc==4.0.22 pyOpenSSL==17.5.0 pyparsing==2.2.0 PySocks==1.6.7 pytest==3.3.2 python-dateutil==2.6.1 pytz==2017.3 PyWavelets==0.5.2 PyYAML==3.12 pyzmq==16.0.3 QtAwesome==0.4.4 qtconsole==4.3.1 QtPy==1.3.1 requests==2.18.4 rope==0.10.7 rsa==4.0 ruamel-yaml==0.15.35 scikit-image==0.13.1 scikit-learn==0.19.1 scipy==1.0.0 seaborn==0.8.1 Send2Trash==1.4.2 simplegeneric==0.8.1 singledispatch==3.4.0.3 six==1.12.0 snowballstemmer==1.2.1 sortedcollections==0.5.3 sortedcontainers==1.5.9 Sphinx==1.6.6 sphinxcontrib-websupport==1.0.1 spyder==3.2.6 SQLAlchemy==1.2.1 statsmodels==0.8.0 sympy==1.1.1 tables==3.4.2 tblib==1.3.2 tensorflow-docs @ git+https://github.com/tensorflow/[email protected] terminado==0.8.1 testpath==0.3.1 toolz==0.9.0 tornado==4.5.3 tqdm==4.46.0 traitlets==4.3.2 typing==3.6.2 unicodecsv==0.14.1 uritemplate==3.0.0 urllib3==1.22 wcwidth==0.1.7 webencodings==0.5.1 Werkzeug==0.14.1 widgetsnbextension==3.1.0 wrapt==1.10.11 xlrd==1.1.0 XlsxWriter==1.0.2 xlwings==0.11.5 xlwt==1.2.0 zict==0.1.3 

To create a requirements.txt file from freeze:

pip freeze > requirements.txt 

Modules using in a python script

To find all modules used in a python script (for example test.py), a solution is to use ModuleFinder:

from modulefinder import ModuleFinder finder = ModuleFinder() finder.run_script('test.py') print('Loaded modules:') for name, mod in finder.modules.items(): print('%s: ' % name, end='') print(','.join(list(mod.globalnames.keys())[:3])) print('-'*50) print('Modules not imported:') print('\n'.join(finder.badmodules.keys())) 

Note that It seems not to work as expected with anaconda (I got a list of all available modules not the modules used in the script only).

Modules using in a jupyter notebook

In a jupyter notebook to get a list of python modules used, a solution is to enter in a cell the following lines:

import types def imports(): for name, val in globals().items(): if isinstance(val, types.ModuleType): yield val.__name__ list(imports()) 

['builtins', 'builtins', 'types', 'tensorflow', 'tensorflow_docs', 'tensorflow_docs', 'matplotlib.pyplot', 'matplotlib', 'matplotlib.cm', 'pandas', 'numpy', 'numpy.ma', 'seaborn', 'os', 'warnings', 'tensorflow_core.keras', 'tensorflow_core.keras.layers'] 

returns all the modules that can be imported in the notebook:

absl-py==0.9.0 alabaster==0.7.10 anaconda-client==1.6.9 anaconda-navigator==1.7.0 anaconda-project==0.8.2 appnope==0.1.0 appscript==1.0.1 asn1crypto==0.24.0 astor==0.8.1 astroid==1.6.1 astropy==2.0.3 attrs==17.4.0 Babel==2.5.3 backports.shutil-get-terminal-size==1.0.0 beautifulsoup4==4.6.0 bitarray==0.8.1 bkcharts==0.2 

References

Benjamin

Greetings, I am Ben! I completed my PhD in Atmospheric Science from the University of Lille, France. Subsequently, for 12 years I was employed at NASA as a Research Scientist focusing on Earth remote sensing. Presently, I work with NOAA concentrating on satellite-based Active Fire detection. Python, Machine Learning and Open Science are special areas of interest to me.

Skills

Источник

How to find which Python modules are being imported from a package?

A file containing Python code, definitions of statements, functions, or classes is known as a module. A module with the name «module» that we will construct is a file named module.py.

To break down complex programmes into smaller, easier-to-understand parts, we use modules. Code reuse is another benefit of modules.

In this article, we will discuss various ways to find which Python modules are being imported from a package.

Using List Comprehension

To iterate over each element in the Python list, a list comprehension is made out of brackets carrying the expression, which is then run for each element.

Python List comprehension offers a much shorter syntax for creating a new list from the elements of an existing list.

Example

The following example returns all imported local module names by default in an unsorted list using the sys library with List Comprenehsion.

Using __name__ (also known as a dunder), this code iterates through sys.modules.values() to check if an item is a locally scoped module. If so, ‘output’ is saved with the module name. For readability, this code arranges the ‘output’ variable and saves it back to itself. List formatted output of these ‘output’ is sent to the terminal.

import sys output = [module.__name__ for module in sys.modules.values() if module] output = sorted(output) print('The list of imported Python modules are :',output) 
 

Output

Following is an output of the above code −

The list of imported Python modules are : ['__main__', '_bootlocale', '_codecs', '_collections', '_functools', '_heapq', '_imp', '_locale', '_operator', '_signal', '_sitebuiltins', '_stat', '_sysconfigdata_m_linux_x86_64-linux-gnu', '_thread', '_warnings', '_weakref', '_weakrefset', 'abc', 'builtins', 'codecs', 'collections', 'collections.abc', 'collections.abc', 'contextlib', 'encodings', 'encodings.aliases', 'encodings.latin_1', 'encodings.utf_8', 'errno', 'functools', 'genericpath', 'heapq', 'importlib', 'importlib._bootstrap', 'importlib._bootstrap', 'importlib._bootstrap_external', 'importlib._bootstrap_external', 'importlib.abc', 'importlib.machinery', 'importlib.util', 'io', 'io', 'itertools', 'keyword', 'marshal', 'mpl_toolkits', 'operator', 'os', 'posix', 'posixpath', 'posixpath', 'reprlib', 'site', 'stat', 'sys', 'sysconfig', 'types', 'warnings', 'weakref', 'zipimport']

Using pip freeze command

This function shows a list of all names and versions of imported global modules, by default arranged alphabetically.

Example

Enter the below command by opening the terminal window in an IDE. In order to execute, press the Enter key −

Output

The terminal receives the output −

aspose-cells==22.7.0 click==8.1.3 cloudpickle==2.1.0 colorama==0.4.5 dask==2022.7.0 et-xmlfile==1.1.0 fsspec==2022.5.0 genno==1.11.0 ixmp==3.5.0 JPype1==1.4.0 llvmlite==0.38.1 locket==1.0.0 message-ix==3.5.0 modcall==0.1.0 mysql-connector-python==8.0.29 namespace==0.1.4 native==0.0.0.0a0.dev20210615 numba==0.55.2 numpy==1.22.4 openpyxl==3.0.10 packaging==21.3 pandas==1.4.3 partd==1.2.0 Pint==0.19.2 protobuf==4.21.2 psycopg2==2.9.3 pycparser==2.21 pyparsing==3.0.9 python-dateutil==2.8.2 python-dotenv==0.20.0 python-magic==0.4.27 pytz==2022.1 PyYAML==6.0 scipy==1.9.1 six==1.16.0 sparse==0.13.0 toolz==0.12.0 walk==0.3.5 workbook==1.1 xarray==2022.3.0 xlrd==2.0.1 xlutils==2.0.0 xlwt==1.3.0

Using dir() method

The dir() function returns all of the given object’s properties and methods, but not their related values. This function will even return built-in attributes that are the default for all objects.

Example

The dir() method is used in the following example to return a sorted list of all local module names −

module = dir() print('The list of imported Python modules are :',module) 
 

Output

The output shown below demonstrate that this script only shows names relevant to our local scope −

The list of imported Python modules are : ['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']

Using inspect.getmember() and a Lambda

A number of helpful functions are offered by the inspect module to help in gathering data on active objects such modules, classes, methods, functions, tracebacks, frame objects, and code objects. It can assist you in examining a class’s contents, retrieving a method’s source code, extracting and formatting a function’s argument list, or gathering all the data required to display a detailed traceback, among other things.

An anonymous, short function is known as a lambda. Although a lambda function can have only one expression, it can have any number of arguments.

Example

The following example returns the imported local modules in a sorted format using inspect.getmember() and a Lambda.

The names of the imported local modules and where they are located on the system are returned by this code as an iterable object. This is iterated through and printed as one line using a for loop.

import inspect import os modules = inspect.getmembers(os) results = filter(lambda m: inspect.ismodule(m[1]), modules) for o in results: print('The list of imported Python modules are :',o) 
 

Output

Following is an output of the above code −

The list of imported Python modules are : ('abc', ) The list of imported Python modules are : ('errno', ) The list of imported Python modules are : ('path', ) The list of imported Python modules are : ('st', ) The list of imported Python modules are : ('sys', )

Using sys module

The sys.modules dict can be used to discover all the Python modules from a specific package that are being utilised by an application. A dictionary that links module names to modules is called sys.modules. To see imported modules, you can look at its keys.

Example

Following is an example to find the imported modules from a package using sys module

from datetime import datetime import sys print (sys.modules.keys()) 
 

Output

Following is an output of the above code −

dict_keys(['builtins', 'sys', '_frozen_importlib', '_imp', '_warnings', '_thread', '_weakref', '_frozen_importlib_external', '_io', 'marshal', 'posix', 'zipimport', 'encodings', 'codecs', '_codecs', 'encodings.aliases', 'encodings.utf_8', '_signal', '__main__', 'encodings.latin_1', 'io', 'abc', '_weakrefset', '_bootlocale', '_locale', 'site', 'os', 'errno', 'stat', '_stat', 'posixpath', 'genericpath', 'os.path', '_collections_abc', '_sitebuiltins', 'sysconfig', '_sysconfigdata_m_linux_x86_64-linux-gnu', 'types', 'functools', '_functools', 'collections', 'operator', '_operator', 'keyword', 'heapq', '_heapq', 'itertools', 'reprlib', '_collections', 'weakref', 'collections.abc', 'importlib', 'importlib._bootstrap', 'importlib._bootstrap_external', 'warnings', 'importlib.util', 'importlib.abc', 'importlib.machinery', 'contextlib', 'mpl_toolkits', 'datetime', 'time', 'math', '_datetime'])

Источник