Python find all imports

lrq3000 / pylistmodules.py

List recursively all imports of modules along with versions done from your Python application. Tested on Python 2.7. No dependencies except standard Python libs.

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters

#!/usr/bin/env python

# encoding: utf-8

# Copyright (C) 2001-2007 Martin Blais. All Rights Reserved

# Copyright (C) 2010 Bear http://code-bear.com/bearlog/

# Copyright (C) 2013 lrq3000

# Excerpt from SnakeFood to recursively list all imports of modules using AST parsing

# Additions to print the versions of each module if available

import os , sys

import compiler

from compiler . ast import Discard , Const

from compiler . visitor import ASTVisitor

import numbers

def pyfiles ( startPath ):

r = []

d = os . path . abspath ( startPath )

if os . path . exists ( d ) and os . path . isdir ( d ):

for root , dirs , files in os . walk ( d ):

for f in files :

n , ext = os . path . splitext ( f )

if ext == ‘.py’ :

r . append ([ root , f ])

return r

class ImportVisitor ( object ):

def __init__ ( self ):

self . modules = []

self . recent = []

def visitImport ( self , node ):

self . accept_imports ()

self . recent . extend (( x [ 0 ], None , x [ 1 ] or x [ 0 ], node . lineno , 0 )

for x in node . names )

def visitFrom ( self , node ):

self . accept_imports ()

modname = node . modname

if modname == ‘__future__’ :

return # Ignore these.

for name , as_ in node . names :

if name == ‘*’ :

# We really don’t know.

mod = ( modname , None , None , node . lineno , node . level )

else :

mod = ( modname , name , as_ or name , node . lineno , node . level )

self . recent . append ( mod )

def default ( self , node ):

pragma = None

if self . recent :

if isinstance ( node , Discard ):

children = node . getChildren ()

if len ( children ) == 1 and isinstance ( children [ 0 ], Const ):

const_node = children [ 0 ]

pragma = const_node . value

self . accept_imports ( pragma )

def accept_imports ( self , pragma = None ):

self . modules . extend (( m , r , l , n , lvl , pragma )

for ( m , r , l , n , lvl ) in self . recent )

self . recent = []

def finalize ( self ):

self . accept_imports ()

return self . modules

class ImportWalker ( ASTVisitor ):

def __init__ ( self , visitor ):

ASTVisitor . __init__ ( self )

self . _visitor = visitor

def default ( self , node , * args ):

self . _visitor . default ( node )

ASTVisitor . default ( self , node , * args )

def parse_python_source ( fn ):

contents = open ( fn , ‘rU’ ). read ()

ast = compiler . parse ( contents )

vis = ImportVisitor ()

compiler . walk ( ast , vis , ImportWalker ( vis ))

return vis . finalize ()

def find_imports_and_print ( startPath ):

for d , f in pyfiles ( startPath ):

print d , f

print parse_python_source ( os . path . join ( d , f ))

def find_imports ( startPath ):

moduleslist = <>

# Get the list of .py files and iterate over

for d , f in pyfiles ( startPath ):

# For each .py file, parse and get the list of imports

mod = parse_python_source ( os . path . join ( d , f ))

# For each imported module, store only the root module (eg: sys.os -> will store only sys)

for m in mod :

moduleslist [ m [ 0 ]. split ( «.» )[ 0 ]] = True

# Return the list of unique modules names

return moduleslist . keys ()

def import_module ( module_name ):

»’ Reliable import, courtesy of Armin Ronacher »’

try :

__import__ ( module_name )

except ImportError :

exc_type , exc_value , tb_root = sys . exc_info ()

tb = tb_root

while tb is not None :

if tb . tb_frame . f_globals . get ( ‘__name__’ ) == module_name :

raise exc_type , exc_value , tb_root

tb = tb . tb_next

return None

return sys . modules [ module_name ]

def find_versions ( moduleslist ):

»’ Find the version of each module if available (and only for modules installed, does not work with locally included files) »’

modver = <>

# For each module

for mod in moduleslist :

ver = ‘NA’

m = import_module ( mod ) # Import the module

if m is None : # The module is not installed

ver = ‘Not installed’

# Else the module is installed and imported, we try to find the version

else :

# Iterate over all keys and try to find the version

verlist = []

for k , v in m . __dict__ . iteritems ():

if ( ‘version’ in k . lower () or ‘__version__’ in k . lower () or ‘ver’ in k . lower () ) \

and isinstance ( v , ( basestring , numbers . Number )) :

verlist . append ( v )

# Store the version

if len ( verlist ) > 1 :

modver [ mod ] = verlist

elif len ( verlist ) == 1 :

modver [ mod ] = verlist [ 0 ]

else :

modver [ mod ] = ver

# Return a dict where the keys are the modules names and values are the versions

return modver

if __name__ == ‘__main__’ :

import pprint

moduleslist = find_imports ( r’path/to/your/pythonic-app/dir/’ )

modver = find_versions ( moduleslist )

print ( ‘List of modules imported:’ )

print ( moduleslist )

print ( ‘-‘ * 50 )

print ( ‘List of modules and versions:’ )

pprint . pprint ( modver )

Источник

modulefinder — Find modules used by a script¶

This module provides a ModuleFinder class that can be used to determine the set of modules imported by a script. modulefinder.py can also be run as a script, giving the filename of a Python script as its argument, after which a report of the imported modules will be printed.

modulefinder. AddPackagePath ( pkg_name , path ) ¶

Record that the package named pkg_name can be found in the specified path.

modulefinder. ReplacePackage ( oldname , newname ) ¶

Allows specifying that the module named oldname is in fact the package named newname.

class modulefinder. ModuleFinder ( path = None , debug = 0 , excludes = [] , replace_paths = [] ) ¶

This class provides run_script() and report() methods to determine the set of modules imported by a script. path can be a list of directories to search for modules; if not specified, sys.path is used. debug sets the debugging level; higher values make the class print debugging messages about what it’s doing. excludes is a list of module names to exclude from the analysis. replace_paths is a list of (oldpath, newpath) tuples that will be replaced in module paths.

Print a report to standard output that lists the modules imported by the script and their paths, as well as modules that are missing or seem to be missing.

Analyze the contents of the pathname file, which must contain Python code.

A dictionary mapping module names to modules. See Example usage of ModuleFinder .

Example usage of ModuleFinder ¶

The script that is going to get analyzed later on (bacon.py):

import re, itertools try: import baconhameggs except ImportError: pass try: import guido.python.ham except ImportError: pass 

The script that will output the report of bacon.py:

from modulefinder import ModuleFinder finder = ModuleFinder() finder.run_script('bacon.py') print('Loaded modules:') for name, mod in finder.modules.items(): print('%s: ' % name, end='') print(','.join(list(mod.globalnames.keys())[:3])) print('-'*50) print('Modules not imported:') print('\n'.join(finder.badmodules.keys())) 

Sample output (may vary depending on the architecture):

Loaded modules: _types: copyreg: _inverted_registry,_slotnames,__all__ re._compiler: isstring,_sre,_optimize_unicode _sre: re._constants: REPEAT_ONE,makedict,AT_END_LINE sys: re: __module__,finditer,_expand itertools: __main__: re,itertools,baconhameggs re._parser: _PATTERNENDERS,SRE_FLAG_UNICODE array: types: __module__,IntType,TypeType --------------------------------------------------- Modules not imported: guido.python.ham baconhameggs 

Источник

findimports 2.3.0

FindImports extracts Python module dependencies by parsing source files. It can report names that are imported but not used, and it can generate module import graphs in ASCII or graphviz formats.

A distinguishing feature of findimports used to be that it could parse doctest code inside docstrings.

Note that not all cases are handled correctly, especially if you use ‘import foo.bar.baz’.

If you need to find unused imports in your codebase, I recommend Pyflakes instead – it’s better maintained and more reliable. For import graphs consider pydeps.

Misc

Changes

2.3.0 (2022-10-27)

• Rewrote command-line parsing to use argparse. Options that select an action ( —imports / —dot / —names / —unused ) now conflict instead of all but the last one being ignored. See pull request #20.
• Add support for Python 3.11.
• Drop support for Python 3.6.

2.2.0 (2021-12-16)

• Add support for Python 3.10.
• Add —ignore-stdlib flag to ignore modules from the Python standard library.

2.1.0 (2021-05-16)

2.0.0 (2021-05-09)

• Add support for Python 3.9.
• Drop support for Python 3.5 and 2.7.
• Fix a bug where the encoding of Python files was not determined in the same way as by Python itself. See issue 15. This requires the use of tokenize.open which is not in Python 2.7.

1.5.2 (2019-10-31)

• Add support for Python 3.8.
• Fix a bug where a package/module with a name that is a prefix of another package/module might accidentally be used instead of the other one (e.g. py instead of pylab). See issue 10.

1.5.1 (2019-04-23)

1.5.0 (2019-03-18)

• Support Python 3.6 and 3.7.
• Drop support for Python 2.6 and 3.3.
• Suppress duplicate import warnings if the line in question has a comment.

1.4.1 (2016-09-28)

1.4.0 (2015-06-04)

1.3.2 (2015-04-13)

1.3.1 (2014-04-16)

1.3.0 (2013-04-10)

• Moved to Github.
• Drop Python 2.4 and 2.5 support.
• Handle unicode docstrings with doctests.

1.2.14 (2012-02-12)

1.2.13 (2011-04-18)

1.2.12 (2011-04-08)

1.2.11 (2011-03-30)

1.2.10 (2010-02-05)

1.2.9 (2009-07-07)

1.2.8 (2009-07-07)

• Is able to find modules inside zip files (e.g. eggs).
• Fixed deprecation warning on Python 2.6.

Источник