Is it possible to import a compiled python file?
I can’t seem to figure out how to import a compiled .pyc module into my code so I can use it within my main script. Is this even possible?
6 Answers 6
If there is foo.pyc, import foo will automatically use foo.pyc whether foo.py exists or not
(If foo.py is newer, it will be used)
Import wasn’t working for me in the interpreter (IPython). I tested it in a script and it worked. Can it be done in the interpreter?
In a nutshell, to import a Python compiled file (e.g. module.pyc) only, simply place it in the same directory where the source (e.g module.py) would be, and ensure that there is no corresponding source file (module.py in our example) there. Then the usual import module will work seamlessly.
If there is a source file in the same directory as the compiled file, Python will use the compiled file in the __pycache__ directory instead, or recompile from source if it’s not there.
If you remove the source file without putting a «.pyc» in the same directory, the import will fail even if the compiled file exists in the __pycache__ directory. Also note that files under __pycache__ follow a different naming convention. If you copy them across, make sure that they are renamed so that it has the same name as the source file, except that the extension must be «pyc» rather than «py».
There is a very nice flow chart in PEP 3147 linked from the documentation.
Import arbitrary python source file. (Python 3.3+)
How can I import an arbitrary python source file (whose filename could contain any characters, and does not always ends with .py ) in Python 3.3+? I used imp.load_module as follows:
>>> import imp >>> path = '/tmp/a-b.txt' >>> with open(path, 'U') as f: . mod = imp.load_module('a_b', f, path, ('.py', 'U', imp.PY_SOURCE)) . >>> mod Deprecated since version 3.3: Unneeded as loaders should be used to load modules and find_module() is deprecated.
What is the proper way to load an arbitrary python source file in Python 3.3+ without using the deprecated imp.load_module function?
Can I ask why you are doing this? I’m the maintainer of importlib and I have been trying to get answers from folks as to why they use imp.load_module() over a straight import statement. Do you expect to import the module by name later (e.g. import a_b )? Do you care that any custom importers won’t be used in this approach? Do you expect the module to be full-featured (e.g. define __name__ and __loader__ )?
@BrettCannon, A third-party program regularly (once a hour) modify a text file that contains python statements (mainly THIS=’blah’ like lines). The name of the file is not ended with .py . My program read that file.
@BrettCannon — I just ran into a case where I needed to import some Python code from within a directory which was named as a version number (e.g., «v1.0.2»). While possible, it would be highly undesirable to rename the directory. I wound up using stefan-scherfke’s solution below.
5 Answers 5
>>> import importlib.machinery >>> loader = importlib.machinery.SourceFileLoader('a_b', '/tmp/a-b.txt') >>> mod = loader.load_module() >>> mod NOTE: only works in Python 3.3+.
UPDATE Loader.load_module is deprecated since Python 3.4. Use Loader.exec_module instead:
>>> import types >>> import importlib.machinery >>> loader = importlib.machinery.SourceFileLoader('a_b', '/tmp/a-b.txt') >>> mod = types.ModuleType(loader.name) >>> loader.exec_module(mod) >>> mod >>> import importlib.machinery >>> import importlib.util >>> loader = importlib.machinery.SourceFileLoader('a_b', '/tmp/a-b.txt') >>> spec = importlib.util.spec_from_loader(loader.name, loader) >>> mod = importlib.util.module_from_spec(spec) >>> loader.exec_module(mod) >>> mod Downvoter: How can I improve the answer? If you have a better way to accomplish what I want, please let me know.
There’s a helpful warning that load_module ignores via warnings.catch_warnings . If you instead use mod = imp.load_source(‘a_b’, ‘/tmp/a-b.txt’) , it raises the following warning (use -Wall ): DeprecationWarning: imp.load_source() is deprecated; use importlib.machinery.SourceFileLoader(name, pathname).load_module() instead .
@eryksun, You’re right. Thank you for the comment. BTW, Python 3.4(rc1) does not display the alternative usage unlike Python 3.3.x.
@ihavenoidea, Please post a separated question, so that others can answer you, and other users can also read answers.
Updated for Python >= 3.8:
>>> # https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly >>> import importlib.util, sys >>> spec = importlib.util.spec_from_file_location(modname, fname) >>> module = importlib.util.module_from_spec(spec) >>> sys.modules[modname] = module >>> spec.loader.exec_module(module) >>> import importlib.util >>> import sys >>> from pathlib import Path >>> from typing import TYPE_CHECKING >>> >>> >>> if TYPE_CHECKING: . import types . . >>> def import_source_file(fname: str | Path, modname: str) -> "types.ModuleType": . """ . Import a Python source file and return the loaded module. . Args: . fname: The full path to the source file. It may container characters like `.` . or `-`. . modname: The name for the loaded module. It may contain `.` and even characters . that would normally not be allowed (e.g., `-`). . Return: . The imported module . Raises: . ImportError: If the file cannot be imported (e.g, if it's not a `.py` file or if . it does not exist). . Exception: Any exception that is raised while executing the module (e.g., . :exc:`SyntaxError). These are errors made by the author of the module! . """ . # https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly . spec = importlib.util.spec_from_file_location(modname, fname) . if spec is None: . raise ImportError(f"Could not load spec for module '' at: ") . module = importlib.util.module_from_spec(spec) . sys.modules[modname] = module . try: . spec.loader.exec_module(module) . except FileNotFoundError as e: . raise ImportError(f": ") from e . return module . >>> import_source_file(Path("/tmp/my_mod.py"), "my_mod") Original answer for Python 3.5 and 3.6
Shorter version of @falsetru ‘s solution:
>>> import importlib.util >>> spec = importlib.util.spec_from_file_location('a_b', '/tmp/a-b.py') >>> mod = importlib.util.module_from_spec(spec) >>> spec.loader.exec_module(mod) >>> mod I tested it with Python 3.5 and 3.6.
According to the comments, it does not work with arbitrary file extensions.