How to create a code object in python?
I’d like to create a new code object with the function types.CodeType() .
There is almost no documentation about this and the existing one says «not for faint of heart»
Tell me what i need and give me some information about each argument passed to types.CodeType ,
possibly posting an example. Note:
In normal use cases you will just need the builtin function compile()
You should use types.CodeType() only if you want to create new instructions that couldn’t be obtained writing normal source code and that require direct access to bytecode.
FWIW, if it’s not mentioned in the language reference, that means that the arguments it expects could be implementation dependent
One wouldn’t normally directly run the constructor. Instead, one would write code (or construct an Abstract Syntax Tree from AST nodes), then use compile().
2 Answers 2
–––––––––––
Disclaimer :
Documentation in this answer is not official and may be incorrect.
This answer is valid only for python version 3.x
In order to create a code object you have to pass to the function CodeType() the following arguments:
CodeType( argcount, # integer kwonlyargcount, # integer nlocals, # integer stacksize, # integer flags, # integer codestring, # bytes consts, # tuple names, # tuple varnames, # tuple filename, # string name, # string firstlineno, # integer lnotab, # bytes freevars, # tuple cellvars # tuple ) Now i will try to explain what is the meaning of each argument.
argcount
Number of arguments to be passed to the function (*args and **kwargs are not included).
kwonlyargcount
Number of keyword-only arguments.
nlocals
Number of local variables ,
namely all variables and parameters(*args and **kwargs included) except global names.
stacksize The amount of stack (virtual machine stack) required by the code ,
if you want to understand how it works , see official Documentation.
flags
A bitmap that says something about the code object:
1 –> code was optimized
2 –> newlocals: there is a new local namespace(for example a function)
4 –> the code accepts an arbitrary number of positional arguments (*args is used)
8 –> the code accepts an arbitrary number of keyworded arguments (*kwargs is used)
32 –> the code is a generator
othes flags are used in older python versions or are activated to say what is imported from __ future __
codestring
A sequence of bytes representing bytecode instructions
if you want a better understanding , see Documentation (same as above)
consts
A tuple containing literals used by the bytecode (for example pre-computed numbers, tuples,and strings)
names
A tuple containing names used by the bytecode
this names are global variables, functions and classes or also attributes loaded from objects
varnames
A tuple containing local names used by the bytecode (arguments first, then local variables)
filename
It is the filename from which the code was compiled.
It can be whatever you want,you are free to lie about this. 😉
name
It gives the name of the function. Also this can be whatever you want,but be careful:
this is the name shown in the traceback,if the name is unclear,the traceback could be unclear,
just think about how lambdas can be annoying.
firstlineno
The first line of the function (for debug purpose if you compiled source code)
lnotab
A mapping of bytes that correlates bytecode offsets to line numbers.
(i think also this is for debug purpose,there is few documentation about this)
freevars
A tuple containing the names of free variables.
Free variables are variables declared in the namespace where the code object was defined, they are used when nested functions are declared;
this doesn’t happen at module level because in that case free variables are also global variables.
cellvars
A tuple containing names of local variables referenced by nested functions.
––––––––––––
Examples :
following examples should clarify the meaning of what has been said above.
Note: in finished code objects attributes mentioned above have the co_ prefix,
and a function stores its executable body in the __code__ attribute
def F(a,b): global c k=a*c w=10 p=(1,"two",3) print(F.__code__.co_argcount) print(F.__code__.co_nlocals , F.__code__.co_varnames) print(F.__code__.co_stacksize) print(F.__code__.co_flags) print(F.__code__.co_names) print(F.__code__.co_consts) 2 5 ('a', 'b', 'k', 'w', 'p') 3 67 ('c' ,) (None, 10, 1, 'two'. 3, (1, 'two', 3)) - there are two arguments passed to this function («a»,»b»)
- this function has two parameters(«a»,»b») and three local variables(«k»,»w»,»p»)
- disassembling the function bytecode we obtain this:
3 0 LOAD_FAST 0 (a) #stack: ["a"] 3 LOAD_GLOBAL 0 (c) #stack: ["a","c"] 6 BINARY_MULTIPLY #stack: [result of a*c] 7 STORE_FAST 2 (k) #stack: [] 4 10 LOAD_CONST 1 (10) #stack: [10] 13 STORE_FAST 3 (w) #stack: [] 5 16 LOAD_CONST 5 ((1, 'two', 3)) #stack: [(1,"two",3)] 19 STORE_FAST 4 (p) #stack: [] 22 LOAD_CONST 0 (None) #stack: [None] 25 RETURN_VALUE #stack: [] - the code is optimized(as most of the automatically generated code is)
- while executing the function bytecode local namespace changes
- 64? Actually i don’t know what is its meaning
- None is the return value of the function
- we explicitly assign the number 10 to w
- we explicitly assign (1, ‘two’, 3) to p
- if the tuple is a constant each element of that tuple is a constant,so 1,»two»,3 are constants
ModuleVar="hi" def F(): FunctionVar=106 UnusedVar=ModuleVar def G(): return (FunctionVar,ModuleVar) print(G.__code__.co_freevars) print(G.__code__.co_names) F() print(F.__code__.co_cellvars) print(F.__code__.co_freevars) print(F.__code__.co_names) ('FunctionVar',) ('ModuleVar',) ('FunctionVar',) () ('print', '__code__', 'co_freevars', 'co_names', 'ModuleVar') the meaning of the output is this:
first and second line are printed when F is executed,so they show co_freevars and co_names of G code:
«FunctionVar» is in the namespace of F function,where G was created,
«ModuleVar» instead is a module variable,so it is considered as global.
following three lines are about co_cellvars,co_freevars and co_names attributes of F code:
«FunctionVar» is referenced in the G nested function ,so it is marked as a cellvar,
«ModuleVar» is in the namespace where F was created,but it is a module variable,
so it is not marked as freevar,but it is found in global names.
also the builtin function print is marked in names , and all the names of attributes used in F.
This is a working code object initialization,
this is unuseful but you can do everything you want with this function.
MyCode= CodeType( 0, 0, 0, 3, 64, bytes([101, 0, 0, #Load print function 101, 1, 0, #Load name 'a' 101, 2, 0, #Load name 'b' 23, #Take first two stack elements and store their sum 131, 1, 0, #Call first element in the stack with one positional argument 1, #Pop top of stack 101, 0, 0, #Load print function 101, 1, 0, #Load name 'a' 101, 2, 0, #Load name 'b' 20, #Take first two stack elements and store their product 131, 1, 0, #Call first element in the stack with one positional argument 1, #Pop top of stack 100, 0, 0, #Load constant None 83]), #Return top of stack (None,), ('print', 'a', 'b'), (), 'PersonalCodeObject', 'MyCode', 1, bytes([14,1]), (), () ) a=2 b=3 exec(MyCode) # code prints the sum and the product of "a" and "b" Code Objects¶
Code objects are a low-level detail of the CPython implementation. Each one represents a chunk of executable code that hasn’t yet been bound into a function.
The C structure of the objects used to describe code objects. The fields of this type are subject to change at any time.
This is an instance of PyTypeObject representing the Python code type.
Return true if co is a code object. This function always succeeds.
int PyCode_GetNumFree ( PyCodeObject * co ) ¶
Return the number of free variables in co.
PyCodeObject * PyCode_New ( int argcount , int kwonlyargcount , int nlocals , int stacksize , int flags , PyObject * code , PyObject * consts , PyObject * names , PyObject * varnames , PyObject * freevars , PyObject * cellvars , PyObject * filename , PyObject * name , int firstlineno , PyObject * linetable , PyObject * exceptiontable ) ¶
Return value: New reference.
Return a new code object. If you need a dummy code object to create a frame, use PyCode_NewEmpty() instead. Calling PyCode_New() directly will bind you to a precise Python version since the definition of the bytecode changes often. The many arguments of this function are inter-dependent in complex ways, meaning that subtle changes to values are likely to result in incorrect execution or VM crashes. Use this function only with extreme care.
Changed in version 3.11: Added exceptiontable parameter.
PyCodeObject * PyCode_NewWithPosOnlyArgs ( int argcount , int posonlyargcount , int kwonlyargcount , int nlocals , int stacksize , int flags , PyObject * code , PyObject * consts , PyObject * names , PyObject * varnames , PyObject * freevars , PyObject * cellvars , PyObject * filename , PyObject * name , int firstlineno , PyObject * linetable , PyObject * exceptiontable ) ¶
Return value: New reference.
Similar to PyCode_New() , but with an extra “posonlyargcount” for positional-only arguments. The same caveats that apply to PyCode_New also apply to this function.
Changed in version 3.11: Added exceptiontable parameter.
PyCodeObject * PyCode_NewEmpty ( const char * filename , const char * funcname , int firstlineno ) ¶
Return value: New reference.
Return a new empty code object with the specified filename, function name, and first line number. The resulting code object will raise an Exception if executed.
int PyCode_Addr2Line ( PyCodeObject * co , int byte_offset ) ¶
Return the line number of the instruction that occurs on or before byte_offset and ends after it. If you just need the line number of a frame, use PyFrame_GetLineNumber() instead.
For efficiently iterating over the line numbers in a code object, use the API described in PEP 626.
int PyCode_Addr2Location ( PyObject * co , int byte_offset , int * start_line , int * start_column , int * end_line , int * end_column ) ¶
Sets the passed int pointers to the source code line and column numbers for the instruction at byte_offset . Sets the value to 0 when information is not available for any particular element.
Returns 1 if the function succeeds and 0 otherwise.
Equivalent to the Python code getattr(co, ‘co_code’) . Returns a strong reference to a PyBytesObject representing the bytecode in a code object. On error, NULL is returned and an exception is raised.
This PyBytesObject may be created on-demand by the interpreter and does not necessarily represent the bytecode actually executed by CPython. The primary use case for this function is debuggers and profilers.
Equivalent to the Python code getattr(co, ‘co_varnames’) . Returns a new reference to a PyTupleObject containing the names of the local variables. On error, NULL is returned and an exception is raised.
Equivalent to the Python code getattr(co, ‘co_cellvars’) . Returns a new reference to a PyTupleObject containing the names of the local variables that are referenced by nested functions. On error, NULL is returned and an exception is raised.
Equivalent to the Python code getattr(co, ‘co_freevars’) . Returns a new reference to a PyTupleObject containing the names of the free variables. On error, NULL is returned and an exception is raised.