Python 編譯：code物件與 pyc檔案

發表於2016-01-09

執行程式

當在shell中敲入python xx.py執行 Python 程式時，就是啟用了 Python 直譯器。

Python 直譯器並不會立即執行程式，而是會對 Python 程式的原始碼進行編譯，產生位元組碼，然後將位元組碼交給虛擬機器一條條順序執行。

原始檔中的內容可以分為：字串、常量、操作。

操作會被編譯為位元組碼指令序列，字串和常量在編譯的過程中會被收集起來。這些編譯後的資訊在程式執行時，會作為執行時物件 PyCodeObject 儲存於記憶體中。執行結束後，PyCodeObject 被放入xx.pyc檔案，儲存在硬碟中。這樣，在下次執行時，可以直接根據.pyc檔案的內容，在記憶體中建立 PyCodeObject ，不需要再進行編譯。

PyCodeObject

在編譯器對原始碼進行編譯時，會為每一個 Code Block 建立一個對應的 PyCodeObject。那麼，什麼是 Code Block 呢？規則是：當進入一個新的名字空間，或者新的作用域，就是進入了一個新 Code Block。名字空間是符號的上下文環境，決定了符號的含義。也就是說，決定了變數名對應的變數值是什麼。

名字空間是可以巢狀的，能夠形成一個名字空間鏈，虛擬機器在執行位元組碼時，一個重要的任務就是從鏈中確定一個符號的物件是什麼。

在 Python 中，類、函式、modules 對應獨立的名字空間，所以都有對應的 PyCodeObject。

PyCodeObject 中co_code域儲存的就是對操作編譯生成的位元組碼指令序列。

產生pyc檔案的方法

上面提到，Python 程式執行結束後，會在硬碟中以.pyc檔案的形式儲存 PyCodeObject，但直接執行 Python 程式並不會產生.pyc檔案。

這可能是因為直接執行的 Python 程式，有些只是臨時使用一次，所以沒有通過.pyc儲存編譯結果的必要。

一種常見的，產生pyc檔案的方法是import機制。當Python 程式執行時，如果遇到 import abc，會到設定好的path中尋找 abc.pyc 檔案，如果沒有，只找到abc.py，會先將 abc.py 編譯成 CodeObject，然後建立 pyc 檔案，將 CodeObject寫入，最後才會對 pyc 進行import操作，將 pyc 中的 CodeObject重新複製到記憶體，並執行。

另外，Python 標準庫中的py_compile和compile可以幫助手動產生 pyc 檔案。

pyc 檔案內容是二進位制的，想要了解 pyc 檔案的格式，就要了解 PyCodeObject 中各個域的作用。

PyCodeObject域

在 Python 中訪問 PyCodeObject

C語言形式的 PyCodeObject 對應 Python 中的 Code物件，Code物件是對 PyCodeObject 的簡單包裝。

因此，可以通過 Code物件訪問 PyCodeObject 的各個域。這就需要使用內建函式 compile。

test.py

import sys

a = 1

def b():
    print a
    a = 2
    print a

import sys

a = 1

def b():

print a

a = 2

print a

>>> source = open('/Users/chao/Desktop/test.py').read()
>>> co = compile(source, 'test.py', 'exec')
>>> type(co)
<type 'code'>
>>> dir(co)
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
>>> print co.co_names
()
>>> print co.co_name
<module>
>>> print co.co_filename
test.py

>>> source = open('/Users/chao/Desktop/test.py').read()

>>> co = compile(source, 'test.py', 'exec')

>>> type(co)

>>> dir(co)

['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']

>>> print co.co_names

()

>>> print co.co_name

>>> print co.co_filename

test.py

建立 pyc 檔案

一個 pyc 檔案包含三部分獨立的資訊：

magic number
pyc 檔案的建立時間資訊
PyCodeObject

import.c

static void
write_compiled_module(PyCodeObject *co, char *cpathname, struct stat *srcstat)
{
    FILE *fp;
    time_t mtime = srcstat->st_mtime;
#ifdef MS_WINDOWS   /* since Windows uses different permissions  */
    mode_t mode = srcstat->st_mode & ~S_IEXEC;
#else
    mode_t mode = srcstat->st_mode & ~S_IXUSR & ~S_IXGRP & ~S_IXOTH;
#endif

    fp = open_exclusive(cpathname, mode);
    if (fp == NULL) {
        if (Py_VerboseFlag)
            PySys_WriteStderr(
                "# can't create %s\n", cpathname);
        return;
    }
    PyMarshal_WriteLongToFile(pyc_magic, fp, Py_MARSHAL_VERSION);     # 寫入`magic number`
    /* First write a 0 for mtime */
    PyMarshal_WriteLongToFile(0L, fp, Py_MARSHAL_VERSION);
    PyMarshal_WriteObjectToFile((PyObject *)co, fp, Py_MARSHAL_VERSION);  # 寫入`PyCodeObject`
    if (fflush(fp) != 0 || ferror(fp)) {
        if (Py_VerboseFlag)
            PySys_WriteStderr("# can't write %s\n", cpathname);
        /* Don't keep partial file */
        fclose(fp);
        (void) unlink(cpathname);
        return;
    }
    /* Now write the true mtime */
    fseek(fp, 4L, 0);
    assert(mtime < LONG_MAX);
    PyMarshal_WriteLongToFile((long)mtime, fp, Py_MARSHAL_VERSION);   # 寫入 pyc 建立時間
    fflush(fp);
    fclose(fp);
    if (Py_VerboseFlag)
        PySys_WriteStderr("# wrote %s\n", cpathname);
}

static void

write_compiled_module(PyCodeObject *co, char *cpathname, struct stat *srcstat)

{

FILE *fp;

time_t mtime = srcstat->st_mtime;

#ifdef MS_WINDOWS /* since Windows uses different permissions */

mode_t mode = srcstat->st_mode & ~S_IEXEC;

#else

mode_t mode = srcstat->st_mode & ~S_IXUSR & ~S_IXGRP & ~S_IXOTH;

#endif

fp = open_exclusive(cpathname, mode);

if (fp == NULL) {

if (Py_VerboseFlag)

PySys_WriteStderr(

"# can't create %s\n", cpathname);

return;

}

PyMarshal_WriteLongToFile(pyc_magic, fp, Py_MARSHAL_VERSION); # 寫入`magic number`

/* First write a 0 for mtime */

PyMarshal_WriteLongToFile(0L, fp, Py_MARSHAL_VERSION);

PyMarshal_WriteObjectToFile((PyObject *)co, fp, Py_MARSHAL_VERSION); # 寫入`PyCodeObject`

if (fflush(fp) != 0 || ferror(fp)) {

if (Py_VerboseFlag)

PySys_WriteStderr("# can't write %s\n", cpathname);

/* Don't keep partial file */

fclose(fp);

(void) unlink(cpathname);

return;

}

/* Now write the true mtime */

fseek(fp, 4L, 0);

assert(mtime < LONG_MAX);

PyMarshal_WriteLongToFile((long)mtime, fp, Py_MARSHAL_VERSION); # 寫入 pyc 建立時間

fflush(fp);

fclose(fp);

if (Py_VerboseFlag)

PySys_WriteStderr("# wrote %s\n", cpathname);

}

下面一一進行說明

1，magic number
是 Python 定義的一個整數值，不同版本定義不同，用來確保 Python 的相容性。Python 在載入 pyc 時首先檢查 magic number ，如果與 Python 自身的 magic number 不同，說明建立 pyc 的 Python 版本與當前版本不相容，會拒絕載入。

為什麼會不相容呢？因為位元組碼指令發生了變化，有刪除或增加。

/* Magic word to reject .pyc files generated by other Python versions.
   It should change for each incompatible change to the bytecode.

   The value of CR and LF is incorporated so if you ever read or write
   a .pyc file in text mode the magic number will be wrong; also, the
   Apple MPW compiler swaps their values, botching string constants.

   The magic numbers must be spaced apart atleast 2 values, as the
   -U interpeter flag will cause MAGIC+1 being used. They have been
   odd numbers for some time now.

   There were a variety of old schemes for setting the magic number.
   The current working scheme is to increment the previous value by
   10.

   Known values:
       Python 1.5:   20121
       Python 1.5.1: 20121
       Python 1.5.2: 20121
       Python 1.6:   50428
       Python 2.0:   50823
       Python 2.0.1: 50823
       Python 2.1:   60202
       Python 2.1.1: 60202
       Python 2.1.2: 60202
       Python 2.2:   60717
       Python 2.3a0: 62011
       Python 2.3a0: 62021
       Python 2.3a0: 62011 (!)
       Python 2.4a0: 62041
       Python 2.4a3: 62051
       Python 2.4b1: 62061
       Python 2.5a0: 62071
       Python 2.5a0: 62081 (ast-branch)
       Python 2.5a0: 62091 (with)
       Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)
       Python 2.5b3: 62101 (fix wrong code: for x, in ...)
       Python 2.5b3: 62111 (fix wrong code: x += yield)
       Python 2.5c1: 62121 (fix wrong lnotab with for loops and
                            storing constants that should have been removed)
       Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp)
       Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode)
       Python 2.6a1: 62161 (WITH_CLEANUP optimization)
.
*/
#define MAGIC (62161 | ((long)'\r'<<16) | ((long)'\n'<<24))

/* Magic word as global; note that _PyImport_Init() can change the
   value of this global to accommodate for alterations of how the
   compiler works which are enabled by command line switches. */
static long pyc_magic = MAGIC;

/* Magic word to reject .pyc files generated by other Python versions.

It should change for each incompatible change to the bytecode.

The value of CR and LF is incorporated so if you ever read or write

a .pyc file in text mode the magic number will be wrong; also, the

Apple MPW compiler swaps their values, botching string constants.

The magic numbers must be spaced apart atleast 2 values, as the

-U interpeter flag will cause MAGIC+1 being used. They have been

odd numbers for some time now.

There were a variety of old schemes for setting the magic number.

The current working scheme is to increment the previous value by

10.

Known values:

Python 1.5: 20121

Python 1.5.1: 20121

Python 1.5.2: 20121

Python 1.6: 50428

Python 2.0: 50823

Python 2.0.1: 50823

Python 2.1: 60202

Python 2.1.1: 60202

Python 2.1.2: 60202

Python 2.2: 60717

Python 2.3a0: 62011

Python 2.3a0: 62021

Python 2.3a0: 62011 (!)

Python 2.4a0: 62041

Python 2.4a3: 62051

Python 2.4b1: 62061

Python 2.5a0: 62071

Python 2.5a0: 62081 (ast-branch)

Python 2.5a0: 62091 (with)

Python 2.5a0: 62092 (changed WITH_CLEANUP opcode)

Python 2.5b3: 62101 (fix wrong code: for x, in ...)

Python 2.5b3: 62111 (fix wrong code: x += yield)

Python 2.5c1: 62121 (fix wrong lnotab with for loops and

storing constants that should have been removed)

Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp)

Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode)

Python 2.6a1: 62161 (WITH_CLEANUP optimization)

#define MAGIC (62161 | ((long)'\r'<<16) | ((long)'\n'<<24))

/* Magic word as global; note that _PyImport_Init() can change the

value of this global to accommodate for alterations of how the

compiler works which are enabled by command line switches. */

static long pyc_magic = MAGIC;

2，pyc 建立時間
使得 Python 自動將 pyc 檔案與最新的 Python 檔案同步。當對 Python 程式進行編譯產生 pyc 後，如果後來進行了修改，此時 Python 在嘗試載入 pyc 時，會發現 pyc 建立時間早於 Python 程式，於是將重新編譯，生成新的 pyc 檔案。

3，PyCodeObject
編譯器會遍歷 PyCodeObject 中的所有域，並依次寫入 pyc。對於 PyCodeObject 中的每一個物件，同樣會進行遍歷，並寫入型別標誌和資料（數值/字串）

型別標誌的三個作用：表明上一個物件的結束、新物件的開始、確定新物件的型別

marshal.h，型別標誌

#define TYPE_NULL               '0'
#define TYPE_NONE               'N'
#define TYPE_FALSE              'F'
#define TYPE_TRUE               'T'
#define TYPE_STOPITER           'S'
#define TYPE_ELLIPSIS           '.'
#define TYPE_INT                'i'
#define TYPE_INT64              'I'
#define TYPE_FLOAT              'f'
#define TYPE_BINARY_FLOAT       'g'
#define TYPE_COMPLEX            'x'
#define TYPE_BINARY_COMPLEX     'y'
#define TYPE_LONG               'l'
#define TYPE_STRING             's'
#define TYPE_INTERNED           't'
#define TYPE_STRINGREF          'R'
#define TYPE_TUPLE              '('
#define TYPE_LIST               '['
#define TYPE_DICT               '{'
#define TYPE_CODE               'c'
#define TYPE_UNICODE            'u'
#define TYPE_UNKNOWN            '?'
#define TYPE_SET                '<'
#define TYPE_FROZENSET          '>'

#define TYPE_NULL '0'

#define TYPE_NONE 'N'

#define TYPE_FALSE 'F'

#define TYPE_TRUE 'T'

#define TYPE_STOPITER 'S'

#define TYPE_ELLIPSIS '.'

#define TYPE_INT 'i'

#define TYPE_INT64 'I'

#define TYPE_FLOAT 'f'

#define TYPE_BINARY_FLOAT 'g'

#define TYPE_COMPLEX 'x'

#define TYPE_BINARY_COMPLEX 'y'

#define TYPE_LONG 'l'

#define TYPE_STRING 's'

#define TYPE_INTERNED 't'

#define TYPE_STRINGREF 'R'

#define TYPE_TUPLE '('

#define TYPE_LIST '['

#define TYPE_DICT '{'

#define TYPE_CODE 'c'

#define TYPE_UNICODE 'u'

#define TYPE_UNKNOWN '?'

#define TYPE_SET '<'

#define TYPE_FROZENSET '>'

向 pyc 寫入字串

部分略

對於巢狀的名字空間，產生的 PyCodeObject 也是遞迴巢狀的，巢狀的 PyCodeObject 在上層 PyCodeObject 的co_consts中。

位元組碼

原始碼編譯為位元組碼指令序列，虛擬機器根據位元組碼進行操作，完成程式的執行，opcode.h中定義了當前版本 Python 支援的位元組碼指令。

位元組碼指令的編碼並不是按順序增長的，中間有跳躍。

Include目錄下的opcode.h定義了位元組碼指令

#define STOP_CODE    0
#define POP_TOP        1
#define ROT_TWO        2
#define ROT_THREE    3
#define DUP_TOP        4
#define ROT_FOUR    5
#define NOP        9

#define UNARY_POSITIVE    10
#define UNARY_NEGATIVE    11
#define UNARY_NOT    12
#define UNARY_CONVERT    13

#define UNARY_INVERT    15

#define LIST_APPEND    18
#define BINARY_POWER    19

#define BINARY_MULTIPLY    20
#define BINARY_DIVIDE    21
#define BINARY_MODULO    22
#define BINARY_ADD    23
#define BINARY_SUBTRACT    24
#define BINARY_SUBSCR    25
#define BINARY_FLOOR_DIVIDE 26
#define BINARY_TRUE_DIVIDE 27
#define INPLACE_FLOOR_DIVIDE 28
#define INPLACE_TRUE_DIVIDE 29

#define SLICE        30
/* Also uses 31-33 */

#define STORE_SLICE    40
/* Also uses 41-43 */

#define DELETE_SLICE    50
/* Also uses 51-53 */

#define STORE_MAP    54
#define INPLACE_ADD    55
#define INPLACE_SUBTRACT    56
#define INPLACE_MULTIPLY    57
#define INPLACE_DIVIDE    58
#define INPLACE_MODULO    59
#define STORE_SUBSCR    60
#define DELETE_SUBSCR    61

#define BINARY_LSHIFT    62
#define BINARY_RSHIFT    63
#define BINARY_AND    64
#define BINARY_XOR    65
#define BINARY_OR    66
#define INPLACE_POWER    67
#define GET_ITER    68

#define PRINT_EXPR    70
#define PRINT_ITEM    71
#define PRINT_NEWLINE    72
#define PRINT_ITEM_TO   73
#define PRINT_NEWLINE_TO 74
#define INPLACE_LSHIFT    75
#define INPLACE_RSHIFT    76
#define INPLACE_AND    77
#define INPLACE_XOR    78
#define INPLACE_OR    79
#define BREAK_LOOP    80
#define WITH_CLEANUP    81
#define LOAD_LOCALS    82
#define RETURN_VALUE    83
#define IMPORT_STAR    84
#define EXEC_STMT    85
#define YIELD_VALUE    86
#define POP_BLOCK    87
#define END_FINALLY    88
#define BUILD_CLASS    89

#define HAVE_ARGUMENT    90    /* Opcodes from here have an argument: */

#define STORE_NAME    90    /* Index in name list */
#define DELETE_NAME    91    /* "" */
#define UNPACK_SEQUENCE    92    /* Number of sequence items */
#define FOR_ITER    93

#define STORE_ATTR    95    /* Index in name list */
#define DELETE_ATTR    96    /* "" */
#define STORE_GLOBAL    97    /* "" */
#define DELETE_GLOBAL    98    /* "" */
#define DUP_TOPX    99    /* number of items to duplicate */
#define LOAD_CONST    100    /* Index in const list */
#define LOAD_NAME    101    /* Index in name list */
#define BUILD_TUPLE    102    /* Number of tuple items */
#define BUILD_LIST    103    /* Number of list items */
#define BUILD_MAP    104    /* Always zero for now */
#define LOAD_ATTR    105    /* Index in name list */
#define COMPARE_OP    106    /* Comparison operator */
#define IMPORT_NAME    107    /* Index in name list */
#define IMPORT_FROM    108    /* Index in name list */

#define JUMP_FORWARD    110    /* Number of bytes to skip */
#define JUMP_IF_FALSE    111    /* "" */
#define JUMP_IF_TRUE    112    /* "" */
#define JUMP_ABSOLUTE    113    /* Target byte offset from beginning of code */

#define LOAD_GLOBAL    116    /* Index in name list */

#define CONTINUE_LOOP    119    /* Start of loop (absolute) */
#define SETUP_LOOP    120    /* Target address (relative) */
#define SETUP_EXCEPT    121    /* "" */
#define SETUP_FINALLY    122    /* "" */

#define LOAD_FAST    124    /* Local variable number */
#define STORE_FAST    125    /* Local variable number */
#define DELETE_FAST    126    /* Local variable number */

#define RAISE_VARARGS    130    /* Number of raise arguments (1, 2 or 3) */
/* CALL_FUNCTION_XXX opcodes defined below depend on this definition */
#define CALL_FUNCTION    131    /* #args + (#kwargs<<8) */
#define MAKE_FUNCTION    132    /* #defaults */
#define BUILD_SLICE     133    /* Number of items */

#define MAKE_CLOSURE    134     /* #free vars */
#define LOAD_CLOSURE    135     /* Load free variable from closure */
#define LOAD_DEREF      136     /* Load and dereference from closure cell */ 
#define STORE_DEREF     137     /* Store into cell */ 

/* The next 3 opcodes must be contiguous and satisfy
   (CALL_FUNCTION_VAR - CALL_FUNCTION) & 3 == 1  */
#define CALL_FUNCTION_VAR          140    /* #args + (#kwargs<<8) */
#define CALL_FUNCTION_KW           141    /* #args + (#kwargs<<8) */
#define CALL_FUNCTION_VAR_KW       142    /* #args + (#kwargs<<8) */

/* Support for opargs more than 16 bits long */
#define EXTENDED_ARG  143

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

#define STOP_CODE 0

#define POP_TOP 1

#define ROT_TWO 2

#define ROT_THREE 3

#define DUP_TOP 4

#define ROT_FOUR 5

#define NOP 9

#define UNARY_POSITIVE 10

#define UNARY_NEGATIVE 11

#define UNARY_NOT 12

#define UNARY_CONVERT 13

#define UNARY_INVERT 15

#define LIST_APPEND 18

#define BINARY_POWER 19

#define BINARY_MULTIPLY 20

#define BINARY_DIVIDE 21

#define BINARY_MODULO 22

#define BINARY_ADD 23

#define BINARY_SUBTRACT 24

#define BINARY_SUBSCR 25

#define BINARY_FLOOR_DIVIDE 26

#define BINARY_TRUE_DIVIDE 27

#define INPLACE_FLOOR_DIVIDE 28

#define INPLACE_TRUE_DIVIDE 29

#define SLICE 30

/* Also uses 31-33 */

#define STORE_SLICE 40

/* Also uses 41-43 */

#define DELETE_SLICE 50

/* Also uses 51-53 */

#define STORE_MAP 54

#define INPLACE_ADD 55

#define INPLACE_SUBTRACT 56

#define INPLACE_MULTIPLY 57

#define INPLACE_DIVIDE 58

#define INPLACE_MODULO 59

#define STORE_SUBSCR 60

#define DELETE_SUBSCR 61

#define BINARY_LSHIFT 62

#define BINARY_RSHIFT 63

#define BINARY_AND 64

#define BINARY_XOR 65

#define BINARY_OR 66

#define INPLACE_POWER 67

#define GET_ITER 68

#define PRINT_EXPR 70

#define PRINT_ITEM 71

#define PRINT_NEWLINE 72

#define PRINT_ITEM_TO 73

#define PRINT_NEWLINE_TO 74

#define INPLACE_LSHIFT 75

#define INPLACE_RSHIFT 76

#define INPLACE_AND 77

#define INPLACE_XOR 78

#define INPLACE_OR 79

#define BREAK_LOOP 80

#define WITH_CLEANUP 81

#define LOAD_LOCALS 82

#define RETURN_VALUE 83

#define IMPORT_STAR 84

#define EXEC_STMT 85

#define YIELD_VALUE 86

#define POP_BLOCK 87

#define END_FINALLY 88

#define BUILD_CLASS 89

#define HAVE_ARGUMENT 90 /* Opcodes from here have an argument: */

#define STORE_NAME 90 /* Index in name list */

#define DELETE_NAME 91 /* "" */

#define UNPACK_SEQUENCE 92 /* Number of sequence items */

#define FOR_ITER 93

#define STORE_ATTR 95 /* Index in name list */

#define DELETE_ATTR 96 /* "" */

#define STORE_GLOBAL 97 /* "" */

#define DELETE_GLOBAL 98 /* "" */

#define DUP_TOPX 99 /* number of items to duplicate */

#define LOAD_CONST 100 /* Index in const list */

#define LOAD_NAME 101 /* Index in name list */

#define BUILD_TUPLE 102 /* Number of tuple items */

#define BUILD_LIST 103 /* Number of list items */

#define BUILD_MAP 104 /* Always zero for now */

#define LOAD_ATTR 105 /* Index in name list */

#define COMPARE_OP 106 /* Comparison operator */

#define IMPORT_NAME 107 /* Index in name list */

#define IMPORT_FROM 108 /* Index in name list */

#define JUMP_FORWARD 110 /* Number of bytes to skip */

#define JUMP_IF_FALSE 111 /* "" */

#define JUMP_IF_TRUE 112 /* "" */

#define JUMP_ABSOLUTE 113 /* Target byte offset from beginning of code */

#define LOAD_GLOBAL 116 /* Index in name list */

#define CONTINUE_LOOP 119 /* Start of loop (absolute) */

#define SETUP_LOOP 120 /* Target address (relative) */

#define SETUP_EXCEPT 121 /* "" */

#define SETUP_FINALLY 122 /* "" */

#define LOAD_FAST 124 /* Local variable number */

#define STORE_FAST 125 /* Local variable number */

#define DELETE_FAST 126 /* Local variable number */

#define RAISE_VARARGS 130 /* Number of raise arguments (1, 2 or 3) */

/* CALL_FUNCTION_XXX opcodes defined below depend on this definition */

#define CALL_FUNCTION 131 /* #args + (#kwargs<<8) */

#define MAKE_FUNCTION 132 /* #defaults */

#define BUILD_SLICE 133 /* Number of items */

#define MAKE_CLOSURE 134 /* #free vars */

#define LOAD_CLOSURE 135 /* Load free variable from closure */

#define LOAD_DEREF 136 /* Load and dereference from closure cell */

#define STORE_DEREF 137 /* Store into cell */

/* The next 3 opcodes must be contiguous and satisfy

(CALL_FUNCTION_VAR - CALL_FUNCTION) & 3 == 1 */

#define CALL_FUNCTION_VAR 140 /* #args + (#kwargs<<8) */

#define CALL_FUNCTION_KW 141 /* #args + (#kwargs<<8) */

#define CALL_FUNCTION_VAR_KW 142 /* #args + (#kwargs<<8) */

/* Support for opargs more than 16 bits long */

#define EXTENDED_ARG 143

解析 pyc

由於包含巢狀 PyCodeObject，pyc 中的二進位制資料實際上是有結構的，可以以 XML格式進行解析，從而視覺化。使用 pycparser。

而 Python 庫中 dis 的 dis 方法可以對 code物件進行解析。接收 code物件，輸出位元組碼指令資訊。

dis.dis 的輸出：

第一列，是位元組碼指令對應的原始碼在 Python 程式中的行數
第二列，是當前位元組碼指令在 co_code 中的偏移位置
第三列，當前的位元組碼指令
第四列，當前位元組碼指令的引數

test.py

import sys

a = 1

def b():
    print a
    a = 2
    print a

import sys

a = 1

def b():

print a

a = 2

print a

>>> source = open('/Users/chao/Desktop/test.py').read()
>>> co = compile(source, 'test.py', 'exec')
>>> import dis
>>> dis.dis(co)
  1           0 LOAD_CONST               0 (-1)
              3 LOAD_CONST               1 (None)
              6 IMPORT_NAME              0 (sys)
              9 STORE_NAME               0 (sys)

  3          12 LOAD_CONST               2 (1)
             15 STORE_NAME               1 (a)

  5          18 LOAD_CONST               3 (<code object b at 0x1005dc930, file "test.py", line 5>)
             21 MAKE_FUNCTION            0
             24 STORE_NAME               2 (b)
             27 LOAD_CONST               1 (None)
             30 RETURN_VALUE
>>> type(co)
<type 'code'>
>>> dir(co)
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
>>> print co.co_names
('sys', 'a', 'b')
>>> print co.co_name
<module>
>>> print co.co_filename
test.py

>>> source = open('/Users/chao/Desktop/test.py').read()

>>> co = compile(source, 'test.py', 'exec')

>>> import dis

>>> dis.dis(co)

1 0 LOAD_CONST 0 (-1)

3 LOAD_CONST 1 (None)

6 IMPORT_NAME 0 (sys)

9 STORE_NAME 0 (sys)

3 12 LOAD_CONST 2 (1)

15 STORE_NAME 1 (a)

5 18 LOAD_CONST 3 (<code object b at 0x1005dc930, file "test.py", line 5>)

21 MAKE_FUNCTION 0

24 STORE_NAME 2 (b)

27 LOAD_CONST 1 (None)

30 RETURN_VALUE

>>> type(co)

>>> dir(co)

>>> print co.co_names

('sys', 'a', 'b')

>>> print co.co_name

>>> print co.co_filename

test.py

參考資料

《Python 原始碼剖析》第七章

[Python]批量編譯pyc檔案
2018-08-28
Python編譯
8.Python3原始碼—Code物件與pyc檔案
2018-06-06
Python原始碼物件
python3 將目錄下py檔案編譯為 pyc
2020-11-02
Python編譯
Python下編譯py成pyc和pyo
2017-12-25
Python編譯
28-Python-Pyc檔案（瞭解）
2018-06-06
Python
pyc檔案花指令
2024-11-13
".pyc"檔案是什麼檔案
2015-08-18
Xcode-新增pch預編譯檔案
2016-01-26
XCode編譯
Python中的.pyc檔案是幹什麼的
2019-03-08
Python
深入理解 python 虛擬機器：pyc 檔案結構
2023-03-28
Python虛擬機
如何配置vscode外掛code runner編譯多個*.c檔案
2024-06-24
VSCode編譯
gcc編譯cpp檔案
2012-11-19
GC編譯
Python將py檔案編譯為exe的方法
2022-02-13
Python編譯
安卓apk檔案反編譯
2014-12-20
安卓APK編譯
windows 下編譯c檔案
2009-11-13
Windows編譯
VSCode 使用 Code Runner 外掛無法編譯執行檔名帶空格的檔案
2021-07-07
VSCode編譯
會Python？請聽題那麼是.pyc和PyCodeObject？
2018-07-02
PythonObject
pycharm不生成pyc檔案的方法是什麼？
2021-09-11
PyCharm
win7 編譯java檔案
2017-04-05
Win7編譯Java
c語言多檔案編譯
2018-05-15
C語言編譯
使用ant編譯Java檔案（一）
2007-09-01
編譯Java
使用ant編譯Java檔案（二）
2007-09-01
編譯Java
python程式碼混淆與編譯
2024-08-08
Python編譯
oracle編譯物件失效
2015-02-12
Oracle編譯物件
編譯無效物件
2015-08-17
編譯物件
[轉]andriod的apk檔案相關的編譯反編譯工具
2011-09-09
APK編譯
sublime Text3編譯java檔案提示Decode error - output not utf-8
2018-02-04
編譯JavaError
深入理解 python 虛擬機器：破解核心魔法——反序列化 pyc 檔案
2023-05-17
Python虛擬機
Tinyx 交叉編譯配置檔案(arm版）
2020-04-06
編譯
Include檔案易犯編譯錯誤
2020-04-07
編譯
Android-ffmpeg編譯so檔案
2018-11-01
Android編譯
go編譯靜態檔案到exe
2022-08-01
Go編譯
C++ 預編譯標頭檔案
2013-11-22
C++編譯
Python編解碼問題與文字檔案處理
2021-06-19
Python
Xcode編譯流程
2023-04-28
XCode編譯
Java編譯與反編譯
2021-03-20
Java編譯
ildasm、ilasm修改、反編譯已經編譯的 dll檔案（c#）
2016-03-20
LDAASM編譯C#
使用ilasm 和 ildasm編譯和反編譯工具對DLL檔案修改
2024-03-07
ASMLDA編譯

Python 編譯：code物件 與 pyc檔案

執行程式

PyCodeObject

產生pyc檔案的方法

在 Python 中訪問 PyCodeObject

建立 pyc 檔案

向 pyc 寫入字串

位元組碼

解析 pyc

參考資料

相關文章

Python 編譯：code物件與 pyc檔案