1 .. highlightlang:: c

     2 

     3 ********************************

     4 Porting Extension Modules to 3.0

     5 ********************************

     6 

     7 :author: Benjamin Peterson

     8 

     9 

    10 .. topic:: Abstract

    11 

    12    Although changing the C-API was not one of Python 3.0's objectives, the many

    13    Python level changes made leaving 2.x's API intact impossible.  In fact, some

    14    changes such as :func:`int` and :func:`long` unification are more obvious on

    15    the C level.  This document endeavors to document incompatibilities and how

    16    they can be worked around.

    17 

    18 

    19 Conditional compilation

    20 =======================

    21 

    22 The easiest way to compile only some code for 3.0 is to check if

    23 :cmacro:`PY_MAJOR_VERSION` is greater than or equal to 3. ::

    24 

    25    #if PY_MAJOR_VERSION >= 3

    26    #define IS_PY3K

    27    #endif

    28 

    29 API functions that are not present can be aliased to their equivalents within

    30 conditional blocks.

    31 

    32 

    33 Changes to Object APIs

    34 ======================

    35 

    36 Python 3.0 merged together some types with similar functions while cleanly

    37 separating others.

    38 

    39 

    40 str/unicode Unification

    41 -----------------------

    42 

    43 

    44 Python 3.0's :func:`str` (``PyString_*`` functions in C) type is equivalent to

    45 2.x's :func:`unicode` (``PyUnicode_*``).  The old 8-bit string type has become

    46 :func:`bytes`.  Python 2.6 and later provide a compatibility header,

    47 :file:`bytesobject.h`, mapping ``PyBytes`` names to ``PyString`` ones.  For best

    48 compatibility with 3.0, :ctype:`PyUnicode` should be used for textual data and

    49 :ctype:`PyBytes` for binary data.  It's also important to remember that

    50 :ctype:`PyBytes` and :ctype:`PyUnicode` in 3.0 are not interchangeable like

    51 :ctype:`PyString` and :ctype:`PyString` are in 2.x.  The following example shows

    52 best practices with regards to :ctype:`PyUnicode`, :ctype:`PyString`, and

    53 :ctype:`PyBytes`. ::

    54 

    55    #include "stdlib.h"

    56    #include "Python.h"

    57    #include "bytesobject.h"

    58 

    59    /* text example */

    60    static PyObject *

    61    say_hello(PyObject *self, PyObject *args) {

    62        PyObject *name, *result;

    63 

    64        if (!PyArg_ParseTuple(args, "U:say_hello", &name))

    65            return NULL;

    66 

    67        result = PyUnicode_FromFormat("Hello, %S!", name);

    68        return result;

    69    }

    70 

    71    /* just a forward */

    72    static char * do_encode(PyObject *);

    73 

    74    /* bytes example */

    75    static PyObject *

    76    encode_object(PyObject *self, PyObject *args) {

    77        char *encoded;

    78        PyObject *result, *myobj;

    79 

    80        if (!PyArg_ParseTuple(args, "O:encode_object", &myobj))

    81            return NULL;

    82 

    83        encoded = do_encode(myobj);

    84        if (encoded == NULL)

    85            return NULL;

    86        result = PyBytes_FromString(encoded);

    87        free(encoded);

    88        return result;

    89    }

    90 

    91 

    92 long/int Unification

    93 --------------------

    94 

    95 In Python 3.0, there is only one integer type.  It is called :func:`int` on the

    96 Python level, but actually corresponds to 2.x's :func:`long` type.  In the

    97 C-API, ``PyInt_*`` functions are replaced by their ``PyLong_*`` neighbors.  The

    98 best course of action here is using the ``PyInt_*`` functions aliased to

    99 ``PyLong_*`` found in :file:`intobject.h`.  The the abstract ``PyNumber_*`` APIs

   100 can also be used in some cases. ::

   101 

   102    #include "Python.h"

   103    #include "intobject.h"

   104 

   105    static PyObject *

   106    add_ints(PyObject *self, PyObject *args) {

   107        int one, two;

   108        PyObject *result;

   109 

   110        if (!PyArg_ParseTuple(args, "ii:add_ints", &one, &two))

   111            return NULL;

   112 

   113        return PyInt_FromLong(one + two);

   114    }

   115 

   116 

   117 

   118 Module initialization and state

   119 ===============================

   120 

   121 Python 3.0 has a revamped extension module initialization system.  (See PEP

   122 :pep:`3121`.)  Instead of storing module state in globals, they should be stored

   123 in an interpreter specific structure.  Creating modules that act correctly in

   124 both 2.x and 3.0 is tricky.  The following simple example demonstrates how. ::

   125 

   126    #include "Python.h"

   127 

   128    struct module_state {

   129        PyObject *error;

   130    };

   131 

   132    #if PY_MAJOR_VERSION >= 3

   133    #define GETSTATE(m) ((struct module_state*)PyModule_GetState(m))

   134    #else

   135    #define GETSTATE(m) (&_state)

   136    static struct module_state _state;

   137    #endif

   138 

   139    static PyObject *

   140    error_out(PyObject *m) {

   141        struct module_state *st = GETSTATE(m);

   142        PyErr_SetString(st->error, "something bad happened");

   143        return NULL;

   144    }

   145 

   146    static PyMethodDef myextension_methods[] = {

   147        {"error_out", (PyCFunction)error_out, METH_NOARGS, NULL},

   148        {NULL, NULL}

   149    };

   150 

   151    #if PY_MAJOR_VERSION >= 3

   152 

   153    static int myextension_traverse(PyObject *m, visitproc visit, void *arg) {

   154        Py_VISIT(GETSTATE(m)->error);

   155        return 0;

   156    }

   157 

   158    static int myextension_clear(PyObject *m) {

   159        Py_CLEAR(GETSTATE(m)->error);

   160        return 0;

   161    }

   162 

   163 

   164    static struct PyModuleDef moduledef = {

   165            PyModuleDef_HEAD_INIT,

   166            "myextension",

   167            NULL,

   168            sizeof(struct module_state),

   169            myextension_methods,

   170            NULL,

   171            myextension_traverse,

   172            myextension_clear,

   173            NULL

   174    };

   175 

   176    #define INITERROR return NULL

   177 

   178    PyObject *

   179    PyInit_myextension(void)

   180 

   181    #else

   182    #define INITERROR return

   183 

   184    void

   185    initmyextension(void)

   186    #endif

   187    {

   188    #if PY_MAJOR_VERSION >= 3

   189        PyObject *module = PyModule_Create(&moduledef);

   190    #else

   191        PyObject *module = Py_InitModule("myextension", myextension_methods);

   192    #endif

   193 

   194        if (module == NULL)

   195            INITERROR;

   196        struct module_state *st = GETSTATE(module);

   197 

   198        st->error = PyErr_NewException("myextension.Error", NULL, NULL);

   199        if (st->error == NULL) {

   200            Py_DECREF(module);

   201            INITERROR;

   202        }

   203 

   204    #if PY_MAJOR_VERSION >= 3

   205        return module;

   206    #endif

   207    }

   208 

   209 

   210 Other options

   211 =============

   212 

   213 If you are writing a new extension module, you might consider `Cython

   214 <http://www.cython.org>`_.  It translates a Python-like language to C.  The

   215 extension modules it creates are compatible with Python 3.x and 2.x.

   216