1 .. highlightlang:: c

     2 

     3 

     4 .. _embedding:

     5 

     6 ***************************************

     7 Embedding Python in Another Application

     8 ***************************************

     9 

    10 The previous chapters discussed how to extend Python, that is, how to extend the

    11 functionality of Python by attaching a library of C functions to it.  It is also

    12 possible to do it the other way around: enrich your C/C++ application by

    13 embedding Python in it.  Embedding provides your application with the ability to

    14 implement some of the functionality of your application in Python rather than C

    15 or C++. This can be used for many purposes; one example would be to allow users

    16 to tailor the application to their needs by writing some scripts in Python.  You

    17 can also use it yourself if some of the functionality can be written in Python

    18 more easily.

    19 

    20 Embedding Python is similar to extending it, but not quite.  The difference is

    21 that when you extend Python, the main program of the application is still the

    22 Python interpreter, while if you embed Python, the main program may have nothing

    23 to do with Python --- instead, some parts of the application occasionally call

    24 the Python interpreter to run some Python code.

    25 

    26 So if you are embedding Python, you are providing your own main program.  One of

    27 the things this main program has to do is initialize the Python interpreter.  At

    28 the very least, you have to call the function :cfunc:`Py_Initialize`.  There are

    29 optional calls to pass command line arguments to Python.  Then later you can

    30 call the interpreter from any part of the application.

    31 

    32 There are several different ways to call the interpreter: you can pass a string

    33 containing Python statements to :cfunc:`PyRun_SimpleString`, or you can pass a

    34 stdio file pointer and a file name (for identification in error messages only)

    35 to :cfunc:`PyRun_SimpleFile`.  You can also call the lower-level operations

    36 described in the previous chapters to construct and use Python objects.

    37 

    38 A simple demo of embedding Python can be found in the directory

    39 :file:`Demo/embed/` of the source distribution.

    40 

    41 

    42 .. seealso::

    43 

    44    :ref:`c-api-index`

    45       The details of Python's C interface are given in this manual. A great deal of

    46       necessary information can be found here.

    47 

    48 

    49 .. _high-level-embedding:

    50 

    51 Very High Level Embedding

    52 =========================

    53 

    54 The simplest form of embedding Python is the use of the very high level

    55 interface. This interface is intended to execute a Python script without needing

    56 to interact with the application directly. This can for example be used to

    57 perform some operation on a file. ::

    58 

    59    #include <Python.h>

    60 

    61    int

    62    main(int argc, char *argv[])

    63    {

    64      Py_Initialize();

    65      PyRun_SimpleString("from time import time,ctime\n"

    66                         "print 'Today is',ctime(time())\n");

    67      Py_Finalize();

    68      return 0;

    69    }

    70 

    71 The above code first initializes the Python interpreter with

    72 :cfunc:`Py_Initialize`, followed by the execution of a hard-coded Python script

    73 that print the date and time.  Afterwards, the :cfunc:`Py_Finalize` call shuts

    74 the interpreter down, followed by the end of the program.  In a real program,

    75 you may want to get the Python script from another source, perhaps a text-editor

    76 routine, a file, or a database.  Getting the Python code from a file can better

    77 be done by using the :cfunc:`PyRun_SimpleFile` function, which saves you the

    78 trouble of allocating memory space and loading the file contents.

    79 

    80 

    81 .. _lower-level-embedding:

    82 

    83 Beyond Very High Level Embedding: An overview

    84 =============================================

    85 

    86 The high level interface gives you the ability to execute arbitrary pieces of

    87 Python code from your application, but exchanging data values is quite

    88 cumbersome to say the least. If you want that, you should use lower level calls.

    89 At the cost of having to write more C code, you can achieve almost anything.

    90 

    91 It should be noted that extending Python and embedding Python is quite the same

    92 activity, despite the different intent. Most topics discussed in the previous

    93 chapters are still valid. To show this, consider what the extension code from

    94 Python to C really does:

    95 

    96 #. Convert data values from Python to C,

    97 

    98 #. Perform a function call to a C routine using the converted values, and

    99 

   100 #. Convert the data values from the call from C to Python.

   101 

   102 When embedding Python, the interface code does:

   103 

   104 #. Convert data values from C to Python,

   105 

   106 #. Perform a function call to a Python interface routine using the converted

   107    values, and

   108 

   109 #. Convert the data values from the call from Python to C.

   110 

   111 As you can see, the data conversion steps are simply swapped to accommodate the

   112 different direction of the cross-language transfer. The only difference is the

   113 routine that you call between both data conversions. When extending, you call a

   114 C routine, when embedding, you call a Python routine.

   115 

   116 This chapter will not discuss how to convert data from Python to C and vice

   117 versa.  Also, proper use of references and dealing with errors is assumed to be

   118 understood.  Since these aspects do not differ from extending the interpreter,

   119 you can refer to earlier chapters for the required information.

   120 

   121 

   122 .. _pure-embedding:

   123 

   124 Pure Embedding

   125 ==============

   126 

   127 The first program aims to execute a function in a Python script. Like in the

   128 section about the very high level interface, the Python interpreter does not

   129 directly interact with the application (but that will change in the next

   130 section).

   131 

   132 The code to run a function defined in a Python script is:

   133 

   134 .. literalinclude:: ../includes/run-func.c

   135 

   136 

   137 This code loads a Python script using ``argv[1]``, and calls the function named

   138 in ``argv[2]``.  Its integer arguments are the other values of the ``argv``

   139 array.  If you compile and link this program (let's call the finished executable

   140 :program:`call`), and use it to execute a Python script, such as::

   141 

   142    def multiply(a,b):

   143        print "Will compute", a, "times", b

   144        c = 0

   145        for i in range(0, a):

   146            c = c + b

   147        return c

   148 

   149 then the result should be::

   150 

   151    $ call multiply multiply 3 2

   152    Will compute 3 times 2

   153    Result of call: 6

   154 

   155 Although the program is quite large for its functionality, most of the code is

   156 for data conversion between Python and C, and for error reporting.  The

   157 interesting part with respect to embedding Python starts with ::

   158 

   159    Py_Initialize();

   160    pName = PyString_FromString(argv[1]);

   161    /* Error checking of pName left out */

   162    pModule = PyImport_Import(pName);

   163 

   164 After initializing the interpreter, the script is loaded using

   165 :cfunc:`PyImport_Import`.  This routine needs a Python string as its argument,

   166 which is constructed using the :cfunc:`PyString_FromString` data conversion

   167 routine. ::

   168 

   169    pFunc = PyObject_GetAttrString(pModule, argv[2]);

   170    /* pFunc is a new reference */

   171 

   172    if (pFunc && PyCallable_Check(pFunc)) {

   173        ...

   174    }

   175    Py_XDECREF(pFunc);

   176 

   177 Once the script is loaded, the name we're looking for is retrieved using

   178 :cfunc:`PyObject_GetAttrString`.  If the name exists, and the object returned is

   179 callable, you can safely assume that it is a function.  The program then

   180 proceeds by constructing a tuple of arguments as normal.  The call to the Python

   181 function is then made with::

   182 

   183    pValue = PyObject_CallObject(pFunc, pArgs);

   184 

   185 Upon return of the function, ``pValue`` is either *NULL* or it contains a

   186 reference to the return value of the function.  Be sure to release the reference

   187 after examining the value.

   188 

   189 

   190 .. _extending-with-embedding:

   191 

   192 Extending Embedded Python

   193 =========================

   194 

   195 Until now, the embedded Python interpreter had no access to functionality from

   196 the application itself.  The Python API allows this by extending the embedded

   197 interpreter.  That is, the embedded interpreter gets extended with routines

   198 provided by the application. While it sounds complex, it is not so bad.  Simply

   199 forget for a while that the application starts the Python interpreter.  Instead,

   200 consider the application to be a set of subroutines, and write some glue code

   201 that gives Python access to those routines, just like you would write a normal

   202 Python extension.  For example::

   203 

   204    static int numargs=0;

   205 

   206    /* Return the number of arguments of the application command line */

   207    static PyObject*

   208    emb_numargs(PyObject *self, PyObject *args)

   209    {

   210        if(!PyArg_ParseTuple(args, ":numargs"))

   211            return NULL;

   212        return Py_BuildValue("i", numargs);

   213    }

   214 

   215    static PyMethodDef EmbMethods[] = {

   216        {"numargs", emb_numargs, METH_VARARGS,

   217         "Return the number of arguments received by the process."},

   218        {NULL, NULL, 0, NULL}

   219    };

   220 

   221 Insert the above code just above the :cfunc:`main` function. Also, insert the

   222 following two statements directly after :cfunc:`Py_Initialize`::

   223 

   224    numargs = argc;

   225    Py_InitModule("emb", EmbMethods);

   226 

   227 These two lines initialize the ``numargs`` variable, and make the

   228 :func:`emb.numargs` function accessible to the embedded Python interpreter.

   229 With these extensions, the Python script can do things like ::

   230 

   231    import emb

   232    print "Number of arguments", emb.numargs()

   233 

   234 In a real application, the methods will expose an API of the application to

   235 Python.

   236 

   237 .. TODO: threads, code examples do not really behave well if errors happen

   238    (what to watch out for)

   239 

   240 

   241 .. _embeddingincplusplus:

   242 

   243 Embedding Python in C++

   244 =======================

   245 

   246 It is also possible to embed Python in a C++ program; precisely how this is done

   247 will depend on the details of the C++ system used; in general you will need to

   248 write the main program in C++, and use the C++ compiler to compile and link your

   249 program.  There is no need to recompile Python itself using C++.

   250 

   251 

   252 .. _link-reqs:

   253 

   254 Linking Requirements

   255 ====================

   256 

   257 While the :program:`configure` script shipped with the Python sources will

   258 correctly build Python to export the symbols needed by dynamically linked

   259 extensions, this is not automatically inherited by applications which embed the

   260 Python library statically, at least on Unix.  This is an issue when the

   261 application is linked to the static runtime library (:file:`libpython.a`) and

   262 needs to load dynamic extensions (implemented as :file:`.so` files).

   263 

   264 The problem is that some entry points are defined by the Python runtime solely

   265 for extension modules to use.  If the embedding application does not use any of

   266 these entry points, some linkers will not include those entries in the symbol

   267 table of the finished executable.  Some additional options are needed to inform

   268 the linker not to remove these symbols.

   269 

   270 Determining the right options to use for any given platform can be quite

   271 difficult, but fortunately the Python configuration already has those values.

   272 To retrieve them from an installed Python interpreter, start an interactive

   273 interpreter and have a short session like this::

   274 

   275    >>> import distutils.sysconfig

   276    >>> distutils.sysconfig.get_config_var('LINKFORSHARED')

   277    '-Xlinker -export-dynamic'

   278 

   279 .. index:: module: distutils.sysconfig

   280 

   281 The contents of the string presented will be the options that should be used.

   282 If the string is empty, there's no need to add any additional options.  The

   283 :const:`LINKFORSHARED` definition corresponds to the variable of the same name

   284 in Python's top-level :file:`Makefile`.

   285