1 ==================

     2 PyInstaller Manual

     3 ==================

     4 :Author: William Caban (based on Gordon McMillan's manual)

     5 :Contact: william@hpcf.upr.edu

     6 :Revision: $Rev: 257 $

     7 :Source URL: $HeadURL: http://svn.pyinstaller.python-hosting.com/trunk/doc/source/Manual.rst $

     8 :Copyright: This document has been placed in the public domain.

     9 

    10 .. contents::

    11 

    12 

    13 Getting Started

    14 +++++++++++++++

    15 

    16 Installing PyInstaller

    17 ----------------------

    18 

    19 First, unpack the archive on you path of choice. Installer is **not** a Python

    20 package, so it doesn't need to go in site-packages, or have a .pth file. For

    21 the purpose of this documentation we will assume |install_path|. You will be

    22 using a couple of scripts in the |install_path| directory, and these will find

    23 everything they need from their own location. For convenience, keep the paths

    24 to these scripts short (don't install in a deeply nested subdirectory).

    25 

    26 |PyInstaller| is dependant to the version of python you configure it for. In

    27 other words, you will need a separate copy of |PyInstaller| for each Python

    28 version you wish to work with *or* you'll need to rerun ``Configure.py`` every

    29 time you switch the Python version).

    30 

    31 |GOBACK|

    32 

    33 

    34 Building the runtime executables

    35 --------------------------------

    36 

    37 *Note:* Windows users can skip this step, because all of Python is contained in

    38 pythonXX.dll, and |PyInstaller| will use your pythonXX.dll.

    39 

    40 On Linux the first thing to do is build the runtime executables.

    41 

    42 Change to the |install_path| ``source/linux`` subdirectory. Run ``Make.py

    43 [-n|-e]`` and then make. This will produce ``support/loader/run`` and

    44 ``support/loader/run_d``, which are the bootloaders.

    45 

    46 .. sidebar:: Bootloader

    47 

    48    The bootloader (also known as *stub* in literature) is the small program

    49    which starts up your packaged program. Usually, the archive containing the

    50    bytecoded modules of your program is simply attended to it. See

    51    `Self-extracting executables`_ for more details on the process.

    52 

    53 *Note:* If you have multiple versions of Python, the Python you use to run

    54 ``Make.py`` is the one whose configuration is used.

    55 

    56 The ``-n`` and ``-e`` options set a non-elf or elf flag in your ``config.dat``.

    57 As of |InitialVersion|, the executable will try both strategies, and this flag

    58 just sets how you want your executables built. In the elf strategy, the archive

    59 is concatenated to the executable. In the non-elf strategy, the executable

    60 expects an archive with the same name as itself in the executable's directory.

    61 Note that the executable chases down symbolic links before determining it's name

    62 and directory, so putting the archive in the same directory as the symbolic link

    63 will not work.

    64 

    65 Windows distributions come with several executables in the ``support/loader``

    66 directory: ``run_*.exe`` (bootloader for regular programs), and

    67 ``inprocsrvr_*.dll`` (bootloader for in-process COM servers). To rebuild this,

    68 you need to install Scons_, and then just run ``scons`` from the |install_path|

    69 directory.

    70 

    71 |GOBACK|

    72 

    73 Configuring your PyInstaller setup

    74 ----------------------------------

    75 

    76 In the |install_path| directory, run ``Configure.py``. This saves some

    77 information into ``config.dat`` that would otherwise be recomputed every time.

    78 It can be rerun at any time if your configuration changes. It must be run before

    79 trying to build anything.

    80 

    81 |GOBACK|

    82 

    83 

    84 Create a spec file for your project

    85 -----------------------------------

    86 

    87 [For Windows COM server support, see section `Windows COM Server Support`_]

    88 

    89 The root directory has a script Makespec.py for this purpose::

    90 

    91        python Makespec.py [opts] <scriptname> [<scriptname> ...]

    92 

    93 Where allowed OPTIONS are:

    94 

    95 -F, --onefile

    96     produce a single file deployment (see below).

    97 

    98 -D, --onedir

    99     produce a single directory deployment (default).

   100 

   101 -K, --tk

   102     include TCL/TK in the deployment.

   103 

   104 -a, --ascii

   105     do not include encodings. The default (on Python versions with unicode

   106     support) is now to include all encodings.

   107 

   108 -d, --debug

   109     use debug (verbose) versions of the executables.

   110 

   111 -w, --windowed, --noconsole

   112     Use the Windows subsystem executable, which does not open

   113     the console when the program is launched. **(Windows only)**

   114 

   115 -c, --nowindowed, --console

   116     Use the console subsystem executable. This is the default. **(Windows only)**

   117 

   118 -s, --strip

   119     the executable and all shared libraries will be run through strip. Note

   120     that cygwin's strip tends to render normal Win32 dlls unusable.

   121 

   122 -X, --upx

   123     if you have UPX installed (detected by Configure), this will use it to

   124     compress your executable (and, on Windows, your dlls). See note below.

   125 

   126 -o DIR, --out=DIR

   127     create the spec file in *directory*. If not specified, and the current

   128     directory is Installer's root directory, an output subdirectory will be

   129     created. Otherwise the current directory is used.

   130 

   131 -p DIR, --paths=DIR

   132     set base path for import (like using PYTHONPATH). Multiple directories are

   133     allowed, separating them with the path separator (';' under Windows, ':'

   134     under Linux), or using this option multiple times.

   135 

   136 --icon=<FILE.ICO>

   137     add *file.ico* to the executable's resources. **(Windows only)**

   138 

   139 --icon=<FILE.EXE,N>

   140     add the *n*-th incon in *file.exe* to the executable's resources. **(Windows

   141     only)**

   142 

   143 -v FILE, --version=FILE

   144     add verfile as a version resource to the executable. **(Windows only)**

   145 

   146 -n NAME, --name=NAME

   147     optional *name* to assign to the project (from which the spec file name is

   148     generated). If omitted, the basename of the (first) script is used.

   149 

   150 [For building with optimization on (like ``Python -O``), see section

   151 `Building Optimized`_]

   152 

   153 For simple projects, the generated spec file will probably be sufficient. For

   154 more complex projects, it should be regarded as a template. The spec file is

   155 actually Python code, and modifying it should be ease. See `Spec Files`_ for

   156 details.

   157 

   158 

   159 |GOBACK|

   160 

   161 Build your project

   162 ------------------

   163 

   164 ::

   165 

   166       python Build.py specfile

   167 

   168 

   169 A ``buildproject`` subdirectory will be created in the specfile's directory. This

   170 is a private workspace so that ``Build.py`` can act like a makefile. Any named

   171 targets will appear in the specfile's directory. For ``--onedir``

   172 configurations, it will create also ``distproject``, which is the directory you're

   173 interested in. For a ``--onefile``, the executable will be in the specfile's

   174 directory.

   175 

   176 In most cases, this will be all you have to do. If not, see `When things go

   177 wrong`_ and be sure to read the introduction to `Spec Files`_.

   178 

   179 |GOBACK|

   180 

   181 Windows COM Server support

   182 --------------------------

   183 

   184 For Windows COM support execute::

   185 

   186        python MakeCOMServer.py [OPTION] script...

   187 

   188 

   189 This will generate a new script ``drivescript.py`` and a spec file for the script.

   190 

   191 These options are allowed:

   192 

   193 --debug

   194     Use the verbose version of the executable.

   195 

   196 --verbose

   197     Register the COM server(s) with the quiet flag off.

   198 

   199 --ascii

   200     do not include encodings (this is passed through to Makespec).

   201 

   202 --out <dir>

   203     Generate the driver script and spec file in dir.

   204 

   205 Now `Build your project`_ on the generated spec file.

   206 

   207 If you have the win32dbg package installed, you can use it with the generated

   208 COM server. In the driver script, set ``debug=1`` in the registration line.

   209 

   210 **Warnings**: the inprocess COM server support will not work when the client

   211 process already has Python loaded. It would be rather tricky to

   212 non-obtrusively hook into an already running Python, but the show-stopper is

   213 that the Python/C API won't let us find out which interpreter instance I should

   214 hook into. (If this is important to you, you might experiment with using

   215 apartment threading, which seems the best possibility to get this to work). To

   216 use a "frozen" COM server from a Python process, you'll have to load it as an

   217 exe::

   218 

   219       o = win32com.client.Dispatch(progid,

   220                        clsctx=pythoncom.CLSCTX_LOCAL_SERVER)

   221 

   222 

   223 MakeCOMServer also assumes that your top level code (registration etc.) is

   224 "normal". If it's not, you will have to edit the generated script.

   225 

   226 |GOBACK|

   227 

   228 

   229 Building Optimized

   230 ------------------

   231 

   232 There are two facets to running optimized: gathering ``.pyo``'s, and setting the

   233 ``Py_OptimizeFlag``. Installer will gather ``.pyo``'s if it is run optimized::

   234 

   235        python -O Build.py ...

   236 

   237 

   238 The ``Py_OptimizeFlag`` will be set if you use a ``('O','','OPTION')`` in one of

   239 the ``TOCs`` building the ``EXE``::

   240 

   241       exe = EXE(pyz,

   242                 a.scripts + [('O','','OPTION')],

   243                 ...

   244 

   245 See `Spec Files`_ for details.

   246 

   247 |GOBACK|

   248 

   249 

   250 A Note on using UPX

   251 -------------------

   252 

   253 On both Windows and Linux, UPX can give truly startling compression - the days

   254 of fitting something useful on a diskette are not gone forever! Installer has

   255 been tested with many UPX versions without problems. Just get it and install it

   256 on your PATH, then rerun configure.

   257 

   258 For Windows, there is a problem of compatibility between UPX and executables

   259 generated by Microsoft Visual Studio .NET 2003 (or the equivalent free

   260 toolkit available for download). This is especially worrisome for users of

   261 Python 2.4+, where most extensions (and Python itself) are compiled with that

   262 compiler. This issue has been fixed in later beta versions of UPX, so you

   263 will need at least UPX 1.92 beta. `Configure.py`_ will check this for you

   264 and complain if you have an older version of UPX and you are using Python 2.4.

   265 

   266 .. sidebar:: UPX and Unix

   267 

   268     Under UNIX, old versions of UPX were not able to expand and execute the

   269     executable in memory, and they were extracting it into a temporary file

   270     in the filesystem, before spawning it. This is no longer valid under Linux,

   271     but the information in this paragraph still needs to be updated.

   272 

   273 .. _`Configure.py`: `Configuring your PyInstaller setup`_

   274 

   275 For Linux, a bit more discussion is in order. First, UPX is only useful on

   276 executables, not shared libs. Installer accounts for that, but to get the full

   277 benefit, you might rebuild Python with more things statically linked.

   278 

   279 More importantly, when ``run`` finds that its ``sys.argv[0]`` does not contain a path,

   280 it will use ``/proc/pid/exe`` to find itself (if it can). This happens, for

   281 example, when executed by Apache. If it has been upx-ed, this symbolic link

   282 points to the tempfile created by the upx stub and |PyInstaller| will fail (please

   283 see the UPX docs for more information). So for now, at least, you can't use upx

   284 for CGI's executed by Apache. Otherwise, you can ignore the warnings in the UPX

   285 docs, since what PyInstaller opens is the executable Installer created, not the

   286 temporary upx-created executable.

   287 

   288 |GOBACK|

   289 

   290 A Note on ``--onefile``

   291 -----------------------

   292 

   293 A ``--onefile`` works by packing all the shared libs / dlls into the archive

   294 attached to the bootloader executable (or next to the executable in a non-elf

   295 configuration). When first started, it finds that it needs to extract these

   296 files before it can run "for real". That's because locating and loading a

   297 shared lib or linked-in dll is a system level action, not user-level. With

   298 |PyInstallerVersion| it always uses a temporary directory (``_MEIpid``) in the

   299 user's temp directory. It then executes itself again, setting things up so

   300 the system will be able to load the shared libs / dlls. When executing is

   301 complete, it recursively removes the entire directory it created.

   302 

   303 This has a number of implications:

   304 

   305 * You can run multiple copies - they won't collide.

   306 

   307 * Running multiple copies will be rather expensive to the system (nothing is

   308   shared).

   309 

   310 * If you're using the cheat of adding user data as ``'BINARY'``, it will be in

   311   ``os.environ['_MEIPASS2']``, not in the executable's directory.

   312 

   313 * On Windows, using Task Manager to kill the parent process will leave the

   314   directory behind.

   315 

   316 * On \*nix, a kill -9 (or crash) will leave the directory behind.

   317 

   318 * Otherwise, on both platforms, the directory will be recursively deleted.

   319 

   320 * So any files you might create in ``os.environ['_MEIPASS2']`` will be deleted.

   321 

   322 * The executable can be in a protected or read-only directory.

   323 

   324 * If for some reason, the ``_MEIpid`` directory already exists, the executable

   325   will fail. It is created mode 0700, so only the one user can modify it

   326   (on \*nix, of course).

   327 

   328 While we are not a security expert, we believe the scheme is good enough for

   329 most of the users.

   330 

   331 **Notes for \*nix users**: Take notice that if the executable does a setuid root,

   332 a determined hacker could possibly (given enough tries) introduce a malicious

   333 lookalike of one of the shared libraries during the hole between when the

   334 library is extracted and when it gets loaded by the execvp'd process. So maybe

   335 you shouldn't do setuid root programs using ``--onefile``. **In fact, we do not

   336 recomend the use of --onefile on setuid programs.**

   337 

   338 |GOBACK|

   339 

   340 A Note on .egg files and setuptools

   341 -----------------------------------

   342 `setuptools`_ is a distutils extensions which provide many benefits, including

   343 the ability to distribute the extension as ``egg`` files. Together with the

   344 nifty `easy_install`_ (a tool which automatically locates, downloads and

   345 installs Python extensions), ``egg`` files are becoming more and more

   346 widespread as a way for distributing Python extensions.

   347 

   348 ``egg`` files are actually ZIP files under the hood, and they rely on the fact

   349 that Python 2.4 is able to transparently import modules stored within ZIP

   350 files. PyInstaller is currently *not* able to import and extract modules

   351 within ZIP files, so code which uses extensions packaged as ``egg`` files

   352 cannot be packaged with PyInstaller.

   353 

   354 The workaround is pretty easy: you can use ``easy_install -Z`` at installation

   355 time to ask ``easy_install`` to always decompress egg files. This will allow

   356 PyInstaller to see the files and make the package correctly. If you have already

   357 installed the modules, you can simply decompress them within a directory with

   358 the same name of the ``egg`` file (including also the extension).

   359 

   360 Support for ``egg`` files is planned for a future release of PyInstaller.

   361 

   362 .. _`setuptools`: http://peak.telecommunity.com/DevCenter/setuptools

   363 .. _`easy_install`: http://peak.telecommunity.com/DevCenter/EasyInstall

   364 

   365 

   366 |GOBACK|

   367 

   368 

   369 PyInstaller Utilities

   370 +++++++++++++++++++++

   371 

   372 ArchiveViewer

   373 -------------

   374 

   375 ::

   376 

   377       python ArchiveViewer.py <archivefile>

   378 

   379 

   380 ArchiveViewer lets you examine the contents of any archive build with

   381 |PyInstaller| or executable (PYZ, PKG or exe). Invoke it with the target as the

   382 first arg (It has been set up as a Send-To so it shows on the context menu in

   383 Explorer). The archive can be navigated using these commands:

   384 

   385 O <nm>

   386     Open the embedded archive <nm> (will prompt if omitted).

   387 

   388 U

   389     Go up one level (go back to viewing the embedding archive).

   390 

   391 X <nm>

   392     Extract nm (will prompt if omitted). Prompts for output filename. If none

   393     given, extracted to stdout.

   394 

   395 Q

   396     Quit.

   397 

   398 

   399 |GOBACK|

   400 

   401 

   402 bindepend

   403 ---------

   404 

   405 ::

   406 

   407     python bindepend.py <executable_or_dynamic_library>

   408 

   409 bindepend will analyze the executable you pass to it, and write to stdout all

   410 its binary dependencies. This is handy to find out which DLLs are required by

   411 an executable or another DLL. This module is used by |PyInstaller| itself to

   412 follow the chain of dependencies of binary extensions and make sure that all

   413 of them get included in the final package.

   414 

   415 

   416 GrabVersion (Windows)

   417 ---------------------

   418 

   419 ::

   420 

   421       python GrabVersion.py <executable_with_version_resource>

   422 

   423 

   424 GrabVersion outputs text which can be eval'ed by ``versionInfo.py`` to reproduce

   425 a version resource. Invoke it with the full path name of a Windows executable

   426 (with a version resource) as the first argument. If you cut & paste (or

   427 redirect to a file), you can then edit the version information. The edited

   428 text file can be used in a ``version = myversion.txt`` option on any executable

   429 in an |PyInstaller| spec file.

   430 

   431 This was done in this way because version resources are rather strange beasts,

   432 and fully understanding them is probably impossible. Some elements are

   433 optional, others required, but you could spend unbounded amounts of time

   434 figuring this out, because it's not well documented. When you view the version

   435 tab on a properties dialog, there's no straightforward relationship between

   436 how the data is displayed and the structure of the resource itself. So the

   437 easiest thing to do is find an executable that displays the kind of

   438 information you want, grab it's resource and edit it. Certainly easier than

   439 the Version resource wizard in VC++.

   440 

   441 |GOBACK|

   442 

   443 

   444 Analyzing Dependencies

   445 ----------------------

   446 

   447 You can interactively track down dependencies, including getting

   448 cross-references by using ``mf.py``, documented in section `mf.py: A modulefinder

   449 Replacement`_

   450 

   451 |GOBACK|

   452 

   453 

   454 Spec Files

   455 ++++++++++

   456 

   457 Introduction

   458 ------------

   459 

   460 Spec files are in Python syntax. They are evaluated by Build.py. A simplistic

   461 spec file might look like this::

   462 

   463       a = Analysis(['myscript.py'])

   464       pyz = PYZ(a.pure)

   465       exe = EXE(pyz, a.scripts, a.binaries, name="myapp.exe")

   466 

   467 This creates a single file deployment with all binaries (extension modules and

   468 their dependencies) packed into the executable.

   469 

   470 A simplistic single directory deployment might look like this::

   471 

   472       a = Analysis(['myscript.py'])

   473       pyz = PYZ(a.pure)

   474       exe = EXE(a.scripts, pyz, name="myapp.exe", exclude_binaries=1)

   475       dist = COLLECT(exe, a.binaries, name="dist")

   476 

   477 

   478 Note that neither of these examples are realistic. Use ``Makespec.py`` (documented

   479 in section `Create a spec file for your project`_) to create your specfile,

   480 and tweak it (if necessary) from there.

   481 

   482 All of the classes you see above are subclasses of ``Build.Target``. A Target acts

   483 like a rule in a makefile. It knows enough to cache its last inputs and

   484 outputs. If its inputs haven't changed, it can assume its outputs wouldn't

   485 change on recomputation. So a spec file acts much like a makefile, only

   486 rebuilding as much as needs rebuilding. This means, for example, that if you

   487 change an ``EXE`` from ``debug=1`` to ``debug=0``, the rebuild will be nearly

   488 instantaneous.

   489 

   490 The high level view is that an ``Analysis`` takes a list of scripts as input,

   491 and generates three "outputs", held in attributes named ``scripts``, ``pure``

   492 and ``binaries``. A ``PYZ`` (a ``.pyz`` archive) is built from the modules in

   493 pure. The ``EXE`` is built from the ``PYZ``, the scripts and, in the case of a

   494 single-file deployment, the binaries. In a single-directory deployment, a

   495 directory is built containing a slim executable and the binaries.

   496 

   497 |GOBACK|

   498 

   499 TOC Class (Table of Contents)

   500 -----------------------------

   501 

   502 Before you can do much with a spec file, you need to understand the

   503 ``TOC`` (Table Of Contents) class.

   504 

   505 A ``TOC`` appears to be a list of tuples of the form (name, path, typecode).

   506 In fact, it's an ordered set, not a list. A TOC contains no duplicates, where

   507 uniqueness is based on name only. Furthermore, within this constraint, a TOC

   508 preserves order.

   509 

   510 Besides the normal list methods and operations, TOC supports taking differences

   511 and intersections (and note that adding or extending is really equivalent to

   512 union). Furthermore, the operations can take a real list of tuples on the right

   513 hand side. This makes excluding modules quite easy. For a pure Python module::

   514 

   515       pyz = PYZ(a.pure - [('badmodule', '', '')])

   516 

   517 

   518 or for an extension module in a single-directory deployment::

   519 

   520       dist = COLLECT(..., a.binaries - [('badmodule', '', '')], ...)

   521 

   522 

   523 or for a single-file deployment::

   524 

   525       exe = EXE(..., a.binaries - [('badmodule', '', '')], ...)

   526 

   527 To add files to a TOC, you need to know about the typecodes (or the step using

   528 the TOC won't know what to do with the entry).

   529 

   530 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   531 | **typecode** 	| **description**					| **name**		| **path**			|

   532 +===============+=======================================================+=======================+===============================+

   533 | 'EXTENSION' 	| An extension module.					| Python internal name.	| Full path name in build.	|

   534 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   535 | 'PYSOURCE'	| A script.						| Python internal name.	| Full path name in build.	|

   536 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   537 | 'PYMODULE'	| A pure Python module (including __init__ modules).	| Python internal name.	| Full path name in build.	|

   538 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   539 | 'PYZ'		| A .pyz archive (archive_rt.ZlibArchive).		| Runtime name.		| Full path name in build.	|

   540 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   541 | 'PKG'		| A pkg archive (carchive4.CArchive).			| Runtime name. 	| Full path name in build.	|

   542 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   543 | 'BINARY' 	| A shared library. 					| Runtime name. 	| Full path name in build.	|

   544 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   545 | 'DATA' 	| Aribitrary files. 					| Runtime name. 	| Full path name in build.	|

   546 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   547 | 'OPTION' 	| A runtime runtime option (frozen into the executable).| The option.		| Unused.			|

   548 +---------------+-------------------------------------------------------+-----------------------+-------------------------------+

   549 

   550 You can force the include of any file in much the same way you do excludes::

   551 

   552       collect = COLLECT(a.binaries +

   553                 [('readme', '/my/project/readme', 'DATA')], ...)

   554 

   555 

   556 or even::

   557 

   558       collect = COLLECT(a.binaries,

   559                 [('readme', '/my/project/readme', 'DATA')], ...)

   560 

   561 

   562 (that is, you can use a list of tuples in place of a ``TOC`` in most cases).

   563 

   564 There's not much reason to use this technique for ``PYSOURCE``, since an ``Analysis``

   565 takes a list of scripts as input. For ``PYMODULEs`` and ``EXTENSIONs``, the hook

   566 mechanism discussed here is better because you won't have to remember how you

   567 got it working next time.

   568 

   569 This technique is most useful for data files (see the ``Tree`` class below for a

   570 way to build a ``TOC`` from a directory tree), and for runtime options. The options

   571 the run executables understand are:

   572 

   573 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   574 | **Option**	| **Description**	| **Example**			| **Notes**												|

   575 +===============+=======================+===============================+=======================================================================================================+

   576 | v 		| Verbose imports	| ('v', '', 'OPTION')		| Same as Python -v ... 										|

   577 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   578 | u		| Unbuffered stdio	| ('u', '', 'OPTION')		| Same as Python -u ... 										|

   579 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   580 | W spec	| Warning option	| ('W ignore', '', 'OPTION')	| Python 2.1+ only. 											|

   581 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   582 | s		| Use site.py		| ('s', '', 'OPTION')		| The opposite of Python's -S flag. Note that site.py must be in the executable's directory to be used. |

   583 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   584 | f		| Force execvp		| ('f', '', 'OPTION')		| Linux/unix only. Ensures that LD_LIBRARY_PATH is set properly.					|

   585 +---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+

   586 

   587 Advanced users should note that by using set differences and intersections, it

   588 becomes possible to factor out common modules, and deploy a project containing

   589 multiple executables with minimal redundancy. You'll need some top level code

   590 in each executable to mount the common ``PYZ``.

   591 

   592 |GOBACK|

   593 

   594 Target Subclasses

   595 -----------------

   596 

   597 Analysis

   598 ********

   599 

   600 ::

   601 

   602       Analysis(scripts, pathex=None, hookspath=None, excludes=None)

   603 

   604 

   605 ``scripts``

   606     a list of scripts specified as file names.

   607 

   608 ``pathex``

   609     an optional list of paths to be searched before sys.path.

   610 

   611 ``hookspath``

   612     an optional list of paths used to extend the hooks package.

   613 

   614 ``excludes``

   615     an optional list of module or package names (their Python names, not path

   616     names) that will be ignored (as though they were not found).

   617 

   618 An Analysis has three outputs, all ``TOCs`` accessed as attributes of the ``Analysis``.

   619 

   620 ``scripts``

   621     The scripts you gave Analysis as input, with any runtime hook scripts

   622     prepended.

   623 

   624 ``pure``

   625     The pure Python modules.

   626 

   627 ``binaries``

   628     The extension modules and their dependencies. The secondary dependencies are

   629     filtered. On Windows, a long list of MS dlls are excluded. On Linux/Unix,

   630     any shared lib in ``/lib`` or ``/usr/lib`` is excluded.

   631 

   632 |GOBACK|

   633 

   634 PYZ

   635 ***

   636 

   637 ::

   638 

   639       PYZ(toc, name=None, level=9)

   640 

   641 

   642 ``toc``

   643     a ``TOC``, normally an ``Analysis.pure``.

   644 

   645 ``name``

   646     A filename for the ``.pyz``. Normally not needed, as the generated name will do fine.

   647 

   648 ``level``

   649     The Zlib compression level to use. If 0, the zlib module is not required.

   650 

   651 

   652 |GOBACK|

   653 

   654 PKG

   655 ***

   656 

   657 Generally, you will not need to create your own ``PKGs``, as the ``EXE`` will do it for

   658 you. This is one way to include read-only data in a single-file deployment,

   659 however. A single-file deployment including TK support will use this technique.

   660 

   661 ::

   662 

   663       PKG(toc, name=None, cdict=None, exclude_binaries=0)

   664 

   665 

   666 ``toc``

   667     a ``TOC``.

   668 

   669 ``name``

   670     a filename for the ``PKG`` (optional).

   671 

   672 ``cdict``

   673     a dictionary that specifies compression by typecode. For example, ``PYZ`` is

   674     left uncompressed so that it can be accessed inside the ``PKG``. The default

   675     uses sensible values. If zlib is not available, no compression is used.

   676 

   677 ``exclude_binaries``

   678     If 1, ``EXTENSIONs`` and ``BINARYs`` will be left out of the ``PKG``, and

   679     forwarded to its container (usually a ``COLLECT``).

   680 

   681 |GOBACK|

   682 

   683 EXE

   684 ***

   685 

   686 ::

   687 

   688       EXE(*args, **kws)

   689 

   690 

   691 ``args``

   692     One or more arguments which are either ``TOCs`` or ``Targets``.

   693 

   694 ``kws``

   695     Possible keyword arguments:

   696 

   697     ``console``

   698         Always 1 on Linux/unix. On Windows, governs whether to use the console

   699         executable, or the Windows subsystem executable.

   700 

   701     ``debug``

   702         Setting to 1 gives you progress messages from the executable (for a

   703         ``console=0``, these will be annoying MessageBoxes).

   704 

   705     ``name``

   706         The filename for the executable.

   707 

   708     ``exclude_binaries``

   709         Forwarded to the ``PKG`` the ``EXE`` builds.

   710 

   711     ``icon``

   712         Windows NT family only. ``icon='myicon.ico'`` to use an icon file, or

   713         ``icon='notepad.exe,0'`` to grab an icon resource.

   714 

   715     ``version``

   716         Windows NT family only. ``version='myversion.txt'``. Use ``GrabVersion.py`` to

   717         steal a version resource from an executable, and then edit the ouput to

   718         create your own. (The syntax of version resources is so arcane that I

   719         wouldn't attempt to write one from scratch.)

   720 

   721 

   722 There are actually two ``EXE`` classes - one for ELF platforms (where the

   723 bootloader, that is the ``run`` executable, and the ``PKG`` are concatenated),

   724 and one for non-ELF platforms (where the run executable is simply renamed, and

   725 expects a ``exename.pkg`` in the same directory). Which class becomes available

   726 as ``EXE`` is determined by a flag in ``config.dat``. This flag is set to

   727 non-ELF when using ``Make.py -n``.

   728 

   729 |GOBACK|

   730 

   731 DLL

   732 ***

   733 

   734 On Windows, this provides support for doing in-process COM servers. It is not

   735 generalized. However, embedders can follow the same model to build a special

   736 purpose DLL so the Python support in their app is hidden. You will need to

   737 write your own dll, but thanks to Allan Green for refactoring the C code and

   738 making that a managable task.

   739 

   740 |GOBACK|

   741 

   742 COLLECT

   743 *******

   744 

   745 ::

   746 

   747       COLLECT(*args, **kws)

   748 

   749 

   750 ``args``

   751     One or more arguments which are either ``TOCs`` or ``Targets``.

   752 

   753 ``kws``

   754     Possible keyword arguments:

   755 

   756     ``name``

   757         The name of the directory to be built.

   758 

   759 |GOBACK|

   760 

   761 Tree

   762 ****

   763 

   764 ::

   765 

   766       Tree(root, prefix=None, excludes=None)

   767 

   768 

   769 ``root``

   770     The root of the tree (on the build system).

   771 

   772 ``prefix``

   773     Optional prefix to the names on the target system.

   774 

   775 ``excludes``

   776     A list of names to exclude. Two forms are allowed:

   777 

   778     ``name``

   779         files with this basename will be excluded (do not include the path).

   780 

   781     ``*.ext``

   782         any file with the given extension will be excluded.

   783 

   784 Since a ``Tree`` is a ``TOC``, you can also use the exclude technique described above

   785 in the section on ``TOCs``.

   786 

   787 

   788 |GOBACK|

   789 

   790 When Things Go Wrong

   791 ++++++++++++++++++++

   792 

   793 Finding out What Went Wrong

   794 ---------------------------

   795 

   796 Buildtime Warnings

   797 ******************

   798 

   799 When an ``Analysis`` step runs, it produces a warnings file (named ``warnproject.txt``)

   800 in the spec file's directory. Generally, most of these warnings are harmless.

   801 For example, ``os.py`` (which is cross-platform) works by figuring out what

   802 platform it is on, then importing (and rebinding names from) the appropriate

   803 platform-specific module. So analyzing ``os.py`` will produce a set of warnings

   804 like::

   805 

   806       W: no module named dos (conditional import by os)

   807       W: no module named ce (conditional import by os)

   808       W: no module named os2 (conditional import by os)

   809 

   810 

   811 Note that the analysis has detected that the import is within a conditional

   812 block (an if statement). The analysis also detects if an import within a

   813 function or class, (delayed) or at the top level. A top-level, non-conditional

   814 import failure is really a hard error. There's at least a reasonable chance

   815 that conditional and / or delayed import will be handled gracefully at runtime.

   816 

   817 Ignorable warnings may also be produced when a class or function is declared in

   818 a package (an ``__init__.py`` module), and the import specifies

   819 ``package.name``. In this case, the analysis can't tell if name is supposed to

   820 refer to a submodule of package.

   821 

   822 Warnings are also produced when an ``__import__``, ``exec`` or ``eval`` statement is

   823 encountered. The ``__import__`` warnings should almost certainly be investigated.

   824 Both ``exec`` and ``eval`` can be used to implement import hacks, but usually their use

   825 is more benign.

   826 

   827 Any problem detected here can be handled by hooking the analysis of the module.

   828 See `Listing Hidden Imports`_ below for how to do it.

   829 

   830 |GOBACK|

   831 

   832 Getting Debug Messages

   833 **********************

   834 

   835 Setting ``debug=1`` on an ``EXE`` will cause the executable to put out progress

   836 messages (for console apps, these go to stdout; for Windows apps, these show as

   837 MessageBoxes). This can be useful if you are doing complex packaging, or your

   838 app doesn't seem to be starting, or just to learn how the runtime works.

   839 

   840 |GOBACK|

   841 

   842 Getting Python's Verbose Imports

   843 ********************************

   844 

   845 You can also pass a ``-v`` (verbose imports) flag to the embedded Python. This can

   846 be extremely useful. I usually try it even on apparently working apps, just to

   847 make sure that I'm always getting my copies of the modules and no import has

   848 leaked out to the installed Python.

   849 

   850 You set this (like the other runtime options) by feeding a phone ``TOC`` entry to

   851 the ``EXE``. The easiest way to do this is to change the ``EXE`` from::

   852 

   853        EXE(..., anal.scripts, ....)

   854 

   855 to::

   856 

   857        EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)

   858 

   859 These messages will always go to ``stdout``, so you won't see them on Windows if

   860 ``console=0``.

   861 

   862 |GOBACK|

   863 

   864 Helping Installer Find Modules

   865 ------------------------------

   866 

   867 Extending the Path

   868 ******************

   869 

   870 When the analysis phase cannot find needed modules, it may be that the code is

   871 manipulating ``sys.path``. The easiest thing to do in this case is tell ``Analysis``

   872 about the new directory through the second arg to the constructor::

   873 

   874        anal = Analysis(['somedir/myscript.py'],

   875                        ['path/to/thisdir', 'path/to/thatdir'])

   876 

   877 

   878 In this case, the ``Analysis`` will have a search path::

   879 

   880        ['somedir', 'path/to/thisdir', 'path/to/thatdir'] + sys.path

   881 

   882 

   883 You can do the same when running ``Makespec.py``::

   884 

   885        Makespec.py --paths=path/to/thisdir;path/to/thatdir ...

   886 

   887 

   888 (on \*nix, use ``:`` as the path separator).

   889 

   890 |GOBACK|

   891 

   892 Listing Hidden Imports

   893 **********************

   894 

   895 Hidden imports are fairly common. These can occur when the code is using

   896 ``__import__`` (or, perhaps ``exec`` or ``eval``), in which case you will see a warning in

   897 the ``warnproject.txt`` file. They can also occur when an extension module uses the

   898 Python/C API to do an import, in which case Analysis can't detect anything. You

   899 can verify that hidden import is the problem by using Python's verbose imports

   900 flag. If the import messages say "module not found", but the ``warnproject.txt``

   901 file has no "no module named..." message for the same module, then the problem

   902 is a hidden import.

   903 

   904 .. sidebar:: Standard hidden imports are already included!

   905 

   906     If you are getting worried while reading this paragraph, do not worry:

   907     having hidden imports is the exception, not the norm! And anyway,

   908     PyInstaller already ships with a large set of hooks that take care of

   909     hidden imports for the most common packages out there. For instance,

   910     PIL_, PyWin32_, PyQt_ are already taken care of.

   911 

   912 Hidden imports are handled by hooking the module (the one doing the hidden

   913 imports) at ``Analysis`` time. Do this by creating a file named ``hook-module.py``

   914 (where module is the fully-qualified Python name, eg, ``hook-xml.dom.py``), and

   915 placing it in the ``hooks`` package under |PyInstaller|'s root directory,

   916 (alternatively, you can save it elsewhere, and then use the ``hookspath`` arg to

   917 ``Analysis`` so your private hooks directory will be searched). Normally, it will

   918 have only one line::

   919 

   920       hiddenimports = ['module1', 'module2']

   921 

   922 When the ``Analysis`` finds this file, it will proceed exactly as though the module

   923 explicitly imported ``module1`` and ``module2``. (Full details on the analysis-time

   924 hook mechanism is in the `Hooks`_ section).

   925 

   926 If you successfully hook a publicly distributed module in this way, please send

   927 us the hook so we can make it available to others.

   928 

   929 |GOBACK|

   930 

   931 Extending a Package's ``__path__``

   932 **********************************

   933 

   934 Python allows a package to extend the search path used to find modules and

   935 sub-packages through the ``__path__`` mechanism. Normally, a package's ``__path__`` has

   936 only one entry - the directory in which the ``__init__.py`` was found. But

   937 ``__init__.py`` is free to extend its ``__path__`` to include other directories. For

   938 example, the ``win32com.shell.shell`` module actually resolves to

   939 ``win32com/win32comext/shell/shell.pyd``. This is because ``win32com/__init__.py``

   940 appends ``../win32comext`` to its ``__path__``.

   941 

   942 Because the ``__init__.py`` is not actually run during an analysis, we use the same

   943 hook mechanism we use for hidden imports. A static list of names won't do,

   944 however, because the new entry on ``__path__`` may well require computation. So

   945 ``hook-module.py`` should define a method ``hook(mod)``. The mod argument is an

   946 instance of ``mf.Module`` which has (more or less) the same attributes as a real

   947 module object. The hook function should return a ``mf.Module`` instance - perhaps

   948 a brand new one, but more likely the same one used as an arg, but mutated.

   949 See `mf.py: A Modulefinder Replacement`_ for details, and `hooks\/hook-win32com.py`_

   950 for an example.

   951 

   952 Note that manipulations of ``__path__`` hooked in this way apply to the analysis,

   953 and only the analysis. That is, at runtime ``win32com.shell`` is resolved the same

   954 way as ``win32com.anythingelse``, and ``win32com.__path__`` knows nothing of ``../win32comext``.

   955 

   956 Once in awhile, that's not enough.

   957 

   958 |GOBACK|

   959 

   960 Changing Runtime Behavior

   961 *************************

   962 

   963 More bizarre situations can be accomodated with runtime hooks. These are small

   964 scripts that manipulate the environment before your main script runs,

   965 effectively providing additional top-level code to your script.

   966 

   967 At the tail end of an analysis, the module list is examined for matches in

   968 ``rthooks.dat``, which is the string representation of a Python dictionary. The

   969 key is the module name, and the value is a list of hook-script pathnames.

   970 

   971 So putting an entry::

   972 

   973        'somemodule': ['path/to/somescript.py'],

   974 

   975 into ``rthooks.dat`` is almost the same thing as doing this::

   976 

   977        anal = Analysis(['path/to/somescript.py', 'main.py'], ...

   978 

   979 

   980 except that in using the hook, ``path/to/somescript.py`` will not be analyzed,

   981 (that's not a feature - we just haven't found a sane way fit the recursion into

   982 my persistence scheme).

   983 

   984 Hooks done in this way, while they need to be careful of what they import, are

   985 free to do almost anything. One provided hook sets things up so that win32com

   986 can generate modules at runtime (to disk), and the generated modules can be

   987 found in the win32com package.

   988 

   989 |GOBACK|

   990 

   991 Adapting to being "frozen"

   992 **************************

   993 

   994 In most sophisticated apps, it becomes necessary to figure out (at runtime)

   995 whether you're running "live" or "frozen". For example, you might have a

   996 configuration file that (running "live") you locate based on a module's

   997 ``__file__`` attribute. That won't work once the code is packaged up. You'll

   998 probably want to look for it based on ``sys.executable`` instead.

   999 

  1000 The bootloaders set ``sys.frozen=1`` (and, for in-process COM servers, the

  1001 embedding DLL sets ``sys.frozen='dll'``).

  1002 

  1003 For really advanced users, you can access the ``iu.ImportManager`` as

  1004 ``sys.importManager``. See `iu.py`_ for how you might make use of this fact.

  1005 

  1006 |GOBACK|

  1007 

  1008 Accessing Data Files

  1009 ********************

  1010 

  1011 In a ``--onedir`` distribution, this is easy: pass a list of your data files

  1012 (in ``TOC`` format) to the ``COLLECT``, and they will show up in the distribution

  1013 directory tree. The name in the ``(name, path, 'DATA')`` tuple can be a relative

  1014 path name. Then, at runtime, you can use code like this to find the file::

  1015 

  1016        os.path.join(os.path.dirname(sys.executable), relativename))

  1017 

  1018 

  1019 In a ``--onefile``, it's a bit trickier. You can cheat, and add the files to the

  1020 ``EXE`` as ``BINARY``. They will then be extracted at runtime into the work directory

  1021 by the C code (which does not create directories, so the name must be a plain

  1022 name), and cleaned up on exit. The work directory is best found by

  1023 ``os.environ['_MEIPASS2']``. Be awawre, though, that if you use ``--strip`` or ``--upx``,

  1024 strange things may happen to your data - ``BINARY`` is really for shared

  1025 libs / dlls.

  1026 

  1027 If you add them as ``'DATA'`` to the ``EXE``, then it's up to you to extract them. Use

  1028 code like this::

  1029 

  1030        import sys, carchive

  1031        this = carchive.CArchive(sys.executable)

  1032        data = this.extract('mystuff')[1]

  1033 

  1034 

  1035 to get the contents as a binary string. See `support\/unpackTK.py`_ for an advanced

  1036 example (the TCL and TK lib files are in a PKG which is opened in place, and

  1037 then extracted to the filesystem).

  1038 

  1039 |GOBACK|

  1040 

  1041 Miscellaneous

  1042 +++++++++++++

  1043 

  1044 Pmw -- Python Mega Widgets

  1045 --------------------------

  1046 

  1047 `Pmw`_ comes with a script named ``bundlepmw`` in the bin directory. If you follow the

  1048 instructions in that script, you'll end up with a module named ``Pmw.py``. Ensure

  1049 that Builder finds that module and not the development package.

  1050 

  1051 |GOBACK|

  1052 

  1053 Win9xpopen

  1054 ----------

  1055 

  1056 If you're using popen on Windows and want the code to work on Win9x, you'll

  1057 need to distribute ``win9xpopen.exe`` with your app. On older Pythons with

  1058 Win32all, this would apply to Win32pipe and ``win32popenWin9x.exe``. (On yet older

  1059 Pythons, no form of popen worked on Win9x).

  1060 

  1061 |GOBACK|

  1062 

  1063 Self-extracting executables

  1064 ---------------------------

  1065 

  1066 The ELF executable format (Windows, Linux and some others) allows arbitrary

  1067 data to be concatenated to the end of the executable without disturbing its

  1068 functionality. For this reason, a ``CArchive``'s Table of Contents is at the end of

  1069 the archive. The executable can open itself as a binary file name, seek to the

  1070 end and 'open' the ``CArchive`` (see figure 3).

  1071 

  1072 On other platforms, the archive and the executable are separate, but the

  1073 archive is named ``executable.pkg``, and expected to be in the same directory.

  1074 Other than that, the process is the same.

  1075 

  1076 |GOBACK|

  1077 

  1078 One Pass Execution

  1079 ******************

  1080 

  1081 In a single directory deployment (``--onedir``, which is the default), all of the

  1082 binaries are already in the file system. In that case, the embedding app:

  1083 

  1084 * opens the archive

  1085 

  1086 * starts Python (on Windows, this is done with dynamic loading so one embedding

  1087   app binary can be used with any Python version)

  1088 

  1089 * imports all the modules which are at the top level of the archive (basically,

  1090   bootstraps the import hooks)

  1091 

  1092 * mounts the ``ZlibArchive(s)`` in the outer archive

  1093 

  1094 * runs all the scripts which are at the top level of the archive

  1095 

  1096 * finalizes Python

  1097 

  1098 |GOBACK|

  1099 

  1100 Two Pass Execution

  1101 ******************

  1102 

  1103 There are a couple situations which require two passes:

  1104 

  1105 * a ``--onefile`` deployment (on Windows, the files can't be cleaned up afterwards

  1106   because Python does not call ``FreeLibrary``; on other platforms, Python won't

  1107   find them if they're extracted in the same process that uses them)

  1108 

  1109 * ``LD_LIBRARY_PATH`` needs to be set to find the binaries (not extension modules,

  1110   but modules the extensions are linked to).

  1111 

  1112 The first pass:

  1113 

  1114 * opens the archive

  1115 

  1116 * extracts all the binaries in the archive (in |PyInstallerVersion|, this is always to a

  1117   temporary directory).

  1118 

  1119 * sets a magic environment variable

  1120 

  1121 * sets ``LD_LIBRARY_PATH`` (non-Windows)

  1122 

  1123 * executes itself as a child process (letting the child use his stdin, stdout

  1124   and stderr)

  1125 

  1126 * waits for the child to exit (on \*nix, the child actually replaces the parent)

  1127 

  1128 * cleans up the extracted binaries (so on \*nix, this is done by the child)

  1129 

  1130 The child process executes as in `One Pass Execution`_ above (the magic

  1131 environment variable is what tells it that this is pass two).

  1132 

  1133 |SE_exeImage| figure 3 - Self Extracting Executable

  1134 

  1135 There are, of course, quite a few differences between the Windows and