==================

PyInstaller Manual

==================

:Author: William Caban (based on Gordon McMillan's manual)

:Contact: william@hpcf.upr.edu

:Revision: $Rev: 257 $

:Source URL: $HeadURL: http://svn.pyinstaller.python-hosting.com/trunk/doc/source/Manual.rst $

:Copyright: This document has been placed in the public domain.

.. contents::

Getting Started

+++++++++++++++

Installing PyInstaller

----------------------

First, unpack the archive on you path of choice. Installer is **not** a Python

package, so it doesn't need to go in site-packages, or have a .pth file. For

the purpose of this documentation we will assume |install_path|. You will be

using a couple of scripts in the |install_path| directory, and these will find

everything they need from their own location. For convenience, keep the paths

to these scripts short (don't install in a deeply nested subdirectory).

|PyInstaller| is dependant to the version of python you configure it for. In

other words, you will need a separate copy of |PyInstaller| for each Python

version you wish to work with *or* you'll need to rerun ``Configure.py`` every

time you switch the Python version).

|GOBACK|

Building the runtime executables

--------------------------------

*Note:* Windows users can skip this step, because all of Python is contained in

pythonXX.dll, and |PyInstaller| will use your pythonXX.dll.

On Linux the first thing to do is build the runtime executables.

Change to the |install_path| ``source/linux`` subdirectory. Run ``Make.py

[-n|-e]`` and then make. This will produce ``support/loader/run`` and

``support/loader/run_d``, which are the bootloaders.

.. sidebar:: Bootloader

   The bootloader (also known as *stub* in literature) is the small program

   which starts up your packaged program. Usually, the archive containing the

   bytecoded modules of your program is simply attended to it. See

   `Self-extracting executables`_ for more details on the process.

*Note:* If you have multiple versions of Python, the Python you use to run

``Make.py`` is the one whose configuration is used.

The ``-n`` and ``-e`` options set a non-elf or elf flag in your ``config.dat``.

As of |InitialVersion|, the executable will try both strategies, and this flag

just sets how you want your executables built. In the elf strategy, the archive

is concatenated to the executable. In the non-elf strategy, the executable

expects an archive with the same name as itself in the executable's directory.

Note that the executable chases down symbolic links before determining it's name

and directory, so putting the archive in the same directory as the symbolic link

will not work.

Windows distributions come with several executables in the ``support/loader``

directory: ``run_*.exe`` (bootloader for regular programs), and

``inprocsrvr_*.dll`` (bootloader for in-process COM servers). To rebuild this,

you need to install Scons_, and then just run ``scons`` from the |install_path|

directory.

|GOBACK|

Configuring your PyInstaller setup

----------------------------------

In the |install_path| directory, run ``Configure.py``. This saves some

information into ``config.dat`` that would otherwise be recomputed every time.

It can be rerun at any time if your configuration changes. It must be run before

trying to build anything.

|GOBACK|

Create a spec file for your project

-----------------------------------

[For Windows COM server support, see section `Windows COM Server Support`_]

The root directory has a script Makespec.py for this purpose::

       python Makespec.py [opts] <scriptname> [<scriptname> ...]

Where allowed OPTIONS are:

-F, --onefile

    produce a single file deployment (see below).

-D, --onedir

    produce a single directory deployment (default).

-K, --tk

    include TCL/TK in the deployment.

-a, --ascii

    do not include encodings. The default (on Python versions with unicode

    support) is now to include all encodings.

-d, --debug

    use debug (verbose) versions of the executables.

-w, --windowed, --noconsole

    Use the Windows subsystem executable, which does not open

    the console when the program is launched. **(Windows only)**

-c, --nowindowed, --console

    Use the console subsystem executable. This is the default. **(Windows only)**

-s, --strip

    the executable and all shared libraries will be run through strip. Note

    that cygwin's strip tends to render normal Win32 dlls unusable.

-X, --upx

    if you have UPX installed (detected by Configure), this will use it to

    compress your executable (and, on Windows, your dlls). See note below.

-o DIR, --out=DIR

    create the spec file in *directory*. If not specified, and the current

    directory is Installer's root directory, an output subdirectory will be

    created. Otherwise the current directory is used.

-p DIR, --paths=DIR

    set base path for import (like using PYTHONPATH). Multiple directories are

    allowed, separating them with the path separator (';' under Windows, ':'

    under Linux), or using this option multiple times.

--icon=<FILE.ICO>

    add *file.ico* to the executable's resources. **(Windows only)**

--icon=<FILE.EXE,N>

    add the *n*-th incon in *file.exe* to the executable's resources. **(Windows

    only)**

-v FILE, --version=FILE

    add verfile as a version resource to the executable. **(Windows only)**

-n NAME, --name=NAME

    optional *name* to assign to the project (from which the spec file name is

    generated). If omitted, the basename of the (first) script is used.

[For building with optimization on (like ``Python -O``), see section

`Building Optimized`_]

For simple projects, the generated spec file will probably be sufficient. For

more complex projects, it should be regarded as a template. The spec file is

actually Python code, and modifying it should be ease. See `Spec Files`_ for

details.

|GOBACK|

Build your project

------------------

::

      python Build.py specfile

A ``buildproject`` subdirectory will be created in the specfile's directory. This

is a private workspace so that ``Build.py`` can act like a makefile. Any named

targets will appear in the specfile's directory. For ``--onedir``

configurations, it will create also ``distproject``, which is the directory you're

interested in. For a ``--onefile``, the executable will be in the specfile's

directory.

In most cases, this will be all you have to do. If not, see `When things go

wrong`_ and be sure to read the introduction to `Spec Files`_.

|GOBACK|

Windows COM Server support

--------------------------

For Windows COM support execute::

       python MakeCOMServer.py [OPTION] script...

This will generate a new script ``drivescript.py`` and a spec file for the script.

These options are allowed:

--debug

    Use the verbose version of the executable.

--verbose

    Register the COM server(s) with the quiet flag off.

--ascii

    do not include encodings (this is passed through to Makespec).

--out <dir>

    Generate the driver script and spec file in dir.

Now `Build your project`_ on the generated spec file.

If you have the win32dbg package installed, you can use it with the generated

COM server. In the driver script, set ``debug=1`` in the registration line.

**Warnings**: the inprocess COM server support will not work when the client

process already has Python loaded. It would be rather tricky to

non-obtrusively hook into an already running Python, but the show-stopper is

that the Python/C API won't let us find out which interpreter instance I should

hook into. (If this is important to you, you might experiment with using

apartment threading, which seems the best possibility to get this to work). To

use a "frozen" COM server from a Python process, you'll have to load it as an

exe::

      o = win32com.client.Dispatch(progid,

                       clsctx=pythoncom.CLSCTX_LOCAL_SERVER)

MakeCOMServer also assumes that your top level code (registration etc.) is

"normal". If it's not, you will have to edit the generated script.

|GOBACK|

Building Optimized

------------------

There are two facets to running optimized: gathering ``.pyo``'s, and setting the

``Py_OptimizeFlag``. Installer will gather ``.pyo``'s if it is run optimized::

       python -O Build.py ...

The ``Py_OptimizeFlag`` will be set if you use a ``('O','','OPTION')`` in one of

the ``TOCs`` building the ``EXE``::

      exe = EXE(pyz,

                a.scripts + [('O','','OPTION')],

                ...

See `Spec Files`_ for details.

|GOBACK|

A Note on using UPX

-------------------

On both Windows and Linux, UPX can give truly startling compression - the days

of fitting something useful on a diskette are not gone forever! Installer has

been tested with many UPX versions without problems. Just get it and install it

on your PATH, then rerun configure.

For Windows, there is a problem of compatibility between UPX and executables

generated by Microsoft Visual Studio .NET 2003 (or the equivalent free

toolkit available for download). This is especially worrisome for users of

Python 2.4+, where most extensions (and Python itself) are compiled with that

compiler. This issue has been fixed in later beta versions of UPX, so you

will need at least UPX 1.92 beta. `Configure.py`_ will check this for you

and complain if you have an older version of UPX and you are using Python 2.4.

.. sidebar:: UPX and Unix

    Under UNIX, old versions of UPX were not able to expand and execute the

    executable in memory, and they were extracting it into a temporary file

    in the filesystem, before spawning it. This is no longer valid under Linux,

    but the information in this paragraph still needs to be updated.

.. _`Configure.py`: `Configuring your PyInstaller setup`_

For Linux, a bit more discussion is in order. First, UPX is only useful on

executables, not shared libs. Installer accounts for that, but to get the full

benefit, you might rebuild Python with more things statically linked.

More importantly, when ``run`` finds that its ``sys.argv[0]`` does not contain a path,

it will use ``/proc/pid/exe`` to find itself (if it can). This happens, for

example, when executed by Apache. If it has been upx-ed, this symbolic link

points to the tempfile created by the upx stub and |PyInstaller| will fail (please

see the UPX docs for more information). So for now, at least, you can't use upx

for CGI's executed by Apache. Otherwise, you can ignore the warnings in the UPX

docs, since what PyInstaller opens is the executable Installer created, not the

temporary upx-created executable.

|GOBACK|

A Note on ``--onefile``

-----------------------

A ``--onefile`` works by packing all the shared libs / dlls into the archive

attached to the bootloader executable (or next to the executable in a non-elf

configuration). When first started, it finds that it needs to extract these

files before it can run "for real". That's because locating and loading a

shared lib or linked-in dll is a system level action, not user-level. With

|PyInstallerVersion| it always uses a temporary directory (``_MEIpid``) in the

user's temp directory. It then executes itself again, setting things up so

the system will be able to load the shared libs / dlls. When executing is

complete, it recursively removes the entire directory it created.

This has a number of implications:

* You can run multiple copies - they won't collide.

* Running multiple copies will be rather expensive to the system (nothing is

  shared).

* If you're using the cheat of adding user data as ``'BINARY'``, it will be in

  ``os.environ['_MEIPASS2']``, not in the executable's directory.

* On Windows, using Task Manager to kill the parent process will leave the

  directory behind.

* On \*nix, a kill -9 (or crash) will leave the directory behind.

* Otherwise, on both platforms, the directory will be recursively deleted.

* So any files you might create in ``os.environ['_MEIPASS2']`` will be deleted.

* The executable can be in a protected or read-only directory.

* If for some reason, the ``_MEIpid`` directory already exists, the executable

  will fail. It is created mode 0700, so only the one user can modify it

  (on \*nix, of course).

While we are not a security expert, we believe the scheme is good enough for

most of the users.

**Notes for \*nix users**: Take notice that if the executable does a setuid root,

a determined hacker could possibly (given enough tries) introduce a malicious

lookalike of one of the shared libraries during the hole between when the

library is extracted and when it gets loaded by the execvp'd process. So maybe

you shouldn't do setuid root programs using ``--onefile``. **In fact, we do not

recomend the use of --onefile on setuid programs.**

|GOBACK|

A Note on .egg files and setuptools

-----------------------------------

`setuptools`_ is a distutils extensions which provide many benefits, including

the ability to distribute the extension as ``egg`` files. Together with the

nifty `easy_install`_ (a tool which automatically locates, downloads and

installs Python extensions), ``egg`` files are becoming more and more

widespread as a way for distributing Python extensions.

``egg`` files are actually ZIP files under the hood, and they rely on the fact

that Python 2.4 is able to transparently import modules stored within ZIP

files. PyInstaller is currently *not* able to import and extract modules

within ZIP files, so code which uses extensions packaged as ``egg`` files

cannot be packaged with PyInstaller.

The workaround is pretty easy: you can use ``easy_install -Z`` at installation

time to ask ``easy_install`` to always decompress egg files. This will allow

PyInstaller to see the files and make the package correctly. If you have already

installed the modules, you can simply decompress them within a directory with

the same name of the ``egg`` file (including also the extension).

Support for ``egg`` files is planned for a future release of PyInstaller.

.. _`setuptools`: http://peak.telecommunity.com/DevCenter/setuptools

.. _`easy_install`: http://peak.telecommunity.com/DevCenter/EasyInstall

|GOBACK|

PyInstaller Utilities

+++++++++++++++++++++

ArchiveViewer

-------------

::

      python ArchiveViewer.py <archivefile>

ArchiveViewer lets you examine the contents of any archive build with

|PyInstaller| or executable (PYZ, PKG or exe). Invoke it with the target as the

first arg (It has been set up as a Send-To so it shows on the context menu in

Explorer). The archive can be navigated using these commands:

O <nm>

    Open the embedded archive <nm> (will prompt if omitted).

U

    Go up one level (go back to viewing the embedding archive).

X <nm>

    Extract nm (will prompt if omitted). Prompts for output filename. If none

    given, extracted to stdout.

Q

    Quit.

|GOBACK|

bindepend

---------

::

    python bindepend.py <executable_or_dynamic_library>

bindepend will analyze the executable you pass to it, and write to stdout all

its binary dependencies. This is handy to find out which DLLs are required by

an executable or another DLL. This module is used by |PyInstaller| itself to

follow the chain of dependencies of binary extensions and make sure that all

of them get included in the final package.

GrabVersion (Windows)

---------------------

::

      python GrabVersion.py <executable_with_version_resource>

GrabVersion outputs text which can be eval'ed by ``versionInfo.py`` to reproduce

a version resource. Invoke it with the full path name of a Windows executable

(with a version resource) as the first argument. If you cut & paste (or

redirect to a file), you can then edit the version information. The edited

text file can be used in a ``version = myversion.txt`` option on any executable

in an |PyInstaller| spec file.

This was done in this way because version resources are rather strange beasts,

and fully understanding them is probably impossible. Some elements are

optional, others required, but you could spend unbounded amounts of time

figuring this out, because it's not well documented. When you view the version

tab on a properties dialog, there's no straightforward relationship between

how the data is displayed and the structure of the resource itself. So the

easiest thing to do is find an executable that displays the kind of

information you want, grab it's resource and edit it. Certainly easier than

the Version resource wizard in VC++.

|GOBACK|

Analyzing Dependencies

----------------------

You can interactively track down dependencies, including getting

cross-references by using ``mf.py``, documented in section `mf.py: A modulefinder

Replacement`_

|GOBACK|

Spec Files

++++++++++

Introduction

------------

Spec files are in Python syntax. They are evaluated by Build.py. A simplistic

spec file might look like this::

      a = Analysis(['myscript.py'])

      pyz = PYZ(a.pure)

      exe = EXE(pyz, a.scripts, a.binaries, name="myapp.exe")

This creates a single file deployment with all binaries (extension modules and

their dependencies) packed into the executable.

A simplistic single directory deployment might look like this::

      a = Analysis(['myscript.py'])

      pyz = PYZ(a.pure)

      exe = EXE(a.scripts, pyz, name="myapp.exe", exclude_binaries=1)

      dist = COLLECT(exe, a.binaries, name="dist")

Note that neither of these examples are realistic. Use ``Makespec.py`` (documented

in section `Create a spec file for your project`_) to create your specfile,

and tweak it (if necessary) from there.

All of the classes you see above are subclasses of ``Build.Target``. A Target acts

like a rule in a makefile. It knows enough to cache its last inputs and

outputs. If its inputs haven't changed, it can assume its outputs wouldn't

change on recomputation. So a spec file acts much like a makefile, only

rebuilding as much as needs rebuilding. This means, for example, that if you

change an ``EXE`` from ``debug=1`` to ``debug=0``, the rebuild will be nearly

instantaneous.

The high level view is that an ``Analysis`` takes a list of scripts as input,

and generates three "outputs", held in attributes named ``scripts``, ``pure``

and ``binaries``. A ``PYZ`` (a ``.pyz`` archive) is built from the modules in

pure. The ``EXE`` is built from the ``PYZ``, the scripts and, in the case of a

single-file deployment, the binaries. In a single-directory deployment, a

directory is built containing a slim executable and the binaries.

|GOBACK|

TOC Class (Table of Contents)

-----------------------------

Before you can do much with a spec file, you need to understand the

``TOC`` (Table Of Contents) class.

A ``TOC`` appears to be a list of tuples of the form (name, path, typecode).

In fact, it's an ordered set, not a list. A TOC contains no duplicates, where

uniqueness is based on name only. Furthermore, within this constraint, a TOC

preserves order.

Besides the normal list methods and operations, TOC supports taking differences

and intersections (and note that adding or extending is really equivalent to

union). Furthermore, the operations can take a real list of tuples on the right

hand side. This makes excluding modules quite easy. For a pure Python module::

      pyz = PYZ(a.pure - [('badmodule', '', '')])

or for an extension module in a single-directory deployment::

      dist = COLLECT(..., a.binaries - [('badmodule', '', '')], ...)

or for a single-file deployment::

      exe = EXE(..., a.binaries - [('badmodule', '', '')], ...)

To add files to a TOC, you need to know about the typecodes (or the step using

the TOC won't know what to do with the entry).