1
|
1 |
==================
|
|
2 |
PyInstaller Manual
|
|
3 |
==================
|
|
4 |
:Author: William Caban (based on Gordon McMillan's manual)
|
|
5 |
:Contact: william@hpcf.upr.edu
|
|
6 |
:Revision: $Rev: 257 $
|
|
7 |
:Source URL: $HeadURL: http://svn.pyinstaller.python-hosting.com/trunk/doc/source/Manual.rst $
|
|
8 |
:Copyright: This document has been placed in the public domain.
|
|
9 |
|
|
10 |
.. contents::
|
|
11 |
|
|
12 |
|
|
13 |
Getting Started
|
|
14 |
+++++++++++++++
|
|
15 |
|
|
16 |
Installing PyInstaller
|
|
17 |
----------------------
|
|
18 |
|
|
19 |
First, unpack the archive on you path of choice. Installer is **not** a Python
|
|
20 |
package, so it doesn't need to go in site-packages, or have a .pth file. For
|
|
21 |
the purpose of this documentation we will assume |install_path|. You will be
|
|
22 |
using a couple of scripts in the |install_path| directory, and these will find
|
|
23 |
everything they need from their own location. For convenience, keep the paths
|
|
24 |
to these scripts short (don't install in a deeply nested subdirectory).
|
|
25 |
|
|
26 |
|PyInstaller| is dependant to the version of python you configure it for. In
|
|
27 |
other words, you will need a separate copy of |PyInstaller| for each Python
|
|
28 |
version you wish to work with *or* you'll need to rerun ``Configure.py`` every
|
|
29 |
time you switch the Python version).
|
|
30 |
|
|
31 |
|GOBACK|
|
|
32 |
|
|
33 |
|
|
34 |
Building the runtime executables
|
|
35 |
--------------------------------
|
|
36 |
|
|
37 |
*Note:* Windows users can skip this step, because all of Python is contained in
|
|
38 |
pythonXX.dll, and |PyInstaller| will use your pythonXX.dll.
|
|
39 |
|
|
40 |
On Linux the first thing to do is build the runtime executables.
|
|
41 |
|
|
42 |
Change to the |install_path| ``source/linux`` subdirectory. Run ``Make.py
|
|
43 |
[-n|-e]`` and then make. This will produce ``support/loader/run`` and
|
|
44 |
``support/loader/run_d``, which are the bootloaders.
|
|
45 |
|
|
46 |
.. sidebar:: Bootloader
|
|
47 |
|
|
48 |
The bootloader (also known as *stub* in literature) is the small program
|
|
49 |
which starts up your packaged program. Usually, the archive containing the
|
|
50 |
bytecoded modules of your program is simply attended to it. See
|
|
51 |
`Self-extracting executables`_ for more details on the process.
|
|
52 |
|
|
53 |
*Note:* If you have multiple versions of Python, the Python you use to run
|
|
54 |
``Make.py`` is the one whose configuration is used.
|
|
55 |
|
|
56 |
The ``-n`` and ``-e`` options set a non-elf or elf flag in your ``config.dat``.
|
|
57 |
As of |InitialVersion|, the executable will try both strategies, and this flag
|
|
58 |
just sets how you want your executables built. In the elf strategy, the archive
|
|
59 |
is concatenated to the executable. In the non-elf strategy, the executable
|
|
60 |
expects an archive with the same name as itself in the executable's directory.
|
|
61 |
Note that the executable chases down symbolic links before determining it's name
|
|
62 |
and directory, so putting the archive in the same directory as the symbolic link
|
|
63 |
will not work.
|
|
64 |
|
|
65 |
Windows distributions come with several executables in the ``support/loader``
|
|
66 |
directory: ``run_*.exe`` (bootloader for regular programs), and
|
|
67 |
``inprocsrvr_*.dll`` (bootloader for in-process COM servers). To rebuild this,
|
|
68 |
you need to install Scons_, and then just run ``scons`` from the |install_path|
|
|
69 |
directory.
|
|
70 |
|
|
71 |
|GOBACK|
|
|
72 |
|
|
73 |
Configuring your PyInstaller setup
|
|
74 |
----------------------------------
|
|
75 |
|
|
76 |
In the |install_path| directory, run ``Configure.py``. This saves some
|
|
77 |
information into ``config.dat`` that would otherwise be recomputed every time.
|
|
78 |
It can be rerun at any time if your configuration changes. It must be run before
|
|
79 |
trying to build anything.
|
|
80 |
|
|
81 |
|GOBACK|
|
|
82 |
|
|
83 |
|
|
84 |
Create a spec file for your project
|
|
85 |
-----------------------------------
|
|
86 |
|
|
87 |
[For Windows COM server support, see section `Windows COM Server Support`_]
|
|
88 |
|
|
89 |
The root directory has a script Makespec.py for this purpose::
|
|
90 |
|
|
91 |
python Makespec.py [opts] <scriptname> [<scriptname> ...]
|
|
92 |
|
|
93 |
Where allowed OPTIONS are:
|
|
94 |
|
|
95 |
-F, --onefile
|
|
96 |
produce a single file deployment (see below).
|
|
97 |
|
|
98 |
-D, --onedir
|
|
99 |
produce a single directory deployment (default).
|
|
100 |
|
|
101 |
-K, --tk
|
|
102 |
include TCL/TK in the deployment.
|
|
103 |
|
|
104 |
-a, --ascii
|
|
105 |
do not include encodings. The default (on Python versions with unicode
|
|
106 |
support) is now to include all encodings.
|
|
107 |
|
|
108 |
-d, --debug
|
|
109 |
use debug (verbose) versions of the executables.
|
|
110 |
|
|
111 |
-w, --windowed, --noconsole
|
|
112 |
Use the Windows subsystem executable, which does not open
|
|
113 |
the console when the program is launched. **(Windows only)**
|
|
114 |
|
|
115 |
-c, --nowindowed, --console
|
|
116 |
Use the console subsystem executable. This is the default. **(Windows only)**
|
|
117 |
|
|
118 |
-s, --strip
|
|
119 |
the executable and all shared libraries will be run through strip. Note
|
|
120 |
that cygwin's strip tends to render normal Win32 dlls unusable.
|
|
121 |
|
|
122 |
-X, --upx
|
|
123 |
if you have UPX installed (detected by Configure), this will use it to
|
|
124 |
compress your executable (and, on Windows, your dlls). See note below.
|
|
125 |
|
|
126 |
-o DIR, --out=DIR
|
|
127 |
create the spec file in *directory*. If not specified, and the current
|
|
128 |
directory is Installer's root directory, an output subdirectory will be
|
|
129 |
created. Otherwise the current directory is used.
|
|
130 |
|
|
131 |
-p DIR, --paths=DIR
|
|
132 |
set base path for import (like using PYTHONPATH). Multiple directories are
|
|
133 |
allowed, separating them with the path separator (';' under Windows, ':'
|
|
134 |
under Linux), or using this option multiple times.
|
|
135 |
|
|
136 |
--icon=<FILE.ICO>
|
|
137 |
add *file.ico* to the executable's resources. **(Windows only)**
|
|
138 |
|
|
139 |
--icon=<FILE.EXE,N>
|
|
140 |
add the *n*-th incon in *file.exe* to the executable's resources. **(Windows
|
|
141 |
only)**
|
|
142 |
|
|
143 |
-v FILE, --version=FILE
|
|
144 |
add verfile as a version resource to the executable. **(Windows only)**
|
|
145 |
|
|
146 |
-n NAME, --name=NAME
|
|
147 |
optional *name* to assign to the project (from which the spec file name is
|
|
148 |
generated). If omitted, the basename of the (first) script is used.
|
|
149 |
|
|
150 |
[For building with optimization on (like ``Python -O``), see section
|
|
151 |
`Building Optimized`_]
|
|
152 |
|
|
153 |
For simple projects, the generated spec file will probably be sufficient. For
|
|
154 |
more complex projects, it should be regarded as a template. The spec file is
|
|
155 |
actually Python code, and modifying it should be ease. See `Spec Files`_ for
|
|
156 |
details.
|
|
157 |
|
|
158 |
|
|
159 |
|GOBACK|
|
|
160 |
|
|
161 |
Build your project
|
|
162 |
------------------
|
|
163 |
|
|
164 |
::
|
|
165 |
|
|
166 |
python Build.py specfile
|
|
167 |
|
|
168 |
|
|
169 |
A ``buildproject`` subdirectory will be created in the specfile's directory. This
|
|
170 |
is a private workspace so that ``Build.py`` can act like a makefile. Any named
|
|
171 |
targets will appear in the specfile's directory. For ``--onedir``
|
|
172 |
configurations, it will create also ``distproject``, which is the directory you're
|
|
173 |
interested in. For a ``--onefile``, the executable will be in the specfile's
|
|
174 |
directory.
|
|
175 |
|
|
176 |
In most cases, this will be all you have to do. If not, see `When things go
|
|
177 |
wrong`_ and be sure to read the introduction to `Spec Files`_.
|
|
178 |
|
|
179 |
|GOBACK|
|
|
180 |
|
|
181 |
Windows COM Server support
|
|
182 |
--------------------------
|
|
183 |
|
|
184 |
For Windows COM support execute::
|
|
185 |
|
|
186 |
python MakeCOMServer.py [OPTION] script...
|
|
187 |
|
|
188 |
|
|
189 |
This will generate a new script ``drivescript.py`` and a spec file for the script.
|
|
190 |
|
|
191 |
These options are allowed:
|
|
192 |
|
|
193 |
--debug
|
|
194 |
Use the verbose version of the executable.
|
|
195 |
|
|
196 |
--verbose
|
|
197 |
Register the COM server(s) with the quiet flag off.
|
|
198 |
|
|
199 |
--ascii
|
|
200 |
do not include encodings (this is passed through to Makespec).
|
|
201 |
|
|
202 |
--out <dir>
|
|
203 |
Generate the driver script and spec file in dir.
|
|
204 |
|
|
205 |
Now `Build your project`_ on the generated spec file.
|
|
206 |
|
|
207 |
If you have the win32dbg package installed, you can use it with the generated
|
|
208 |
COM server. In the driver script, set ``debug=1`` in the registration line.
|
|
209 |
|
|
210 |
**Warnings**: the inprocess COM server support will not work when the client
|
|
211 |
process already has Python loaded. It would be rather tricky to
|
|
212 |
non-obtrusively hook into an already running Python, but the show-stopper is
|
|
213 |
that the Python/C API won't let us find out which interpreter instance I should
|
|
214 |
hook into. (If this is important to you, you might experiment with using
|
|
215 |
apartment threading, which seems the best possibility to get this to work). To
|
|
216 |
use a "frozen" COM server from a Python process, you'll have to load it as an
|
|
217 |
exe::
|
|
218 |
|
|
219 |
o = win32com.client.Dispatch(progid,
|
|
220 |
clsctx=pythoncom.CLSCTX_LOCAL_SERVER)
|
|
221 |
|
|
222 |
|
|
223 |
MakeCOMServer also assumes that your top level code (registration etc.) is
|
|
224 |
"normal". If it's not, you will have to edit the generated script.
|
|
225 |
|
|
226 |
|GOBACK|
|
|
227 |
|
|
228 |
|
|
229 |
Building Optimized
|
|
230 |
------------------
|
|
231 |
|
|
232 |
There are two facets to running optimized: gathering ``.pyo``'s, and setting the
|
|
233 |
``Py_OptimizeFlag``. Installer will gather ``.pyo``'s if it is run optimized::
|
|
234 |
|
|
235 |
python -O Build.py ...
|
|
236 |
|
|
237 |
|
|
238 |
The ``Py_OptimizeFlag`` will be set if you use a ``('O','','OPTION')`` in one of
|
|
239 |
the ``TOCs`` building the ``EXE``::
|
|
240 |
|
|
241 |
exe = EXE(pyz,
|
|
242 |
a.scripts + [('O','','OPTION')],
|
|
243 |
...
|
|
244 |
|
|
245 |
See `Spec Files`_ for details.
|
|
246 |
|
|
247 |
|GOBACK|
|
|
248 |
|
|
249 |
|
|
250 |
A Note on using UPX
|
|
251 |
-------------------
|
|
252 |
|
|
253 |
On both Windows and Linux, UPX can give truly startling compression - the days
|
|
254 |
of fitting something useful on a diskette are not gone forever! Installer has
|
|
255 |
been tested with many UPX versions without problems. Just get it and install it
|
|
256 |
on your PATH, then rerun configure.
|
|
257 |
|
|
258 |
For Windows, there is a problem of compatibility between UPX and executables
|
|
259 |
generated by Microsoft Visual Studio .NET 2003 (or the equivalent free
|
|
260 |
toolkit available for download). This is especially worrisome for users of
|
|
261 |
Python 2.4+, where most extensions (and Python itself) are compiled with that
|
|
262 |
compiler. This issue has been fixed in later beta versions of UPX, so you
|
|
263 |
will need at least UPX 1.92 beta. `Configure.py`_ will check this for you
|
|
264 |
and complain if you have an older version of UPX and you are using Python 2.4.
|
|
265 |
|
|
266 |
.. sidebar:: UPX and Unix
|
|
267 |
|
|
268 |
Under UNIX, old versions of UPX were not able to expand and execute the
|
|
269 |
executable in memory, and they were extracting it into a temporary file
|
|
270 |
in the filesystem, before spawning it. This is no longer valid under Linux,
|
|
271 |
but the information in this paragraph still needs to be updated.
|
|
272 |
|
|
273 |
.. _`Configure.py`: `Configuring your PyInstaller setup`_
|
|
274 |
|
|
275 |
For Linux, a bit more discussion is in order. First, UPX is only useful on
|
|
276 |
executables, not shared libs. Installer accounts for that, but to get the full
|
|
277 |
benefit, you might rebuild Python with more things statically linked.
|
|
278 |
|
|
279 |
More importantly, when ``run`` finds that its ``sys.argv[0]`` does not contain a path,
|
|
280 |
it will use ``/proc/pid/exe`` to find itself (if it can). This happens, for
|
|
281 |
example, when executed by Apache. If it has been upx-ed, this symbolic link
|
|
282 |
points to the tempfile created by the upx stub and |PyInstaller| will fail (please
|
|
283 |
see the UPX docs for more information). So for now, at least, you can't use upx
|
|
284 |
for CGI's executed by Apache. Otherwise, you can ignore the warnings in the UPX
|
|
285 |
docs, since what PyInstaller opens is the executable Installer created, not the
|
|
286 |
temporary upx-created executable.
|
|
287 |
|
|
288 |
|GOBACK|
|
|
289 |
|
|
290 |
A Note on ``--onefile``
|
|
291 |
-----------------------
|
|
292 |
|
|
293 |
A ``--onefile`` works by packing all the shared libs / dlls into the archive
|
|
294 |
attached to the bootloader executable (or next to the executable in a non-elf
|
|
295 |
configuration). When first started, it finds that it needs to extract these
|
|
296 |
files before it can run "for real". That's because locating and loading a
|
|
297 |
shared lib or linked-in dll is a system level action, not user-level. With
|
|
298 |
|PyInstallerVersion| it always uses a temporary directory (``_MEIpid``) in the
|
|
299 |
user's temp directory. It then executes itself again, setting things up so
|
|
300 |
the system will be able to load the shared libs / dlls. When executing is
|
|
301 |
complete, it recursively removes the entire directory it created.
|
|
302 |
|
|
303 |
This has a number of implications:
|
|
304 |
|
|
305 |
* You can run multiple copies - they won't collide.
|
|
306 |
|
|
307 |
* Running multiple copies will be rather expensive to the system (nothing is
|
|
308 |
shared).
|
|
309 |
|
|
310 |
* If you're using the cheat of adding user data as ``'BINARY'``, it will be in
|
|
311 |
``os.environ['_MEIPASS2']``, not in the executable's directory.
|
|
312 |
|
|
313 |
* On Windows, using Task Manager to kill the parent process will leave the
|
|
314 |
directory behind.
|
|
315 |
|
|
316 |
* On \*nix, a kill -9 (or crash) will leave the directory behind.
|
|
317 |
|
|
318 |
* Otherwise, on both platforms, the directory will be recursively deleted.
|
|
319 |
|
|
320 |
* So any files you might create in ``os.environ['_MEIPASS2']`` will be deleted.
|
|
321 |
|
|
322 |
* The executable can be in a protected or read-only directory.
|
|
323 |
|
|
324 |
* If for some reason, the ``_MEIpid`` directory already exists, the executable
|
|
325 |
will fail. It is created mode 0700, so only the one user can modify it
|
|
326 |
(on \*nix, of course).
|
|
327 |
|
|
328 |
While we are not a security expert, we believe the scheme is good enough for
|
|
329 |
most of the users.
|
|
330 |
|
|
331 |
**Notes for \*nix users**: Take notice that if the executable does a setuid root,
|
|
332 |
a determined hacker could possibly (given enough tries) introduce a malicious
|
|
333 |
lookalike of one of the shared libraries during the hole between when the
|
|
334 |
library is extracted and when it gets loaded by the execvp'd process. So maybe
|
|
335 |
you shouldn't do setuid root programs using ``--onefile``. **In fact, we do not
|
|
336 |
recomend the use of --onefile on setuid programs.**
|
|
337 |
|
|
338 |
|GOBACK|
|
|
339 |
|
|
340 |
A Note on .egg files and setuptools
|
|
341 |
-----------------------------------
|
|
342 |
`setuptools`_ is a distutils extensions which provide many benefits, including
|
|
343 |
the ability to distribute the extension as ``egg`` files. Together with the
|
|
344 |
nifty `easy_install`_ (a tool which automatically locates, downloads and
|
|
345 |
installs Python extensions), ``egg`` files are becoming more and more
|
|
346 |
widespread as a way for distributing Python extensions.
|
|
347 |
|
|
348 |
``egg`` files are actually ZIP files under the hood, and they rely on the fact
|
|
349 |
that Python 2.4 is able to transparently import modules stored within ZIP
|
|
350 |
files. PyInstaller is currently *not* able to import and extract modules
|
|
351 |
within ZIP files, so code which uses extensions packaged as ``egg`` files
|
|
352 |
cannot be packaged with PyInstaller.
|
|
353 |
|
|
354 |
The workaround is pretty easy: you can use ``easy_install -Z`` at installation
|
|
355 |
time to ask ``easy_install`` to always decompress egg files. This will allow
|
|
356 |
PyInstaller to see the files and make the package correctly. If you have already
|
|
357 |
installed the modules, you can simply decompress them within a directory with
|
|
358 |
the same name of the ``egg`` file (including also the extension).
|
|
359 |
|
|
360 |
Support for ``egg`` files is planned for a future release of PyInstaller.
|
|
361 |
|
|
362 |
.. _`setuptools`: http://peak.telecommunity.com/DevCenter/setuptools
|
|
363 |
.. _`easy_install`: http://peak.telecommunity.com/DevCenter/EasyInstall
|
|
364 |
|
|
365 |
|
|
366 |
|GOBACK|
|
|
367 |
|
|
368 |
|
|
369 |
PyInstaller Utilities
|
|
370 |
+++++++++++++++++++++
|
|
371 |
|
|
372 |
ArchiveViewer
|
|
373 |
-------------
|
|
374 |
|
|
375 |
::
|
|
376 |
|
|
377 |
python ArchiveViewer.py <archivefile>
|
|
378 |
|
|
379 |
|
|
380 |
ArchiveViewer lets you examine the contents of any archive build with
|
|
381 |
|PyInstaller| or executable (PYZ, PKG or exe). Invoke it with the target as the
|
|
382 |
first arg (It has been set up as a Send-To so it shows on the context menu in
|
|
383 |
Explorer). The archive can be navigated using these commands:
|
|
384 |
|
|
385 |
O <nm>
|
|
386 |
Open the embedded archive <nm> (will prompt if omitted).
|
|
387 |
|
|
388 |
U
|
|
389 |
Go up one level (go back to viewing the embedding archive).
|
|
390 |
|
|
391 |
X <nm>
|
|
392 |
Extract nm (will prompt if omitted). Prompts for output filename. If none
|
|
393 |
given, extracted to stdout.
|
|
394 |
|
|
395 |
Q
|
|
396 |
Quit.
|
|
397 |
|
|
398 |
|
|
399 |
|GOBACK|
|
|
400 |
|
|
401 |
|
|
402 |
bindepend
|
|
403 |
---------
|
|
404 |
|
|
405 |
::
|
|
406 |
|
|
407 |
python bindepend.py <executable_or_dynamic_library>
|
|
408 |
|
|
409 |
bindepend will analyze the executable you pass to it, and write to stdout all
|
|
410 |
its binary dependencies. This is handy to find out which DLLs are required by
|
|
411 |
an executable or another DLL. This module is used by |PyInstaller| itself to
|
|
412 |
follow the chain of dependencies of binary extensions and make sure that all
|
|
413 |
of them get included in the final package.
|
|
414 |
|
|
415 |
|
|
416 |
GrabVersion (Windows)
|
|
417 |
---------------------
|
|
418 |
|
|
419 |
::
|
|
420 |
|
|
421 |
python GrabVersion.py <executable_with_version_resource>
|
|
422 |
|
|
423 |
|
|
424 |
GrabVersion outputs text which can be eval'ed by ``versionInfo.py`` to reproduce
|
|
425 |
a version resource. Invoke it with the full path name of a Windows executable
|
|
426 |
(with a version resource) as the first argument. If you cut & paste (or
|
|
427 |
redirect to a file), you can then edit the version information. The edited
|
|
428 |
text file can be used in a ``version = myversion.txt`` option on any executable
|
|
429 |
in an |PyInstaller| spec file.
|
|
430 |
|
|
431 |
This was done in this way because version resources are rather strange beasts,
|
|
432 |
and fully understanding them is probably impossible. Some elements are
|
|
433 |
optional, others required, but you could spend unbounded amounts of time
|
|
434 |
figuring this out, because it's not well documented. When you view the version
|
|
435 |
tab on a properties dialog, there's no straightforward relationship between
|
|
436 |
how the data is displayed and the structure of the resource itself. So the
|
|
437 |
easiest thing to do is find an executable that displays the kind of
|
|
438 |
information you want, grab it's resource and edit it. Certainly easier than
|
|
439 |
the Version resource wizard in VC++.
|
|
440 |
|
|
441 |
|GOBACK|
|
|
442 |
|
|
443 |
|
|
444 |
Analyzing Dependencies
|
|
445 |
----------------------
|
|
446 |
|
|
447 |
You can interactively track down dependencies, including getting
|
|
448 |
cross-references by using ``mf.py``, documented in section `mf.py: A modulefinder
|
|
449 |
Replacement`_
|
|
450 |
|
|
451 |
|GOBACK|
|
|
452 |
|
|
453 |
|
|
454 |
Spec Files
|
|
455 |
++++++++++
|
|
456 |
|
|
457 |
Introduction
|
|
458 |
------------
|
|
459 |
|
|
460 |
Spec files are in Python syntax. They are evaluated by Build.py. A simplistic
|
|
461 |
spec file might look like this::
|
|
462 |
|
|
463 |
a = Analysis(['myscript.py'])
|
|
464 |
pyz = PYZ(a.pure)
|
|
465 |
exe = EXE(pyz, a.scripts, a.binaries, name="myapp.exe")
|
|
466 |
|
|
467 |
This creates a single file deployment with all binaries (extension modules and
|
|
468 |
their dependencies) packed into the executable.
|
|
469 |
|
|
470 |
A simplistic single directory deployment might look like this::
|
|
471 |
|
|
472 |
a = Analysis(['myscript.py'])
|
|
473 |
pyz = PYZ(a.pure)
|
|
474 |
exe = EXE(a.scripts, pyz, name="myapp.exe", exclude_binaries=1)
|
|
475 |
dist = COLLECT(exe, a.binaries, name="dist")
|
|
476 |
|
|
477 |
|
|
478 |
Note that neither of these examples are realistic. Use ``Makespec.py`` (documented
|
|
479 |
in section `Create a spec file for your project`_) to create your specfile,
|
|
480 |
and tweak it (if necessary) from there.
|
|
481 |
|
|
482 |
All of the classes you see above are subclasses of ``Build.Target``. A Target acts
|
|
483 |
like a rule in a makefile. It knows enough to cache its last inputs and
|
|
484 |
outputs. If its inputs haven't changed, it can assume its outputs wouldn't
|
|
485 |
change on recomputation. So a spec file acts much like a makefile, only
|
|
486 |
rebuilding as much as needs rebuilding. This means, for example, that if you
|
|
487 |
change an ``EXE`` from ``debug=1`` to ``debug=0``, the rebuild will be nearly
|
|
488 |
instantaneous.
|
|
489 |
|
|
490 |
The high level view is that an ``Analysis`` takes a list of scripts as input,
|
|
491 |
and generates three "outputs", held in attributes named ``scripts``, ``pure``
|
|
492 |
and ``binaries``. A ``PYZ`` (a ``.pyz`` archive) is built from the modules in
|
|
493 |
pure. The ``EXE`` is built from the ``PYZ``, the scripts and, in the case of a
|
|
494 |
single-file deployment, the binaries. In a single-directory deployment, a
|
|
495 |
directory is built containing a slim executable and the binaries.
|
|
496 |
|
|
497 |
|GOBACK|
|
|
498 |
|
|
499 |
TOC Class (Table of Contents)
|
|
500 |
-----------------------------
|
|
501 |
|
|
502 |
Before you can do much with a spec file, you need to understand the
|
|
503 |
``TOC`` (Table Of Contents) class.
|
|
504 |
|
|
505 |
A ``TOC`` appears to be a list of tuples of the form (name, path, typecode).
|
|
506 |
In fact, it's an ordered set, not a list. A TOC contains no duplicates, where
|
|
507 |
uniqueness is based on name only. Furthermore, within this constraint, a TOC
|
|
508 |
preserves order.
|
|
509 |
|
|
510 |
Besides the normal list methods and operations, TOC supports taking differences
|
|
511 |
and intersections (and note that adding or extending is really equivalent to
|
|
512 |
union). Furthermore, the operations can take a real list of tuples on the right
|
|
513 |
hand side. This makes excluding modules quite easy. For a pure Python module::
|
|
514 |
|
|
515 |
pyz = PYZ(a.pure - [('badmodule', '', '')])
|
|
516 |
|
|
517 |
|
|
518 |
or for an extension module in a single-directory deployment::
|
|
519 |
|
|
520 |
dist = COLLECT(..., a.binaries - [('badmodule', '', '')], ...)
|
|
521 |
|
|
522 |
|
|
523 |
or for a single-file deployment::
|
|
524 |
|
|
525 |
exe = EXE(..., a.binaries - [('badmodule', '', '')], ...)
|
|
526 |
|
|
527 |
To add files to a TOC, you need to know about the typecodes (or the step using
|
|
528 |
the TOC won't know what to do with the entry).
|
|
529 |
|
|
530 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
531 |
| **typecode** | **description** | **name** | **path** |
|
|
532 |
+===============+=======================================================+=======================+===============================+
|
|
533 |
| 'EXTENSION' | An extension module. | Python internal name. | Full path name in build. |
|
|
534 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
535 |
| 'PYSOURCE' | A script. | Python internal name. | Full path name in build. |
|
|
536 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
537 |
| 'PYMODULE' | A pure Python module (including __init__ modules). | Python internal name. | Full path name in build. |
|
|
538 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
539 |
| 'PYZ' | A .pyz archive (archive_rt.ZlibArchive). | Runtime name. | Full path name in build. |
|
|
540 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
541 |
| 'PKG' | A pkg archive (carchive4.CArchive). | Runtime name. | Full path name in build. |
|
|
542 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
543 |
| 'BINARY' | A shared library. | Runtime name. | Full path name in build. |
|
|
544 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
545 |
| 'DATA' | Aribitrary files. | Runtime name. | Full path name in build. |
|
|
546 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
547 |
| 'OPTION' | A runtime runtime option (frozen into the executable).| The option. | Unused. |
|
|
548 |
+---------------+-------------------------------------------------------+-----------------------+-------------------------------+
|
|
549 |
|
|
550 |
You can force the include of any file in much the same way you do excludes::
|
|
551 |
|
|
552 |
collect = COLLECT(a.binaries +
|
|
553 |
[('readme', '/my/project/readme', 'DATA')], ...)
|
|
554 |
|
|
555 |
|
|
556 |
or even::
|
|
557 |
|
|
558 |
collect = COLLECT(a.binaries,
|
|
559 |
[('readme', '/my/project/readme', 'DATA')], ...)
|
|
560 |
|
|
561 |
|
|
562 |
(that is, you can use a list of tuples in place of a ``TOC`` in most cases).
|
|
563 |
|
|
564 |
There's not much reason to use this technique for ``PYSOURCE``, since an ``Analysis``
|
|
565 |
takes a list of scripts as input. For ``PYMODULEs`` and ``EXTENSIONs``, the hook
|
|
566 |
mechanism discussed here is better because you won't have to remember how you
|
|
567 |
got it working next time.
|
|
568 |
|
|
569 |
This technique is most useful for data files (see the ``Tree`` class below for a
|
|
570 |
way to build a ``TOC`` from a directory tree), and for runtime options. The options
|
|
571 |
the run executables understand are:
|
|
572 |
|
|
573 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
574 |
| **Option** | **Description** | **Example** | **Notes** |
|
|
575 |
+===============+=======================+===============================+=======================================================================================================+
|
|
576 |
| v | Verbose imports | ('v', '', 'OPTION') | Same as Python -v ... |
|
|
577 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
578 |
| u | Unbuffered stdio | ('u', '', 'OPTION') | Same as Python -u ... |
|
|
579 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
580 |
| W spec | Warning option | ('W ignore', '', 'OPTION') | Python 2.1+ only. |
|
|
581 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
582 |
| s | Use site.py | ('s', '', 'OPTION') | The opposite of Python's -S flag. Note that site.py must be in the executable's directory to be used. |
|
|
583 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
584 |
| f | Force execvp | ('f', '', 'OPTION') | Linux/unix only. Ensures that LD_LIBRARY_PATH is set properly. |
|
|
585 |
+---------------+-----------------------+-------------------------------+-------------------------------------------------------------------------------------------------------+
|
|
586 |
|
|
587 |
Advanced users should note that by using set differences and intersections, it
|
|
588 |
becomes possible to factor out common modules, and deploy a project containing
|
|
589 |
multiple executables with minimal redundancy. You'll need some top level code
|
|
590 |
in each executable to mount the common ``PYZ``.
|
|
591 |
|
|
592 |
|GOBACK|
|
|
593 |
|
|
594 |
Target Subclasses
|
|
595 |
-----------------
|
|
596 |
|
|
597 |
Analysis
|
|
598 |
********
|
|
599 |
|
|
600 |
::
|
|
601 |
|
|
602 |
Analysis(scripts, pathex=None, hookspath=None, excludes=None)
|
|
603 |
|
|
604 |
|
|
605 |
``scripts``
|
|
606 |
a list of scripts specified as file names.
|
|
607 |
|
|
608 |
``pathex``
|
|
609 |
an optional list of paths to be searched before sys.path.
|
|
610 |
|
|
611 |
``hookspath``
|
|
612 |
an optional list of paths used to extend the hooks package.
|
|
613 |
|
|
614 |
``excludes``
|
|
615 |
an optional list of module or package names (their Python names, not path
|
|
616 |
names) that will be ignored (as though they were not found).
|
|
617 |
|
|
618 |
An Analysis has three outputs, all ``TOCs`` accessed as attributes of the ``Analysis``.
|
|
619 |
|
|
620 |
``scripts``
|
|
621 |
The scripts you gave Analysis as input, with any runtime hook scripts
|
|
622 |
prepended.
|
|
623 |
|
|
624 |
``pure``
|
|
625 |
The pure Python modules.
|
|
626 |
|
|
627 |
``binaries``
|
|
628 |
The extension modules and their dependencies. The secondary dependencies are
|
|
629 |
filtered. On Windows, a long list of MS dlls are excluded. On Linux/Unix,
|
|
630 |
any shared lib in ``/lib`` or ``/usr/lib`` is excluded.
|
|
631 |
|
|
632 |
|GOBACK|
|
|
633 |
|
|
634 |
PYZ
|
|
635 |
***
|
|
636 |
|
|
637 |
::
|
|
638 |
|
|
639 |
PYZ(toc, name=None, level=9)
|
|
640 |
|
|
641 |
|
|
642 |
``toc``
|
|
643 |
a ``TOC``, normally an ``Analysis.pure``.
|
|
644 |
|
|
645 |
``name``
|
|
646 |
A filename for the ``.pyz``. Normally not needed, as the generated name will do fine.
|
|
647 |
|
|
648 |
``level``
|
|
649 |
The Zlib compression level to use. If 0, the zlib module is not required.
|
|
650 |
|
|
651 |
|
|
652 |
|GOBACK|
|
|
653 |
|
|
654 |
PKG
|
|
655 |
***
|
|
656 |
|
|
657 |
Generally, you will not need to create your own ``PKGs``, as the ``EXE`` will do it for
|
|
658 |
you. This is one way to include read-only data in a single-file deployment,
|
|
659 |
however. A single-file deployment including TK support will use this technique.
|
|
660 |
|
|
661 |
::
|
|
662 |
|
|
663 |
PKG(toc, name=None, cdict=None, exclude_binaries=0)
|
|
664 |
|
|
665 |
|
|
666 |
``toc``
|
|
667 |
a ``TOC``.
|
|
668 |
|
|
669 |
``name``
|
|
670 |
a filename for the ``PKG`` (optional).
|
|
671 |
|
|
672 |
``cdict``
|
|
673 |
a dictionary that specifies compression by typecode. For example, ``PYZ`` is
|
|
674 |
left uncompressed so that it can be accessed inside the ``PKG``. The default
|
|
675 |
uses sensible values. If zlib is not available, no compression is used.
|
|
676 |
|
|
677 |
``exclude_binaries``
|
|
678 |
If 1, ``EXTENSIONs`` and ``BINARYs`` will be left out of the ``PKG``, and
|
|
679 |
forwarded to its container (usually a ``COLLECT``).
|
|
680 |
|
|
681 |
|GOBACK|
|
|
682 |
|
|
683 |
EXE
|
|
684 |
***
|
|
685 |
|
|
686 |
::
|
|
687 |
|
|
688 |
EXE(*args, **kws)
|
|
689 |
|
|
690 |
|
|
691 |
``args``
|
|
692 |
One or more arguments which are either ``TOCs`` or ``Targets``.
|
|
693 |
|
|
694 |
``kws``
|
|
695 |
Possible keyword arguments:
|
|
696 |
|
|
697 |
``console``
|
|
698 |
Always 1 on Linux/unix. On Windows, governs whether to use the console
|
|
699 |
executable, or the Windows subsystem executable.
|
|
700 |
|
|
701 |
``debug``
|
|
702 |
Setting to 1 gives you progress messages from the executable (for a
|
|
703 |
``console=0``, these will be annoying MessageBoxes).
|
|
704 |
|
|
705 |
``name``
|
|
706 |
The filename for the executable.
|
|
707 |
|
|
708 |
``exclude_binaries``
|
|
709 |
Forwarded to the ``PKG`` the ``EXE`` builds.
|
|
710 |
|
|
711 |
``icon``
|
|
712 |
Windows NT family only. ``icon='myicon.ico'`` to use an icon file, or
|
|
713 |
``icon='notepad.exe,0'`` to grab an icon resource.
|
|
714 |
|
|
715 |
``version``
|
|
716 |
Windows NT family only. ``version='myversion.txt'``. Use ``GrabVersion.py`` to
|
|
717 |
steal a version resource from an executable, and then edit the ouput to
|
|
718 |
create your own. (The syntax of version resources is so arcane that I
|
|
719 |
wouldn't attempt to write one from scratch.)
|
|
720 |
|
|
721 |
|
|
722 |
There are actually two ``EXE`` classes - one for ELF platforms (where the
|
|
723 |
bootloader, that is the ``run`` executable, and the ``PKG`` are concatenated),
|
|
724 |
and one for non-ELF platforms (where the run executable is simply renamed, and
|
|
725 |
expects a ``exename.pkg`` in the same directory). Which class becomes available
|
|
726 |
as ``EXE`` is determined by a flag in ``config.dat``. This flag is set to
|
|
727 |
non-ELF when using ``Make.py -n``.
|
|
728 |
|
|
729 |
|GOBACK|
|
|
730 |
|
|
731 |
DLL
|
|
732 |
***
|
|
733 |
|
|
734 |
On Windows, this provides support for doing in-process COM servers. It is not
|
|
735 |
generalized. However, embedders can follow the same model to build a special
|
|
736 |
purpose DLL so the Python support in their app is hidden. You will need to
|
|
737 |
write your own dll, but thanks to Allan Green for refactoring the C code and
|
|
738 |
making that a managable task.
|
|
739 |
|
|
740 |
|GOBACK|
|
|
741 |
|
|
742 |
COLLECT
|
|
743 |
*******
|
|
744 |
|
|
745 |
::
|
|
746 |
|
|
747 |
COLLECT(*args, **kws)
|
|
748 |
|
|
749 |
|
|
750 |
``args``
|
|
751 |
One or more arguments which are either ``TOCs`` or ``Targets``.
|
|
752 |
|
|
753 |
``kws``
|
|
754 |
Possible keyword arguments:
|
|
755 |
|
|
756 |
``name``
|
|
757 |
The name of the directory to be built.
|
|
758 |
|
|
759 |
|GOBACK|
|
|
760 |
|
|
761 |
Tree
|
|
762 |
****
|
|
763 |
|
|
764 |
::
|
|
765 |
|
|
766 |
Tree(root, prefix=None, excludes=None)
|
|
767 |
|
|
768 |
|
|
769 |
``root``
|
|
770 |
The root of the tree (on the build system).
|
|
771 |
|
|
772 |
``prefix``
|
|
773 |
Optional prefix to the names on the target system.
|
|
774 |
|
|
775 |
``excludes``
|
|
776 |
A list of names to exclude. Two forms are allowed:
|
|
777 |
|
|
778 |
``name``
|
|
779 |
files with this basename will be excluded (do not include the path).
|
|
780 |
|
|
781 |
``*.ext``
|
|
782 |
any file with the given extension will be excluded.
|
|
783 |
|
|
784 |
Since a ``Tree`` is a ``TOC``, you can also use the exclude technique described above
|
|
785 |
in the section on ``TOCs``.
|
|
786 |
|
|
787 |
|
|
788 |
|GOBACK|
|
|
789 |
|
|
790 |
When Things Go Wrong
|
|
791 |
++++++++++++++++++++
|
|
792 |
|
|
793 |
Finding out What Went Wrong
|
|
794 |
---------------------------
|
|
795 |
|
|
796 |
Buildtime Warnings
|
|
797 |
******************
|
|
798 |
|
|
799 |
When an ``Analysis`` step runs, it produces a warnings file (named ``warnproject.txt``)
|
|
800 |
in the spec file's directory. Generally, most of these warnings are harmless.
|
|
801 |
For example, ``os.py`` (which is cross-platform) works by figuring out what
|
|
802 |
platform it is on, then importing (and rebinding names from) the appropriate
|
|
803 |
platform-specific module. So analyzing ``os.py`` will produce a set of warnings
|
|
804 |
like::
|
|
805 |
|
|
806 |
W: no module named dos (conditional import by os)
|
|
807 |
W: no module named ce (conditional import by os)
|
|
808 |
W: no module named os2 (conditional import by os)
|
|
809 |
|
|
810 |
|
|
811 |
Note that the analysis has detected that the import is within a conditional
|
|
812 |
block (an if statement). The analysis also detects if an import within a
|
|
813 |
function or class, (delayed) or at the top level. A top-level, non-conditional
|
|
814 |
import failure is really a hard error. There's at least a reasonable chance
|
|
815 |
that conditional and / or delayed import will be handled gracefully at runtime.
|
|
816 |
|
|
817 |
Ignorable warnings may also be produced when a class or function is declared in
|
|
818 |
a package (an ``__init__.py`` module), and the import specifies
|
|
819 |
``package.name``. In this case, the analysis can't tell if name is supposed to
|
|
820 |
refer to a submodule of package.
|
|
821 |
|
|
822 |
Warnings are also produced when an ``__import__``, ``exec`` or ``eval`` statement is
|
|
823 |
encountered. The ``__import__`` warnings should almost certainly be investigated.
|
|
824 |
Both ``exec`` and ``eval`` can be used to implement import hacks, but usually their use
|
|
825 |
is more benign.
|
|
826 |
|
|
827 |
Any problem detected here can be handled by hooking the analysis of the module.
|
|
828 |
See `Listing Hidden Imports`_ below for how to do it.
|
|
829 |
|
|
830 |
|GOBACK|
|
|
831 |
|
|
832 |
Getting Debug Messages
|
|
833 |
**********************
|
|
834 |
|
|
835 |
Setting ``debug=1`` on an ``EXE`` will cause the executable to put out progress
|
|
836 |
messages (for console apps, these go to stdout; for Windows apps, these show as
|
|
837 |
MessageBoxes). This can be useful if you are doing complex packaging, or your
|
|
838 |
app doesn't seem to be starting, or just to learn how the runtime works.
|
|
839 |
|
|
840 |
|GOBACK|
|
|
841 |
|
|
842 |
Getting Python's Verbose Imports
|
|
843 |
********************************
|
|
844 |
|
|
845 |
You can also pass a ``-v`` (verbose imports) flag to the embedded Python. This can
|
|
846 |
be extremely useful. I usually try it even on apparently working apps, just to
|
|
847 |
make sure that I'm always getting my copies of the modules and no import has
|
|
848 |
leaked out to the installed Python.
|
|
849 |
|
|
850 |
You set this (like the other runtime options) by feeding a phone ``TOC`` entry to
|
|
851 |
the ``EXE``. The easiest way to do this is to change the ``EXE`` from::
|
|
852 |
|
|
853 |
EXE(..., anal.scripts, ....)
|
|
854 |
|
|
855 |
to::
|
|
856 |
|
|
857 |
EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)
|
|
858 |
|
|
859 |
These messages will always go to ``stdout``, so you won't see them on Windows if
|
|
860 |
``console=0``.
|
|
861 |
|
|
862 |
|GOBACK|
|
|
863 |
|
|
864 |
Helping Installer Find Modules
|
|
865 |
------------------------------
|
|
866 |
|
|
867 |
Extending the Path
|
|
868 |
******************
|
|
869 |
|
|
870 |
When the analysis phase cannot find needed modules, it may be that the code is
|
|
871 |
manipulating ``sys.path``. The easiest thing to do in this case is tell ``Analysis``
|
|
872 |
about the new directory through the second arg to the constructor::
|
|
873 |
|
|
874 |
anal = Analysis(['somedir/myscript.py'],
|
|
875 |
['path/to/thisdir', 'path/to/thatdir'])
|
|
876 |
|
|
877 |
|
|
878 |
In this case, the ``Analysis`` will have a search path::
|
|
879 |
|
|
880 |
['somedir', 'path/to/thisdir', 'path/to/thatdir'] + sys.path
|
|
881 |
|
|
882 |
|
|
883 |
You can do the same when running ``Makespec.py``::
|
|
884 |
|
|
885 |
Makespec.py --paths=path/to/thisdir;path/to/thatdir ...
|
|
886 |
|
|
887 |
|
|
888 |
(on \*nix, use ``:`` as the path separator).
|
|
889 |
|
|
890 |
|GOBACK|
|
|
891 |
|
|
892 |
Listing Hidden Imports
|
|
893 |
**********************
|
|
894 |
|
|
895 |
Hidden imports are fairly common. These can occur when the code is using
|
|
896 |
``__import__`` (or, perhaps ``exec`` or ``eval``), in which case you will see a warning in
|
|
897 |
the ``warnproject.txt`` file. They can also occur when an extension module uses the
|
|
898 |
Python/C API to do an import, in which case Analysis can't detect anything. You
|
|
899 |
can verify that hidden import is the problem by using Python's verbose imports
|
|
900 |
flag. If the import messages say "module not found", but the ``warnproject.txt``
|
|
901 |
file has no "no module named..." message for the same module, then the problem
|
|
902 |
is a hidden import.
|
|
903 |
|
|
904 |
.. sidebar:: Standard hidden imports are already included!
|
|
905 |
|
|
906 |
If you are getting worried while reading this paragraph, do not worry:
|
|
907 |
having hidden imports is the exception, not the norm! And anyway,
|
|
908 |
PyInstaller already ships with a large set of hooks that take care of
|
|
909 |
hidden imports for the most common packages out there. For instance,
|
|
910 |
PIL_, PyWin32_, PyQt_ are already taken care of.
|
|
911 |
|
|
912 |
Hidden imports are handled by hooking the module (the one doing the hidden
|
|
913 |
imports) at ``Analysis`` time. Do this by creating a file named ``hook-module.py``
|
|
914 |
(where module is the fully-qualified Python name, eg, ``hook-xml.dom.py``), and
|
|
915 |
placing it in the ``hooks`` package under |PyInstaller|'s root directory,
|
|
916 |
(alternatively, you can save it elsewhere, and then use the ``hookspath`` arg to
|
|
917 |
``Analysis`` so your private hooks directory will be searched). Normally, it will
|
|
918 |
have only one line::
|
|
919 |
|
|
920 |
hiddenimports = ['module1', 'module2']
|
|
921 |
|
|
922 |
When the ``Analysis`` finds this file, it will proceed exactly as though the module
|
|
923 |
explicitly imported ``module1`` and ``module2``. (Full details on the analysis-time
|
|
924 |
hook mechanism is in the `Hooks`_ section).
|
|
925 |
|
|
926 |
If you successfully hook a publicly distributed module in this way, please send
|
|
927 |
us the hook so we can make it available to others.
|
|
928 |
|
|
929 |
|GOBACK|
|
|
930 |
|
|
931 |
Extending a Package's ``__path__``
|
|
932 |
**********************************
|
|
933 |
|
|
934 |
Python allows a package to extend the search path used to find modules and
|
|
935 |
sub-packages through the ``__path__`` mechanism. Normally, a package's ``__path__`` has
|
|
936 |
only one entry - the directory in which the ``__init__.py`` was found. But
|
|
937 |
``__init__.py`` is free to extend its ``__path__`` to include other directories. For
|
|
938 |
example, the ``win32com.shell.shell`` module actually resolves to
|
|
939 |
``win32com/win32comext/shell/shell.pyd``. This is because ``win32com/__init__.py``
|
|
940 |
appends ``../win32comext`` to its ``__path__``.
|
|
941 |
|
|
942 |
Because the ``__init__.py`` is not actually run during an analysis, we use the same
|
|
943 |
hook mechanism we use for hidden imports. A static list of names won't do,
|
|
944 |
however, because the new entry on ``__path__`` may well require computation. So
|
|
945 |
``hook-module.py`` should define a method ``hook(mod)``. The mod argument is an
|
|
946 |
instance of ``mf.Module`` which has (more or less) the same attributes as a real
|
|
947 |
module object. The hook function should return a ``mf.Module`` instance - perhaps
|
|
948 |
a brand new one, but more likely the same one used as an arg, but mutated.
|
|
949 |
See `mf.py: A Modulefinder Replacement`_ for details, and `hooks\/hook-win32com.py`_
|
|
950 |
for an example.
|
|
951 |
|
|
952 |
Note that manipulations of ``__path__`` hooked in this way apply to the analysis,
|
|
953 |
and only the analysis. That is, at runtime ``win32com.shell`` is resolved the same
|
|
954 |
way as ``win32com.anythingelse``, and ``win32com.__path__`` knows nothing of ``../win32comext``.
|
|
955 |
|
|
956 |
Once in awhile, that's not enough.
|
|
957 |
|
|
958 |
|GOBACK|
|
|
959 |
|
|
960 |
Changing Runtime Behavior
|
|
961 |
*************************
|
|
962 |
|
|
963 |
More bizarre situations can be accomodated with runtime hooks. These are small
|
|
964 |
scripts that manipulate the environment before your main script runs,
|
|
965 |
effectively providing additional top-level code to your script.
|
|
966 |
|
|
967 |
At the tail end of an analysis, the module list is examined for matches in
|
|
968 |
``rthooks.dat``, which is the string representation of a Python dictionary. The
|
|
969 |
key is the module name, and the value is a list of hook-script pathnames.
|
|
970 |
|
|
971 |
So putting an entry::
|
|
972 |
|
|
973 |
'somemodule': ['path/to/somescript.py'],
|
|
974 |
|
|
975 |
into ``rthooks.dat`` is almost the same thing as doing this::
|
|
976 |
|
|
977 |
anal = Analysis(['path/to/somescript.py', 'main.py'], ...
|
|
978 |
|
|
979 |
|
|
980 |
except that in using the hook, ``path/to/somescript.py`` will not be analyzed,
|
|
981 |
(that's not a feature - we just haven't found a sane way fit the recursion into
|
|
982 |
my persistence scheme).
|
|
983 |
|
|
984 |
Hooks done in this way, while they need to be careful of what they import, are
|
|
985 |
free to do almost anything. One provided hook sets things up so that win32com
|
|
986 |
can generate modules at runtime (to disk), and the generated modules can be
|
|
987 |
found in the win32com package.
|
|
988 |
|
|
989 |
|GOBACK|
|
|
990 |
|
|
991 |
Adapting to being "frozen"
|
|
992 |
**************************
|
|
993 |
|
|
994 |
In most sophisticated apps, it becomes necessary to figure out (at runtime)
|
|
995 |
whether you're running "live" or "frozen". For example, you might have a
|
|
996 |
configuration file that (running "live") you locate based on a module's
|
|
997 |
``__file__`` attribute. That won't work once the code is packaged up. You'll
|
|
998 |
probably want to look for it based on ``sys.executable`` instead.
|
|
999 |
|
|
1000 |
The bootloaders set ``sys.frozen=1`` (and, for in-process COM servers, the
|
|
1001 |
embedding DLL sets ``sys.frozen='dll'``).
|
|
1002 |
|
|
1003 |
For really advanced users, you can access the ``iu.ImportManager`` as
|
|
1004 |
``sys.importManager``. See `iu.py`_ for how you might make use of this fact.
|
|
1005 |
|
|
1006 |
|GOBACK|
|
|
1007 |
|
|
1008 |
Accessing Data Files
|
|
1009 |
********************
|
|
1010 |
|
|
1011 |
In a ``--onedir`` distribution, this is easy: pass a list of your data files
|
|
1012 |
(in ``TOC`` format) to the ``COLLECT``, and they will show up in the distribution
|
|
1013 |
directory tree. The name in the ``(name, path, 'DATA')`` tuple can be a relative
|
|
1014 |
path name. Then, at runtime, you can use code like this to find the file::
|
|
1015 |
|
|
1016 |
os.path.join(os.path.dirname(sys.executable), relativename))
|
|
1017 |
|
|
1018 |
|
|
1019 |
In a ``--onefile``, it's a bit trickier. You can cheat, and add the files to the
|
|
1020 |
``EXE`` as ``BINARY``. They will then be extracted at runtime into the work directory
|
|
1021 |
by the C code (which does not create directories, so the name must be a plain
|
|
1022 |
name), and cleaned up on exit. The work directory is best found by
|
|
1023 |
``os.environ['_MEIPASS2']``. Be awawre, though, that if you use ``--strip`` or ``--upx``,
|
|
1024 |
strange things may happen to your data - ``BINARY`` is really for shared
|
|
1025 |
libs / dlls.
|
|
1026 |
|
|
1027 |
If you add them as ``'DATA'`` to the ``EXE``, then it's up to you to extract them. Use
|
|
1028 |
code like this::
|
|
1029 |
|
|
1030 |
import sys, carchive
|
|
1031 |
this = carchive.CArchive(sys.executable)
|
|
1032 |
data = this.extract('mystuff')[1]
|
|
1033 |
|
|
1034 |
|
|
1035 |
to get the contents as a binary string. See `support\/unpackTK.py`_ for an advanced
|
|
1036 |
example (the TCL and TK lib files are in a PKG which is opened in place, and
|
|
1037 |
then extracted to the filesystem).
|
|
1038 |
|
|
1039 |
|GOBACK|
|
|
1040 |
|
|
1041 |
Miscellaneous
|
|
1042 |
+++++++++++++
|
|
1043 |
|
|
1044 |
Pmw -- Python Mega Widgets
|
|
1045 |
--------------------------
|
|
1046 |
|
|
1047 |
`Pmw`_ comes with a script named ``bundlepmw`` in the bin directory. If you follow the
|
|
1048 |
instructions in that script, you'll end up with a module named ``Pmw.py``. Ensure
|
|
1049 |
that Builder finds that module and not the development package.
|
|
1050 |
|
|
1051 |
|GOBACK|
|
|
1052 |
|
|
1053 |
Win9xpopen
|
|
1054 |
----------
|
|
1055 |
|
|
1056 |
If you're using popen on Windows and want the code to work on Win9x, you'll
|
|
1057 |
need to distribute ``win9xpopen.exe`` with your app. On older Pythons with
|
|
1058 |
Win32all, this would apply to Win32pipe and ``win32popenWin9x.exe``. (On yet older
|
|
1059 |
Pythons, no form of popen worked on Win9x).
|
|
1060 |
|
|
1061 |
|GOBACK|
|
|
1062 |
|
|
1063 |
Self-extracting executables
|
|
1064 |
---------------------------
|
|
1065 |
|
|
1066 |
The ELF executable format (Windows, Linux and some others) allows arbitrary
|
|
1067 |
data to be concatenated to the end of the executable without disturbing its
|
|
1068 |
functionality. For this reason, a ``CArchive``'s Table of Contents is at the end of
|
|
1069 |
the archive. The executable can open itself as a binary file name, seek to the
|
|
1070 |
end and 'open' the ``CArchive`` (see figure 3).
|
|
1071 |
|
|
1072 |
On other platforms, the archive and the executable are separate, but the
|
|
1073 |
archive is named ``executable.pkg``, and expected to be in the same directory.
|
|
1074 |
Other than that, the process is the same.
|
|
1075 |
|
|
1076 |
|GOBACK|
|
|
1077 |
|
|
1078 |
One Pass Execution
|
|
1079 |
******************
|
|
1080 |
|
|
1081 |
In a single directory deployment (``--onedir``, which is the default), all of the
|
|
1082 |
binaries are already in the file system. In that case, the embedding app:
|
|
1083 |
|
|
1084 |
* opens the archive
|
|
1085 |
|
|
1086 |
* starts Python (on Windows, this is done with dynamic loading so one embedding
|
|
1087 |
app binary can be used with any Python version)
|
|
1088 |
|
|
1089 |
* imports all the modules which are at the top level of the archive (basically,
|
|
1090 |
bootstraps the import hooks)
|
|
1091 |
|
|
1092 |
* mounts the ``ZlibArchive(s)`` in the outer archive
|
|
1093 |
|
|
1094 |
* runs all the scripts which are at the top level of the archive
|
|
1095 |
|
|
1096 |
* finalizes Python
|
|
1097 |
|
|
1098 |
|GOBACK|
|
|
1099 |
|
|
1100 |
Two Pass Execution
|
|
1101 |
******************
|
|
1102 |
|
|
1103 |
There are a couple situations which require two passes:
|
|
1104 |
|
|
1105 |
* a ``--onefile`` deployment (on Windows, the files can't be cleaned up afterwards
|
|
1106 |
because Python does not call ``FreeLibrary``; on other platforms, Python won't
|
|
1107 |
find them if they're extracted in the same process that uses them)
|
|
1108 |
|
|
1109 |
* ``LD_LIBRARY_PATH`` needs to be set to find the binaries (not extension modules,
|
|
1110 |
but modules the extensions are linked to).
|
|
1111 |
|
|
1112 |
The first pass:
|
|
1113 |
|
|
1114 |
* opens the archive
|
|
1115 |
|
|
1116 |
* extracts all the binaries in the archive (in |PyInstallerVersion|, this is always to a
|
|
1117 |
temporary directory).
|
|
1118 |
|
|
1119 |
* sets a magic environment variable
|
|
1120 |
|
|
1121 |
* sets ``LD_LIBRARY_PATH`` (non-Windows)
|
|
1122 |
|
|
1123 |
* executes itself as a child process (letting the child use his stdin, stdout
|
|
1124 |
and stderr)
|
|
1125 |
|
|
1126 |
* waits for the child to exit (on \*nix, the child actually replaces the parent)
|
|
1127 |
|
|
1128 |
* cleans up the extracted binaries (so on \*nix, this is done by the child)
|
|
1129 |
|
|
1130 |
The child process executes as in `One Pass Execution`_ above (the magic
|
|
1131 |
environment variable is what tells it that this is pass two).
|
|
1132 |
|
|
1133 |
|SE_exeImage| figure 3 - Self Extracting Executable
|
|
1134 |
|
|
1135 |
There are, of course, quite a few differences between the Windows and
|
|
1136 |
Unix/Linux versions. The major one is that because all of Python on Windows is
|
|
1137 |
in ``pythonXX.dll``, and dynamic loading is so simple-minded, that one binary can
|
|
1138 |
be use with any version of Python. There's much in common, though, and that C
|
|
1139 |
code can be found in `source/common/launch.c`_.
|
|
1140 |
|
|
1141 |
The Unix/Linux build process (which you need to run just once for any version
|
|
1142 |
of Python) makes use of the config information in your install (if you
|
|
1143 |
installed from RPM, you need the Python-development RPM). It also overrides
|
|
1144 |
``getpath.c`` since we don't want it hunting around the filesystem to build
|
|
1145 |
``sys.path``.
|
|
1146 |
|
|
1147 |
In both cases, while one |PyInstaller| download can be used with any Python
|
|
1148 |
version, you need to have separate installations for each Python version.
|
|
1149 |
|
|
1150 |
|GOBACK|
|
|
1151 |
|
|
1152 |
PyInstaller Archives
|
|
1153 |
++++++++++++++++++++
|
|
1154 |
|
|
1155 |
Archives Introduction
|
|
1156 |
---------------------
|
|
1157 |
You know what an archive is: a ``.tar`` file, a ``.jar`` file, a ``.zip`` file. Two kinds
|
|
1158 |
of archives are used here. One is equivalent to a Java ``.jar`` file - it allows
|
|
1159 |
Python modules to be stored efficiently and, (with some import hooks) imported
|
|
1160 |
directly. This is a ``ZlibArchive``. The other (a ``CArchive``) is equivalent to a
|
|
1161 |
``.zip`` file - a general way of packing up (and optionally compressing) arbitrary
|
|
1162 |
blobs of data. It gets its name from the fact that it can be manipulated easily
|
|
1163 |
from C, as well as from Python. Both of these derive from a common base class,
|
|
1164 |
making it fairly easy to create new kinds of archives.
|
|
1165 |
|
|
1166 |
|GOBACK|
|
|
1167 |
|
|
1168 |
``ZlibArchive``
|
|
1169 |
---------------
|
|
1170 |
A ``ZlibArchive`` contains compressed ``.pyc`` (or ``.pyo``) files. The Table of Contents
|
|
1171 |
is a marshalled dictionary, with the key (the module's name as given in an
|
|
1172 |
``import`` statement) associated with a seek position and length. Because it is
|
|
1173 |
all marshalled Python, ``ZlibArchives`` are completely cross-platform.
|
|
1174 |
|
|
1175 |
A ``ZlibArchive`` hooks in with `iu.py`_ so that, with a little setup, the archived
|
|
1176 |
modules can be imported transparently. Even with compression at level 9, this
|
|
1177 |
works out to being faster than the normal import. Instead of searching
|
|
1178 |
``sys.path``, there's a lookup in the dictionary. There's no ``stat``-ing of the ``.py``
|
|
1179 |
and ``.pyc`` and no file opens (the file is already open). There's just a seek, a
|
|
1180 |
read and a decompress. A traceback will point to the source file the archive
|
|
1181 |
entry was created from (the ``__file__`` attribute from the time the ``.pyc`` was
|
|
1182 |
compiled). On a user's box with no source installed, this is not terribly
|
|
1183 |
useful, but if they send you the traceback, at least you can make sense of it.
|
|
1184 |
|
|
1185 |
|ZlibArchiveImage|
|
|
1186 |
|
|
1187 |
|GOBACK|
|
|
1188 |
|
|
1189 |
``CArchive``
|
|
1190 |
------------
|
|
1191 |
A ``CArchive`` contains whatever you want to stuff into it. It's very much like a
|
|
1192 |
``.zip`` file. They are easy to create in Python and unpack from C code. ``CArchives``
|
|
1193 |
can be appended to other files (like ELF and COFF executables, for example).
|
|
1194 |
To allow this, they are opened from the end, so the ``TOC`` for a ``CArchive`` is at
|
|
1195 |
the back, followed only by a cookie that tells you where the ``TOC`` starts and
|
|
1196 |
where the archive itself starts.
|
|
1197 |
|
|
1198 |
``CArchives`` can also be embedded within other ``CArchives``. The inner archive can be
|
|
1199 |
opened in place (without extraction).
|
|
1200 |
|
|
1201 |
Each ``TOC`` entry is variable length. The first field in the entry tells you the
|
|
1202 |
length of the entry. The last field is the name of the corresponding packed
|
|
1203 |
file. The name is null terminated. Compression is optional by member.
|
|
1204 |
|
|
1205 |
There is also a type code associated with each entry. If you're using a
|
|
1206 |
``CArchive`` as a ``.zip`` file, you don't need to worry about this. The type codes
|
|
1207 |
are used by the self-extracting executables.
|
|
1208 |
|
|
1209 |
|CArchiveImage|
|
|
1210 |
|
|
1211 |
|GOBACK|
|
|
1212 |
|
|
1213 |
|
|
1214 |
License
|
|
1215 |
+++++++
|
|
1216 |
PyInstaller is mainly distributed under the
|
|
1217 |
`GPL License <http://pyinstaller.hpcf.upr.edu/pyinstaller/browser/trunk/doc/LICENSE.GPL?rev=latest>`_
|
|
1218 |
but it has an exception such that you can use it to compile commercial products.
|
|
1219 |
|
|
1220 |
In a nutshell, the license is GPL for the source code with the exception that:
|
|
1221 |
|
|
1222 |
#. You may use PyInstaller to compile commercial applications out of your
|
|
1223 |
source code.
|
|
1224 |
|
|
1225 |
#. The resulting binaries generated by PyInstaller from your source code can be
|
|
1226 |
shipped with whatever license you want.
|
|
1227 |
|
|
1228 |
#. You may modify PyInstaller for your own needs but *these* changes to the
|
|
1229 |
PyInstaller source code falls under the terms of the GPL license. In other
|
|
1230 |
words, any modifications to will *have* to be distributed under GPL.
|
|
1231 |
|
|
1232 |
For updated information or clarification see our
|
|
1233 |
`FAQ <http://pyinstaller.hpcf.upr.edu/pyinstaller/wiki/FAQ>`_ at `PyInstaller`_
|
|
1234 |
home page: http://pyinstaller.hpcf.upr.edu
|
|
1235 |
|
|
1236 |
|
|
1237 |
|
|
1238 |
|GOBACK|
|
|
1239 |
|
|
1240 |
Appendix
|
|
1241 |
++++++++
|
|
1242 |
|
|
1243 |
.. sidebar:: You can stop reading here...
|
|
1244 |
|
|
1245 |
... if you are not interested in technical details. This appendix contains
|
|
1246 |
insights of the internal workings of |PyInstaller|, and you do not need this
|
|
1247 |
information unless you plan to work on |PyInstaller| itself.
|
|
1248 |
|
|
1249 |
|
|
1250 |
``mf.py``: A Modulefinder Replacement
|
|
1251 |
-------------------------------------
|
|
1252 |
|
|
1253 |
Module ``mf`` is modelled after ``iu``.
|
|
1254 |
|
|
1255 |
It also uses ``ImportDirectors`` and ``Owners`` to partition the import name space.
|
|
1256 |
Except for the fact that these return ``Module`` instances instead of real module
|
|
1257 |
objects, they are identical.
|
|
1258 |
|
|
1259 |
Instead of an ``ImportManager``, ``mf`` has an ``ImportTracker`` managing things.
|
|
1260 |
|
|
1261 |
|GOBACK|
|
|
1262 |
|
|
1263 |
ImportTracker
|
|
1264 |
*************
|
|
1265 |
|
|
1266 |
``ImportTracker`` can be called in two ways: ``analyze_one(name, importername=None)``
|
|
1267 |
or ``analyze_r(name, importername=None)``. The second method does what modulefinder
|
|
1268 |
does - it recursively finds all the module names that importing name would
|
|
1269 |
cause to appear in ``sys.modules``. The first method is non-recursive. This is
|
|
1270 |
useful, because it is the only way of answering the question "Who imports
|
|
1271 |
name?" But since it is somewhat unrealistic (very few real imports do not
|
|
1272 |
involve recursion), it deserves some explanation.
|
|
1273 |
|
|
1274 |
|GOBACK|
|
|
1275 |
|
|
1276 |
``analyze_one()``
|
|
1277 |
*****************
|
|
1278 |
|
|
1279 |
When a name is imported, there are structural and dynamic effects. The dynamic
|
|
1280 |
effects are due to the execution of the top-level code in the module (or
|
|
1281 |
modules) that get imported. The structural effects have to do with whether the
|
|
1282 |
import is relative or absolute, and whether the name is a dotted name (if there
|
|
1283 |
are N dots in the name, then N+1 modules will be imported even without any code
|
|
1284 |
running).
|
|
1285 |
|
|
1286 |
The analyze_one method determines the structural effects, and defers the
|
|
1287 |
dynamic effects. For example, ``analyze_one("B.C", "A")`` could return ``["B", "B.C"]``
|
|
1288 |
or ``["A.B", "A.B.C"]`` depending on whether the import turns out to be relative or
|
|
1289 |
absolute. In addition, ImportTracker's modules dict will have Module instances
|
|
1290 |
for them.
|
|
1291 |
|
|
1292 |
|GOBACK|
|
|
1293 |
|
|
1294 |
Module Classes
|
|
1295 |
**************
|
|
1296 |
|
|
1297 |
There are Module subclasses for builtins, extensions, packages and (normal)
|
|
1298 |
modules. Besides the normal module object attributes, they have an attribute
|
|
1299 |
imports. For packages and normal modules, imports is a list populated by
|
|
1300 |
scanning the code object (and therefor, the names in this list may be relative
|
|
1301 |
or absolute names - we don't know until they have been analyzed).
|
|
1302 |
|
|
1303 |
The highly astute will notice that there is a hole in ``analyze_one()`` here. The
|
|
1304 |
first thing that happens when ``B.C`` is being imported is that ``B`` is imported and
|
|
1305 |
it's top-level code executed. That top-level code can do various things so that
|
|
1306 |
when the import of ``B.C`` finally occurs, something completely different happens
|
|
1307 |
(from what a structural analysis would predict). But mf can handle this through
|
|
1308 |
it's hooks mechanism.
|
|
1309 |
|
|
1310 |
|GOBACK|
|
|
1311 |
|
|
1312 |
code scanning
|
|
1313 |
*************
|
|
1314 |
|
|
1315 |
Like modulefinder, ``mf`` scans the byte code of a module, looking for imports. In
|
|
1316 |
addition, ``mf`` will pick out a module's ``__all__`` attribute, if it is built as a
|
|
1317 |
list of constant names. This means that if a package declares an ``__all__`` list
|
|
1318 |
as a list of names, ImportTracker will track those names if asked to analyze
|
|
1319 |
``package.*``. The code scan also notes the occurance of ``__import__``, ``exec`` and ``eval``,
|
|
1320 |
and can issue warnings when they're found.
|
|
1321 |
|
|
1322 |
The code scanning also keeps track (as well as it can) of the context of an
|
|
1323 |
import. It recognizes when imports are found at the top-level, and when they
|
|
1324 |
are found inside definitions (deferred imports). Within that, it also tracks
|
|
1325 |
whether the import is inside a condition (conditional imports).
|
|
1326 |
|
|
1327 |
|GOBACK|
|
|
1328 |
|
|
1329 |
Hooks
|
|
1330 |
*****
|
|
1331 |
|
|
1332 |
In modulefinder, scanning the code takes the place of executing the code
|
|
1333 |
object. ``mf`` goes further and allows a module to be hooked (after it has been
|
|
1334 |
scanned, but before analyze_one is done with it). A hook is a module named
|
|
1335 |
``hook-fullyqualifiedname`` in the ``hooks`` package. These modules should have one or
|
|
1336 |
more of the following three global names defined:
|
|
1337 |
|
|
1338 |
``hiddenimports``
|
|
1339 |
a list of modules names (relative or absolute) that the module imports in some untrackable way.
|
|
1340 |
|
|
1341 |
``attrs``
|
|
1342 |
a list of ``(name, value)`` pairs (where value is normally meaningless).
|
|
1343 |
|
|
1344 |
``hook(mod)``
|
|
1345 |
a function taking a ``Module`` instance and returning a ``Module`` instance (so it can modify or replace).
|
|
1346 |
|
|
1347 |
|
|
1348 |
The first hook (``hiddenimports``) extends the list created by scanning the code.
|
|
1349 |
``ExtensionModules``, of course, don't get scanned, so this is the only way of
|
|
1350 |
recording any imports they do.
|
|
1351 |
|
|
1352 |
The second hook (``attrs``) exists mainly so that ImportTracker won't issue
|
|
1353 |
spurious warnings when the rightmost node in a dotted name turns out to be an
|
|
1354 |
attribute in a package module, instead of a missing submodule.
|
|
1355 |
|
|
1356 |
The callable hook exists for things like dynamic modification of a package's
|
|
1357 |
``__path__`` or perverse situations, like ``xml.__init__`` replacing itself in
|
|
1358 |
``sys.modules`` with ``_xmlplus.__init__``. (It takes nine hook modules to properly
|
|
1359 |
trace through PyXML-using code, and I can't believe that it's any easier for
|
|
1360 |
the poor programmer using that package). The ``hook(mod)`` (if it exists) is
|
|
1361 |
called before looking at the others - that way it can, for example, test
|
|
1362 |
``sys.version`` and adjust what's in ``hiddenimports``.
|
|
1363 |
|
|
1364 |
|GOBACK|
|
|
1365 |
|
|
1366 |
Warnings
|
|
1367 |
********
|
|
1368 |
|
|
1369 |
``ImportTracker`` has a ``getwarnings()`` method that returns all the warnings
|
|
1370 |
accumulated by the instance, and by the ``Module`` instances in its modules dict.
|
|
1371 |
Generally, it is ``ImportTracker`` who will accumulate the warnings generated
|
|
1372 |
during the structural phase, and ``Modules`` that will get the warnings generated
|
|
1373 |
during the code scan.
|
|
1374 |
|
|
1375 |
Note that by using a hook module, you can silence some particularly tiresome
|
|
1376 |
warnings, but not all of them.
|
|
1377 |
|
|
1378 |
|GOBACK|
|
|
1379 |
|
|
1380 |
Cross Reference
|
|
1381 |
***************
|
|
1382 |
|
|
1383 |
Once a full analysis (that is, an ``analyze_r`` call) has been done, you can get a
|
|
1384 |
cross reference by using ``getxref()``. This returns a list of tuples. Each tuple
|
|
1385 |
is ``(modulename, importers)``, where importers is a list of the (fully qualified)
|
|
1386 |
names of the modules importing ``modulename``. Both the returned list and the
|
|
1387 |
importers list are sorted.
|
|
1388 |
|
|
1389 |
|GOBACK|
|
|
1390 |
|
|
1391 |
Usage
|
|
1392 |
*****
|
|
1393 |
|
|
1394 |
A simple example follows:
|
|
1395 |
|
|
1396 |
>>> import mf
|
|
1397 |
>>> a = mf.ImportTracker()
|
|
1398 |
>>> a.analyze_r("os")
|
|
1399 |
['os', 'sys', 'posixpath', 'nt', 'stat', 'string', 'strop',
|
|
1400 |
're', 'pcre', 'ntpath', 'dospath', 'macpath', 'win32api',
|
|
1401 |
'UserDict', 'copy', 'types', 'repr', 'tempfile']
|
|
1402 |
>>> a.analyze_one("os")
|
|
1403 |
['os']
|
|
1404 |
>>> a.modules['string'].imports
|
|
1405 |
[('strop', 0, 0), ('strop.*', 0, 0), ('re', 1, 1)]
|
|
1406 |
>>>
|
|
1407 |
|
|
1408 |
|
|
1409 |
The tuples in the imports list are (name, delayed, conditional).
|
|
1410 |
|
|
1411 |
>>> for w in a.modules['string'].warnings: print w
|
|
1412 |
...
|
|
1413 |
W: delayed eval hack detected at line 359
|
|
1414 |
W: delayed eval hack detected at line 389
|
|
1415 |
W: delayed eval hack detected at line 418
|
|
1416 |
>>> for w in a.getwarnings(): print w
|
|
1417 |
...
|
|
1418 |
W: no module named pwd (delayed, conditional import by posixpath)
|
|
1419 |
W: no module named dos (conditional import by os)
|
|
1420 |
W: no module named os2 (conditional import by os)
|
|
1421 |
W: no module named posix (conditional import by os)
|
|
1422 |
W: no module named mac (conditional import by os)
|
|
1423 |
W: no module named MACFS (delayed, conditional import by tempfile)
|
|
1424 |
W: no module named macfs (delayed, conditional import by tempfile)
|
|
1425 |
W: top-level conditional exec statment detected at line 47
|
|
1426 |
- os (C:\Program Files\Python\Lib\os.py)
|
|
1427 |
W: delayed eval hack detected at line 359
|
|
1428 |
- string (C:\Program Files\Python\Lib\string.py)
|
|
1429 |
W: delayed eval hack detected at line 389
|
|
1430 |
- string (C:\Program Files\Python\Lib\string.py)
|
|
1431 |
W: delayed eval hack detected at line 418
|
|
1432 |
- string (C:\Program Files\Python\Lib\string.py)
|
|
1433 |
>>>
|
|
1434 |
|
|
1435 |
|
|
1436 |
|GOBACK|
|
|
1437 |
|
|
1438 |
|
|
1439 |
.. _iu.py:
|
|
1440 |
|
|
1441 |
``iu.py``: An *imputil* Replacement
|
|
1442 |
-----------------------------------
|
|
1443 |
|
|
1444 |
Module ``iu`` grows out of the pioneering work that Greg Stein did with ``imputil``
|
|
1445 |
(actually, it includes some verbatim ``imputil`` code, but since Greg didn't
|
|
1446 |
copyright it, we won't mention it). Both modules can take over Python's
|
|
1447 |
builtin import and ease writing of at least certain kinds of import hooks.
|
|
1448 |
|
|
1449 |
``iu`` differs from ``imputil``:
|
|
1450 |
* faster
|
|
1451 |
* better emulation of builtin import
|
|
1452 |
* more managable
|
|
1453 |
|
|
1454 |
There is an ``ImportManager`` which provides the replacement for builtin import
|
|
1455 |
and hides all the semantic complexities of a Python import request from it's
|
|
1456 |
delegates.
|
|
1457 |
|
|
1458 |
|GOBACK|
|
|
1459 |
|
|
1460 |
``ImportManager``
|
|
1461 |
*****************
|
|
1462 |
|
|
1463 |
``ImportManager`` formalizes the concept of a metapath. This concept implicitly
|
|
1464 |
exists in native Python in that builtins and frozen modules are searched
|
|
1465 |
before ``sys.path``, (on Windows there's also a search of the registry while on
|
|
1466 |
Mac, resources may be searched). This metapath is a list populated with
|
|
1467 |
``ImportDirector`` instances. There are ``ImportDirector`` subclasses for builtins,
|
|
1468 |
frozen modules, (on Windows) modules found through the registry and a
|
|
1469 |
``PathImportDirector`` for handling ``sys.path``. For a top-level import (that is, not
|
|
1470 |
an import of a module in a package), ``ImportManager`` tries each director on it's
|
|
1471 |
metapath until one succeeds.
|
|
1472 |
|
|
1473 |
``ImportManager`` hides the semantic complexity of an import from the directors.
|
|
1474 |
It's up to the ``ImportManager`` to decide if an import is relative or absolute;
|
|
1475 |
to see if the module has already been imported; to keep ``sys.modules`` up to
|
|
1476 |
date; to handle the fromlist and return the correct module object.
|
|
1477 |
|
|
1478 |
|GOBACK|
|
|
1479 |
|
|
1480 |
``ImportDirector``
|
|
1481 |
******************
|
|
1482 |
|
|
1483 |
An ``ImportDirector`` just needs to respond to ``getmod(name)`` by returning a module
|
|
1484 |
object or ``None``. As you will see, an ``ImportDirector`` can consider name to be
|
|
1485 |
atomic - it has no need to examine name to see if it is dotted.
|
|
1486 |
|
|
1487 |
To see how this works, we need to examine the ``PathImportDirector``.
|
|
1488 |
|
|
1489 |
|GOBACK|
|
|
1490 |
|
|
1491 |
``PathImportDirector``
|
|
1492 |
**********************
|
|
1493 |
|
|
1494 |
The ``PathImportDirector`` subclass manages a list of names - most notably,
|
|
1495 |
``sys.path``. To do so, it maintains a shadowpath - a dictionary mapping the names
|
|
1496 |
on its pathlist (eg, ``sys.path``) to their associated ``Owners``. (It could do this
|
|
1497 |
directly, but the assumption that sys.path is occupied solely by strings seems
|
|
1498 |
ineradicable.) ``Owners`` of the appropriate kind are created as needed (if all
|
|
1499 |
your imports are satisfied by the first two elements of ``sys.path``, the
|
|
1500 |
``PathImportDirector``'s shadowpath will only have two entries).
|
|
1501 |
|
|
1502 |
|GOBACK|
|
|
1503 |
|
|
1504 |
``Owner``
|
|
1505 |
*********
|
|
1506 |
|
|
1507 |
An ``Owner`` is much like an ``ImportDirector`` but manages a much more concrete piece
|
|
1508 |
of turf. For example, a ``DirOwner`` manages one directory. Since there are no
|
|
1509 |
other officially recognized filesystem-like namespaces for importing, that's
|
|
1510 |
all that's included in iu, but it's easy to imagine ``Owners`` for zip files
|
|
1511 |
(and I have one for my own ``.pyz`` archive format) or even URLs.
|
|
1512 |
|
|
1513 |
As with ``ImportDirectors``, an ``Owner`` just needs to respond to ``getmod(name)`` by
|
|
1514 |
returning a module object or ``None``, and it can consider name to be atomic.
|
|
1515 |
|
|
1516 |
So structurally, we have a tree, rooted at the ``ImportManager``. At the next
|
|
1517 |
level, we have a set of ``ImportDirectors``. At least one of those directors, the
|
|
1518 |
``PathImportDirector`` in charge of ``sys.path``, has another level beneath it,
|
|
1519 |
consisting of ``Owners``. This much of the tree covers the entire top-level import
|
|
1520 |
namespace.
|
|
1521 |
|
|
1522 |
The rest of the import namespace is covered by treelets, each rooted in a
|
|
1523 |
package module (an ``__init__.py``).
|
|
1524 |
|
|
1525 |
|GOBACK|
|
|
1526 |
|
|
1527 |
Packages
|
|
1528 |
********
|
|
1529 |
|
|
1530 |
To make this work, ``Owners`` need to recognize when a module is a package. For a
|
|
1531 |
``DirOwner``, this means that name is a subdirectory which contains an ``__init__.py``.
|
|
1532 |
The ``__init__`` module is loaded and its ``__path__`` is initialized with the
|
|
1533 |
subdirectory. Then, a ``PathImportDirector`` is created to manage this ``__path__``.
|
|
1534 |
Finally the new ``PathImportDirector``'s ``getmod`` is assigned to the package's
|
|
1535 |
``__importsub__`` function.
|
|
1536 |
|
|
1537 |
When a module within the package is imported, the request is routed (by the
|
|
1538 |
``ImportManager``) diretly to the package's ``__importsub__``. In a hierarchical
|
|
1539 |
namespace (like a filesystem), this means that ``__importsub__`` (which is really
|
|
1540 |
the bound getmod method of a ``PathImportDirector`` instance) needs only the
|
|
1541 |
module name, not the package name or the fully qualified name. And that's
|
|
1542 |
exactly what it gets. (In a flat namespace - like most archives - it is
|
|
1543 |
perfectly easy to route the request back up the package tree to the archive
|
|
1544 |
``Owner``, qualifying the name at each step.)
|
|
1545 |
|
|
1546 |
|GOBACK|
|
|
1547 |
|
|
1548 |
Possibilities
|
|
1549 |
*************
|
|
1550 |
|
|
1551 |
Let's say we want to import from zip files. So, we subclass ``Owner``. The
|
|
1552 |
``__init__`` method should take a filename, and raise a ``ValueError`` if the file is
|
|
1553 |
not an acceptable ``.zip`` file, (when a new name is encountered on ``sys.path`` or a
|
|
1554 |
package's ``__path__``, registered Owners are tried until one accepts the name).
|
|
1555 |
The ``getmod`` method would check the zip file's contents and return ``None`` if the
|
|
1556 |
name is not found. Otherwise, it would extract the marshalled code object from
|
|
1557 |
the zip, create a new module object and perform a bit of initialization (12
|
|
1558 |
lines of code all told for my own archive format, including initializing a pack
|
|
1559 |
age with it's ``__subimporter__``).
|
|
1560 |
|
|
1561 |
Once the new ``Owner`` class is registered with ``iu``, you can put a zip file on
|
|
1562 |
``sys.path``. A package could even put a zip file on its ``__path__``.
|
|
1563 |
|
|
1564 |
|GOBACK|
|
|
1565 |
|
|
1566 |
Compatibility
|
|
1567 |
*************
|
|
1568 |
|
|
1569 |
This code has been tested with the PyXML, mxBase and Win32 packages, covering
|
|
1570 |
over a dozen import hacks from manipulations of ``__path__`` to replacing a module
|
|
1571 |
in ``sys.modules`` with a different one. Emulation of Python's native import is
|
|
1572 |
nearly exact, including the names recorded in ``sys.modules`` and module attributes
|
|
1573 |
(packages imported through ``iu`` have an extra attribute - ``__importsub__``).
|
|
1574 |
|
|
1575 |
|GOBACK|
|
|
1576 |
|
|
1577 |
Performance
|
|
1578 |
***********
|
|
1579 |
|
|
1580 |
In most cases, ``iu`` is slower than builtin import (by 15 to 20%) but faster than
|
|
1581 |
``imputil`` (by 15 to 20%). By inserting archives at the front of ``sys.path``
|
|
1582 |
containing the standard lib and the package being tested, this can be reduced
|
|
1583 |
to 5 to 10% slower (or, on my 1.52 box, 10% faster!) than builtin import. A bit
|
|
1584 |
more can be shaved off by manipulating the ``ImportManager``'s metapath.
|
|
1585 |
|
|
1586 |
|GOBACK|
|
|
1587 |
|
|
1588 |
Limitations
|
|
1589 |
***********
|
|
1590 |
|
|
1591 |
This module makes no attempt to facilitate policy import hacks. It is easy to
|
|
1592 |
implement certain kinds of policies within a particular domain, but
|
|
1593 |
fundamentally iu works by dividing up the import namespace into independent
|
|
1594 |
domains.
|
|
1595 |
|
|
1596 |
Quite simply, I think cross-domain import hacks are a very bad idea. As author
|
|
1597 |
of the original package on which |PyInstaller| is based, McMillan worked with
|
|
1598 |
import hacks for many years. Many of them are highly fragile; they often rely
|
|
1599 |
on undocumented (maybe even accidental) features of implementation.
|
|
1600 |
A cross-domain import hack is not likely to work with PyXML, for example.
|
|
1601 |
|
|
1602 |
That rant aside, you can modify ``ImportManger`` to implement different policies.
|
|
1603 |
For example, a version that implements three import primitives: absolute
|
|
1604 |
import, relative import and recursive-relative import. No idea what the Python
|
|
1605 |
syntax for those should be, but ``__aimport__``, ``__rimport__`` and ``__rrimport__`` were
|
|
1606 |
easy to implement.
|
|
1607 |
|
|
1608 |
|
|
1609 |
Usage
|
|
1610 |
*****
|
|
1611 |
|
|
1612 |
Here's a simple example of using ``iu`` as a builtin import replacement.
|
|
1613 |
|
|
1614 |
>>> import iu
|
|
1615 |
>>> iu.ImportManager().install()
|
|
1616 |
>>>
|
|
1617 |
>>> import DateTime
|
|
1618 |
>>> DateTime.__importsub__
|
|
1619 |
<method PathImportDirector.getmod
|
|
1620 |
of PathImportDirector instance at 825900>
|
|
1621 |
>>>
|
|
1622 |
|
|
1623 |
|GOBACK|
|
|
1624 |
|
|
1625 |
.. _PyInstaller: http://pyinstaller.hpcf.upr.edu/pyinstaller
|
|
1626 |
.. _Roadmap: http://pyinstaller.hpcf.upr.edu/pyinstaller/roadmap
|
|
1627 |
.. _`Submit a Bug`: http://pyinstaller.hpcf.upr.edu/pyinstaller/newticket
|
|
1628 |
.. _Scons: http://www.scons.org
|
|
1629 |
.. _hooks\/hook-win32com.py: http://pyinstaller.hpcf.upr.edu/pyinstaller/browser/trunk/hooks/hook-win32com.py?rev=latest
|
|
1630 |
.. _support\/unpackTK.py: http://pyinstaller.hpcf.upr.edu/pyinstaller/browser/trunk/support/unpackTK.py?rev=latest
|
|
1631 |
.. _source/common/launch.c: http://pyinstaller.hpcf.upr.edu/pyinstaller/browser/trunk/source/common/launch.c?rev=latest
|
|
1632 |
.. _Pmw: http://pmw.sourceforge.net/
|
|
1633 |
.. _PIL: http://www.pythonware.com/products/pil/
|
|
1634 |
.. _PyQt: http://www.riverbankcomputing.co.uk/pyqt/index.php
|
|
1635 |
.. _PyWin32: http://starship.python.net/crew/mhammond/win32/
|
|
1636 |
.. |ZlibArchiveImage| image:: images/ZlibArchive.png
|
|
1637 |
.. |CArchiveImage| image:: images/CArchive.png
|
|
1638 |
.. |SE_exeImage| image:: images/SE_exe.png
|
|
1639 |
.. |PyInstaller| replace:: PyInstaller
|
|
1640 |
.. |PyInstallerVersion| replace:: PyInstaller v1.0
|
|
1641 |
.. |InitialVersion| replace:: v1.0
|
|
1642 |
.. |install_path| replace:: /your/path/to/pyinstaller/
|
|
1643 |
.. |GOBACK| replace:: `Back to Top`_
|
|
1644 |
.. _`Back to Top`: `PyInstaller Manual`_
|