--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/symbian-qemu-0.9.1-12/python-2.6.1/Tools/pybench/README Fri Jul 31 15:01:17 2009 +0100
@@ -0,0 +1,368 @@
+________________________________________________________________________
+
+PYBENCH - A Python Benchmark Suite
+________________________________________________________________________
+
+ Extendable suite of of low-level benchmarks for measuring
+ the performance of the Python implementation
+ (interpreter, compiler or VM).
+
+pybench is a collection of tests that provides a standardized way to
+measure the performance of Python implementations. It takes a very
+close look at different aspects of Python programs and let's you
+decide which factors are more important to you than others, rather
+than wrapping everything up in one number, like the other performance
+tests do (e.g. pystone which is included in the Python Standard
+Library).
+
+pybench has been used in the past by several Python developers to
+track down performance bottlenecks or to demonstrate the impact of
+optimizations and new features in Python.
+
+The command line interface for pybench is the file pybench.py. Run
+this script with option '--help' to get a listing of the possible
+options. Without options, pybench will simply execute the benchmark
+and then print out a report to stdout.
+
+
+Micro-Manual
+------------
+
+Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run
+the benchmark suite using default settings and 'pybench.py -f <file>'
+to have it store the results in a file too.
+
+It is usually a good idea to run pybench.py multiple times to see
+whether the environment, timers and benchmark run-times are suitable
+for doing benchmark tests.
+
+You can use the comparison feature of pybench.py ('pybench.py -c
+<file>') to check how well the system behaves in comparison to a
+reference run.
+
+If the differences are well below 10% for each test, then you have a
+system that is good for doing benchmark testings. Of you get random
+differences of more than 10% or significant differences between the
+values for minimum and average time, then you likely have some
+background processes running which cause the readings to become
+inconsistent. Examples include: web-browsers, email clients, RSS
+readers, music players, backup programs, etc.
+
+If you are only interested in a few tests of the whole suite, you can
+use the filtering option, e.g. 'pybench.py -t string' will only
+run/show the tests that have 'string' in their name.
+
+This is the current output of pybench.py --help:
+
+"""
+------------------------------------------------------------------------
+PYBENCH - a benchmark test suite for Python interpreters/compilers.
+------------------------------------------------------------------------
+
+Synopsis:
+ pybench.py [option] files...
+
+Options and default settings:
+ -n arg number of rounds (10)
+ -f arg save benchmark to file arg ()
+ -c arg compare benchmark with the one in file arg ()
+ -s arg show benchmark in file arg, then exit ()
+ -w arg set warp factor to arg (10)
+ -t arg run only tests with names matching arg ()
+ -C arg set the number of calibration runs to arg (20)
+ -d hide noise in comparisons (0)
+ -v verbose output (not recommended) (0)
+ --with-gc enable garbage collection (0)
+ --with-syscheck use default sys check interval (0)
+ --timer arg use given timer (time.time)
+ -h show this help text
+ --help show this help text
+ --debug enable debugging
+ --copyright show copyright
+ --examples show examples of usage
+
+Version:
+ 2.0
+
+The normal operation is to run the suite and display the
+results. Use -f to save them for later reuse or comparisons.
+
+Available timers:
+
+ time.time
+ time.clock
+ systimes.processtime
+
+Examples:
+
+python2.1 pybench.py -f p21.pybench
+python2.5 pybench.py -f p25.pybench
+python pybench.py -s p25.pybench -c p21.pybench
+"""
+
+License
+-------
+
+See LICENSE file.
+
+
+Sample output
+-------------
+
+"""
+-------------------------------------------------------------------------------
+PYBENCH 2.0
+-------------------------------------------------------------------------------
+* using Python 2.4.2
+* disabled garbage collection
+* system check interval set to maximum: 2147483647
+* using timer: time.time
+
+Calibrating tests. Please wait...
+
+Running 10 round(s) of the suite at warp factor 10:
+
+* Round 1 done in 6.388 seconds.
+* Round 2 done in 6.485 seconds.
+* Round 3 done in 6.786 seconds.
+...
+* Round 10 done in 6.546 seconds.
+
+-------------------------------------------------------------------------------
+Benchmark: 2006-06-12 12:09:25
+-------------------------------------------------------------------------------
+
+ Rounds: 10
+ Warp: 10
+ Timer: time.time
+
+ Machine Details:
+ Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64
+ Processor: x86_64
+
+ Python:
+ Executable: /usr/local/bin/python
+ Version: 2.4.2
+ Compiler: GCC 3.3.4 (pre 3.3.5 20040809)
+ Bits: 64bit
+ Build: Oct 1 2005 15:24:35 (#1)
+ Unicode: UCS2
+
+
+Test minimum average operation overhead
+-------------------------------------------------------------------------------
+ BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms
+ BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms
+ CompareFloats: 109ms 110ms 0.09us 0.361ms
+ CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms
+ CompareIntegers: 137ms 138ms 0.08us 0.542ms
+ CompareInternedStrings: 124ms 127ms 0.08us 1.367ms
+ CompareLongs: 100ms 104ms 0.10us 0.316ms
+ CompareStrings: 111ms 115ms 0.12us 0.929ms
+ CompareUnicode: 108ms 128ms 0.17us 0.693ms
+ ConcatStrings: 142ms 155ms 0.31us 0.562ms
+ ConcatUnicode: 119ms 127ms 0.42us 0.384ms
+ CreateInstances: 123ms 128ms 1.14us 0.367ms
+ CreateNewInstances: 121ms 126ms 1.49us 0.335ms
+ CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms
+ CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms
+ DictCreation: 108ms 109ms 0.27us 0.361ms
+ DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms
+ DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms
+ DictWithStringKeys: 114ms 117ms 0.10us 0.905ms
+ ForLoops: 110ms 111ms 4.46us 0.063ms
+ IfThenElse: 118ms 119ms 0.09us 0.685ms
+ ListSlicing: 116ms 120ms 8.59us 0.103ms
+ NestedForLoops: 125ms 137ms 0.09us 0.019ms
+ NormalClassAttribute: 124ms 136ms 0.11us 0.457ms
+ NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms
+ PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms
+ PythonMethodCalls: 140ms 149ms 0.66us 0.141ms
+ Recursion: 156ms 166ms 3.32us 0.452ms
+ SecondImport: 112ms 118ms 1.18us 0.180ms
+ SecondPackageImport: 118ms 127ms 1.27us 0.180ms
+ SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms
+ SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms
+ SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms
+ SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms
+ SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms
+ SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms
+ SimpleListManipulation: 103ms 113ms 0.10us 0.587ms
+ SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms
+ SmallLists: 105ms 116ms 0.17us 0.366ms
+ SmallTuples: 108ms 128ms 0.24us 0.406ms
+ SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms
+ SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms
+ StringMappings: 115ms 121ms 0.48us 0.405ms
+ StringPredicates: 120ms 129ms 0.18us 2.064ms
+ StringSlicing: 111ms 127ms 0.23us 0.781ms
+ TryExcept: 125ms 126ms 0.06us 0.681ms
+ TryRaiseExcept: 133ms 137ms 2.14us 0.361ms
+ TupleSlicing: 117ms 120ms 0.46us 0.066ms
+ UnicodeMappings: 156ms 160ms 4.44us 0.429ms
+ UnicodePredicates: 117ms 121ms 0.22us 2.487ms
+ UnicodeProperties: 115ms 153ms 0.38us 2.070ms
+ UnicodeSlicing: 126ms 129ms 0.26us 0.689ms
+-------------------------------------------------------------------------------
+Totals: 6283ms 6673ms
+"""
+________________________________________________________________________
+
+Writing New Tests
+________________________________________________________________________
+
+pybench tests are simple modules defining one or more pybench.Test
+subclasses.
+
+Writing a test essentially boils down to providing two methods:
+.test() which runs .rounds number of .operations test operations each
+and .calibrate() which does the same except that it doesn't actually
+execute the operations.
+
+
+Here's an example:
+------------------
+
+from pybench import Test
+
+class IntegerCounting(Test):
+
+ # Version number of the test as float (x.yy); this is important
+ # for comparisons of benchmark runs - tests with unequal version
+ # number will not get compared.
+ version = 1.0
+
+ # The number of abstract operations done in each round of the
+ # test. An operation is the basic unit of what you want to
+ # measure. The benchmark will output the amount of run-time per
+ # operation. Note that in order to raise the measured timings
+ # significantly above noise level, it is often required to repeat
+ # sets of operations more than once per test round. The measured
+ # overhead per test round should be less than 1 second.
+ operations = 20
+
+ # Number of rounds to execute per test run. This should be
+ # adjusted to a figure that results in a test run-time of between
+ # 1-2 seconds (at warp 1).
+ rounds = 100000
+
+ def test(self):
+
+ """ Run the test.
+
+ The test needs to run self.rounds executing
+ self.operations number of operations each.
+
+ """
+ # Init the test
+ a = 1
+
+ # Run test rounds
+ #
+ # NOTE: Use xrange() for all test loops unless you want to face
+ # a 20MB process !
+ #
+ for i in xrange(self.rounds):
+
+ # Repeat the operations per round to raise the run-time
+ # per operation significantly above the noise level of the
+ # for-loop overhead.
+
+ # Execute 20 operations (a += 1):
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+ a += 1
+
+ def calibrate(self):
+
+ """ Calibrate the test.
+
+ This method should execute everything that is needed to
+ setup and run the test - except for the actual operations
+ that you intend to measure. pybench uses this method to
+ measure the test implementation overhead.
+
+ """
+ # Init the test
+ a = 1
+
+ # Run test rounds (without actually doing any operation)
+ for i in xrange(self.rounds):
+
+ # Skip the actual execution of the operations, since we
+ # only want to measure the test's administration overhead.
+ pass
+
+Registering a new test module
+-----------------------------
+
+To register a test module with pybench, the classes need to be
+imported into the pybench.Setup module. pybench will then scan all the
+symbols defined in that module for subclasses of pybench.Test and
+automatically add them to the benchmark suite.
+
+
+Breaking Comparability
+----------------------
+
+If a change is made to any individual test that means it is no
+longer strictly comparable with previous runs, the '.version' class
+variable should be updated. Therefafter, comparisons with previous
+versions of the test will list as "n/a" to reflect the change.
+
+
+Version History
+---------------
+
+ 2.0: rewrote parts of pybench which resulted in more repeatable
+ timings:
+ - made timer a parameter
+ - changed the platform default timer to use high-resolution
+ timers rather than process timers (which have a much lower
+ resolution)
+ - added option to select timer
+ - added process time timer (using systimes.py)
+ - changed to use min() as timing estimator (average
+ is still taken as well to provide an idea of the difference)
+ - garbage collection is turned off per default
+ - sys check interval is set to the highest possible value
+ - calibration is now a separate step and done using
+ a different strategy that allows measuring the test
+ overhead more accurately
+ - modified the tests to each give a run-time of between
+ 100-200ms using warp 10
+ - changed default warp factor to 10 (from 20)
+ - compared results with timeit.py and confirmed measurements
+ - bumped all test versions to 2.0
+ - updated platform.py to the latest version
+ - changed the output format a bit to make it look
+ nicer
+ - refactored the APIs somewhat
+ 1.3+: Steve Holden added the NewInstances test and the filtering
+ option during the NeedForSpeed sprint; this also triggered a long
+ discussion on how to improve benchmark timing and finally
+ resulted in the release of 2.0
+ 1.3: initial checkin into the Python SVN repository
+
+
+Have fun,
+--
+Marc-Andre Lemburg
+mal@lemburg.com