1 #! /usr/bin/env python

     2 

     3 """Regression test.

     4 

     5 This will find all modules whose name is "test_*" in the test

     6 directory, and run them.  Various command line options provide

     7 additional facilities.

     8 

     9 Command line options:

    10 

    11 -v: verbose    -- run tests in verbose mode with output to stdout

    12 -w: verbose2   -- re-run failed tests in verbose mode

    13 -q: quiet      -- don't print anything except if a test fails

    14 -x: exclude    -- arguments are tests to *exclude*

    15 -s: single     -- run only a single test (see below)

    16 -S: slow       -- print the slowest 10 tests

    17 -r: random     -- randomize test execution order

    18 -f: fromfile   -- read names of tests to run from a file (see below)

    19 -l: findleaks  -- if GC is available detect tests that leak memory

    20 -u: use        -- specify which special resource intensive tests to run

    21 -h: help       -- print this text and exit

    22 -t: threshold  -- call gc.set_threshold(N)

    23 -T: coverage   -- turn on code coverage using the trace module

    24 -D: coverdir   -- Directory where coverage files are put

    25 -N: nocoverdir -- Put coverage files alongside modules

    26 -L: runleaks   -- run the leaks(1) command just before exit

    27 -R: huntrleaks -- search for reference leaks (needs debug build, v. slow)

    28 -M: memlimit   -- run very large memory-consuming tests

    29 

    30 If non-option arguments are present, they are names for tests to run,

    31 unless -x is given, in which case they are names for tests not to run.

    32 If no test names are given, all tests are run.

    33 

    34 -T turns on code coverage tracing with the trace module.

    35 

    36 -D specifies the directory where coverage files are put.

    37 

    38 -N Put coverage files alongside modules.

    39 

    40 -s means to run only a single test and exit.  This is useful when

    41 doing memory analysis on the Python interpreter (which tend to consume

    42 too many resources to run the full regression test non-stop).  The

    43 file /tmp/pynexttest is read to find the next test to run.  If this

    44 file is missing, the first test_*.py file in testdir or on the command

    45 line is used.  (actually tempfile.gettempdir() is used instead of

    46 /tmp).

    47 

    48 -f reads the names of tests from the file given as f's argument, one

    49 or more test names per line.  Whitespace is ignored.  Blank lines and

    50 lines beginning with '#' are ignored.  This is especially useful for

    51 whittling down failures involving interactions among tests.

    52 

    53 -L causes the leaks(1) command to be run just before exit if it exists.

    54 leaks(1) is available on Mac OS X and presumably on some other

    55 FreeBSD-derived systems.

    56 

    57 -R runs each test several times and examines sys.gettotalrefcount() to

    58 see if the test appears to be leaking references.  The argument should

    59 be of the form stab:run:fname where 'stab' is the number of times the

    60 test is run to let gettotalrefcount settle down, 'run' is the number

    61 of times further it is run and 'fname' is the name of the file the

    62 reports are written to.  These parameters all have defaults (5, 4 and

    63 "reflog.txt" respectively), so the minimal invocation is '-R ::'.

    64 

    65 -M runs tests that require an exorbitant amount of memory. These tests

    66 typically try to ascertain containers keep working when containing more than

    67 2 billion objects, which only works on 64-bit systems. There are also some

    68 tests that try to exhaust the address space of the process, which only makes

    69 sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,

    70 which is a string in the form of '2.5Gb', determines howmuch memory the

    71 tests will limit themselves to (but they may go slightly over.) The number

    72 shouldn't be more memory than the machine has (including swap memory). You

    73 should also keep in mind that swap memory is generally much, much slower

    74 than RAM, and setting memlimit to all available RAM or higher will heavily

    75 tax the machine. On the other hand, it is no use running these tests with a

    76 limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect

    77 to use more than memlimit memory will be skipped. The big-memory tests

    78 generally run very, very long.

    79 

    80 -u is used to specify which special resource intensive tests to run,

    81 such as those requiring large file support or network connectivity.

    82 The argument is a comma-separated list of words indicating the

    83 resources to test.  Currently only the following are defined:

    84 

    85     all -       Enable all special resources.

    86 

    87     audio -     Tests that use the audio device.  (There are known

    88                 cases of broken audio drivers that can crash Python or

    89                 even the Linux kernel.)

    90 

    91     curses -    Tests that use curses and will modify the terminal's

    92                 state and output modes.

    93 

    94     lib2to3 -   Run the tests for 2to3 (They take a while.)

    95 

    96     largefile - It is okay to run some test that may create huge

    97                 files.  These tests can take a long time and may

    98                 consume >2GB of disk space temporarily.

    99 

   100     network -   It is okay to run tests that use external network

   101                 resource, e.g. testing SSL support for sockets.

   102 

   103     bsddb -     It is okay to run the bsddb testsuite, which takes

   104                 a long time to complete.

   105 

   106     decimal -   Test the decimal module against a large suite that

   107                 verifies compliance with standards.

   108 

   109     compiler -  Test the compiler package by compiling all the source

   110                 in the standard library and test suite.  This takes

   111                 a long time.  Enabling this resource also allows

   112                 test_tokenize to verify round-trip lexing on every

   113                 file in the test library.

   114 

   115     subprocess  Run all tests for the subprocess module.

   116 

   117     urlfetch -  It is okay to download files required on testing.

   118 

   119 To enable all resources except one, use '-uall,-<resource>'.  For

   120 example, to run all the tests except for the bsddb tests, give the

   121 option '-uall,-bsddb'.

   122 """

   123 

   124 import cStringIO

   125 import getopt

   126 import os

   127 import random

   128 import re

   129 import sys

   130 import time

   131 import traceback

   132 import warnings

   133 

   134 # I see no other way to suppress these warnings;

   135 # putting them in test_grammar.py has no effect:

   136 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,

   137                         ".*test.test_grammar$")

   138 if sys.maxint > 0x7fffffff:

   139     # Also suppress them in <string>, because for 64-bit platforms,

   140     # that's where test_grammar.py hides them.

   141     warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,

   142                             "<string>")

   143 

   144 # Ignore ImportWarnings that only occur in the source tree,

   145 # (because of modules with the same name as source-directories in Modules/)

   146 for mod in ("ctypes", "gzip", "zipfile", "tarfile", "encodings.zlib_codec",

   147             "test.test_zipimport", "test.test_zlib", "test.test_zipfile",

   148             "test.test_codecs", "test.string_tests"):

   149     warnings.filterwarnings(module=".*%s$" % (mod,),

   150                             action="ignore", category=ImportWarning)

   151 

   152 # MacOSX (a.k.a. Darwin) has a default stack size that is too small

   153 # for deeply recursive regular expressions.  We see this as crashes in

   154 # the Python test suite when running test_re.py and test_sre.py.  The

   155 # fix is to set the stack limit to 2048.

   156 # This approach may also be useful for other Unixy platforms that

   157 # suffer from small default stack limits.

   158 if sys.platform == 'darwin':

   159     try:

   160         import resource

   161     except ImportError:

   162         pass

   163     else:

   164         soft, hard = resource.getrlimit(resource.RLIMIT_STACK)

   165         newsoft = min(hard, max(soft, 1024*2048))

   166         resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))

   167 

   168 from test import test_support

   169 

   170 RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',

   171                   'decimal', 'compiler', 'subprocess', 'urlfetch')

   172 

   173 

   174 def usage(code, msg=''):

   175     print __doc__

   176     if msg: print msg

   177     sys.exit(code)

   178 

   179 

   180 def main(tests=None, testdir=None, verbose=0, quiet=False,

   181          exclude=False, single=False, randomize=False, fromfile=None,

   182          findleaks=False, use_resources=None, trace=False, coverdir='coverage',

   183          runleaks=False, huntrleaks=False, verbose2=False, print_slow=False):

   184     """Execute a test suite.

   185 

   186     This also parses command-line options and modifies its behavior

   187     accordingly.

   188 

   189     tests -- a list of strings containing test names (optional)

   190     testdir -- the directory in which to look for tests (optional)

   191 

   192     Users other than the Python test suite will certainly want to

   193     specify testdir; if it's omitted, the directory containing the

   194     Python test suite is searched for.

   195 

   196     If the tests argument is omitted, the tests listed on the

   197     command-line will be used.  If that's empty, too, then all *.py

   198     files beginning with test_ will be used.

   199 

   200     The other default arguments (verbose, quiet, exclude,

   201     single, randomize, findleaks, use_resources, trace, coverdir, and

   202     print_slow) allow programmers calling main() directly to set the

   203     values that would normally be set by flags on the command line.

   204     """

   205 

   206     test_support.record_original_stdout(sys.stdout)

   207     try:

   208         opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsSrf:lu:t:TD:NLR:wM:',

   209                                    ['help', 'verbose', 'quiet', 'exclude',

   210                                     'single', 'slow', 'random', 'fromfile',

   211                                     'findleaks', 'use=', 'threshold=', 'trace',

   212                                     'coverdir=', 'nocoverdir', 'runleaks',

   213                                     'huntrleaks=', 'verbose2', 'memlimit=',

   214                                     ])

   215     except getopt.error, msg:

   216         usage(2, msg)

   217 

   218     # Defaults

   219     if use_resources is None:

   220         use_resources = []

   221     for o, a in opts:

   222         if o in ('-h', '--help'):

   223             usage(0)

   224         elif o in ('-v', '--verbose'):

   225             verbose += 1

   226         elif o in ('-w', '--verbose2'):

   227             verbose2 = True

   228         elif o in ('-q', '--quiet'):

   229             quiet = True;

   230             verbose = 0

   231         elif o in ('-x', '--exclude'):

   232             exclude = True

   233         elif o in ('-s', '--single'):

   234             single = True

   235         elif o in ('-S', '--slow'):

   236             print_slow = True

   237         elif o in ('-r', '--randomize'):

   238             randomize = True

   239         elif o in ('-f', '--fromfile'):

   240             fromfile = a

   241         elif o in ('-l', '--findleaks'):

   242             findleaks = True

   243         elif o in ('-L', '--runleaks'):

   244             runleaks = True

   245         elif o in ('-t', '--threshold'):

   246             import gc

   247             gc.set_threshold(int(a))

   248         elif o in ('-T', '--coverage'):

   249             trace = True

   250         elif o in ('-D', '--coverdir'):

   251             coverdir = os.path.join(os.getcwd(), a)

   252         elif o in ('-N', '--nocoverdir'):

   253             coverdir = None

   254         elif o in ('-R', '--huntrleaks'):

   255             huntrleaks = a.split(':')

   256             if len(huntrleaks) != 3:

   257                 print a, huntrleaks

   258                 usage(2, '-R takes three colon-separated arguments')

   259             if len(huntrleaks[0]) == 0:

   260                 huntrleaks[0] = 5

   261             else:

   262                 huntrleaks[0] = int(huntrleaks[0])

   263             if len(huntrleaks[1]) == 0:

   264                 huntrleaks[1] = 4

   265             else:

   266                 huntrleaks[1] = int(huntrleaks[1])

   267             if len(huntrleaks[2]) == 0:

   268                 huntrleaks[2] = "reflog.txt"

   269         elif o in ('-M', '--memlimit'):

   270             test_support.set_memlimit(a)

   271         elif o in ('-u', '--use'):

   272             u = [x.lower() for x in a.split(',')]

   273             for r in u:

   274                 if r == 'all':

   275                     use_resources[:] = RESOURCE_NAMES

   276                     continue

   277                 remove = False

   278                 if r[0] == '-':

   279                     remove = True

   280                     r = r[1:]

   281                 if r not in RESOURCE_NAMES:

   282                     usage(1, 'Invalid -u/--use option: ' + a)

   283                 if remove:

   284                     if r in use_resources:

   285                         use_resources.remove(r)

   286                 elif r not in use_resources:

   287                     use_resources.append(r)

   288     if single and fromfile:

   289         usage(2, "-s and -f don't go together!")

   290 

   291     good = []

   292     bad = []

   293     skipped = []

   294     resource_denieds = []

   295 

   296     if findleaks:

   297         try:

   298             import gc

   299         except ImportError:

   300             print 'No GC available, disabling findleaks.'

   301             findleaks = False

   302         else:

   303             # Uncomment the line below to report garbage that is not

   304             # freeable by reference counting alone.  By default only

   305             # garbage that is not collectable by the GC is reported.

   306             #gc.set_debug(gc.DEBUG_SAVEALL)

   307             found_garbage = []

   308 

   309     if single:

   310         from tempfile import gettempdir

   311         filename = os.path.join(gettempdir(), 'pynexttest')

   312         try:

   313             fp = open(filename, 'r')

   314             next = fp.read().strip()

   315             tests = [next]

   316             fp.close()

   317         except IOError:

   318             pass

   319 

   320     if fromfile:

   321         tests = []

   322         fp = open(fromfile)

   323         for line in fp:

   324             guts = line.split() # assuming no test has whitespace in its name

   325             if guts and not guts[0].startswith('#'):

   326                 tests.extend(guts)

   327         fp.close()

   328 

   329     # Strip .py extensions.

   330     if args:

   331         args = map(removepy, args)

   332     if tests:

   333         tests = map(removepy, tests)

   334 

   335     stdtests = STDTESTS[:]

   336     nottests = NOTTESTS[:]

   337     if exclude:

   338         for arg in args:

   339             if arg in stdtests:

   340                 stdtests.remove(arg)

   341         nottests[:0] = args

   342         args = []

   343     tests = tests or args or findtests(testdir, stdtests, nottests)

   344     if single:

   345         tests = tests[:1]

   346     if randomize:

   347         random.shuffle(tests)

   348     if trace:

   349         import trace

   350         tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],

   351                              trace=False, count=True)

   352     test_times = []

   353     test_support.verbose = verbose      # Tell tests to be moderately quiet

   354     test_support.use_resources = use_resources

   355     save_modules = sys.modules.keys()

   356     for test in tests:

   357         if not quiet:

   358             print test

   359             sys.stdout.flush()

   360         if trace:

   361             # If we're tracing code coverage, then we don't exit with status

   362             # if on a false return value from main.

   363             tracer.runctx('runtest(test, verbose, quiet,'

   364                           '        test_times, testdir)',

   365                           globals=globals(), locals=vars())

   366         else:

   367             try:

   368                 ok = runtest(test, verbose, quiet, test_times,

   369                              testdir, huntrleaks)

   370             except KeyboardInterrupt:

   371                 # print a newline separate from the ^C

   372                 print

   373                 break

   374             except:

   375                 raise

   376             if ok > 0:

   377                 good.append(test)

   378             elif ok == 0:

   379                 bad.append(test)

   380             else:

   381                 skipped.append(test)

   382                 if ok == -2:

   383                     resource_denieds.append(test)

   384         if findleaks:

   385             gc.collect()

   386             if gc.garbage:

   387                 print "Warning: test created", len(gc.garbage),

   388                 print "uncollectable object(s)."

   389                 # move the uncollectable objects somewhere so we don't see

   390                 # them again

   391                 found_garbage.extend(gc.garbage)

   392                 del gc.garbage[:]

   393         # Unload the newly imported modules (best effort finalization)

   394         for module in sys.modules.keys():

   395             if module not in save_modules and module.startswith("test."):

   396                 test_support.unload(module)

   397 

   398     # The lists won't be sorted if running with -r

   399     good.sort()

   400     bad.sort()

   401     skipped.sort()

   402 

   403     if good and not quiet:

   404         if not bad and not skipped and len(good) > 1:

   405             print "All",

   406         print count(len(good), "test"), "OK."

   407     if print_slow:

   408         test_times.sort(reverse=True)

   409         print "10 slowest tests:"

   410         for time, test in test_times[:10]:

   411             print "%s: %.1fs" % (test, time)

   412     if bad:

   413         print count(len(bad), "test"), "failed:"

   414         printlist(bad)

   415     if skipped and not quiet:

   416         print count(len(skipped), "test"), "skipped:"

   417         printlist(skipped)

   418 

   419         e = _ExpectedSkips()

   420         plat = sys.platform

   421         if e.isvalid():

   422             surprise = set(skipped) - e.getexpected() - set(resource_denieds)

   423             if surprise:

   424                 print count(len(surprise), "skip"), \

   425                       "unexpected on", plat + ":"

   426                 printlist(surprise)

   427             else:

   428                 print "Those skips are all expected on", plat + "."

   429         else:

   430             print "Ask someone to teach regrtest.py about which tests are"

   431             print "expected to get skipped on", plat + "."

   432 

   433     if verbose2 and bad:

   434         print "Re-running failed tests in verbose mode"

   435         for test in bad:

   436             print "Re-running test %r in verbose mode" % test

   437             sys.stdout.flush()

   438             try:

   439                 test_support.verbose = True

   440                 ok = runtest(test, True, quiet, test_times, testdir,

   441                              huntrleaks)

   442             except KeyboardInterrupt:

   443                 # print a newline separate from the ^C

   444                 print

   445                 break

   446             except:

   447                 raise

   448 

   449     if single:

   450         alltests = findtests(testdir, stdtests, nottests)

   451         for i in range(len(alltests)):

   452             if tests[0] == alltests[i]:

   453                 if i == len(alltests) - 1:

   454                     os.unlink(filename)

   455                 else:

   456                     fp = open(filename, 'w')

   457                     fp.write(alltests[i+1] + '\n')

   458                     fp.close()

   459                 break

   460         else:

   461             os.unlink(filename)

   462 

   463     if trace:

   464         r = tracer.results()

   465         r.write_results(show_missing=True, summary=True, coverdir=coverdir)

   466 

   467     if runleaks:

   468         os.system("leaks %d" % os.getpid())

   469 

   470     sys.exit(len(bad) > 0)

   471 

   472 

   473 STDTESTS = [

   474     'test_grammar',

   475     'test_opcodes',

   476     'test_dict',

   477     'test_builtin',

   478     'test_exceptions',

   479     'test_types',

   480     'test_unittest',

   481     'test_doctest',

   482     'test_doctest2',

   483    ]

   484 

   485 NOTTESTS = [

   486     'test_support',

   487     'test_future1',

   488     'test_future2',

   489     ]

   490 

   491 def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):

   492     """Return a list of all applicable test modules."""

   493     if not testdir: testdir = findtestdir()

   494     names = os.listdir(testdir)

   495     tests = []

   496     for name in names:

   497         if name[:5] == "test_" and name[-3:] == os.extsep+"py":

   498             modname = name[:-3]

   499             if modname not in stdtests and modname not in nottests:

   500                 tests.append(modname)

   501     tests.sort()

   502     return stdtests + tests

   503 

   504 def runtest(test, verbose, quiet, test_times,

   505             testdir=None, huntrleaks=False):

   506     """Run a single test.

   507 

   508     test -- the name of the test

   509     verbose -- if true, print more messages

   510     quiet -- if true, don't print 'skipped' messages (probably redundant)

   511     test_times -- a list of (time, test_name) pairs

   512     testdir -- test directory

   513     huntrleaks -- run multiple times to test for leaks; requires a debug

   514                   build; a triple corresponding to -R's three arguments

   515     Return:

   516         -2  test skipped because resource denied

   517         -1  test skipped for some other reason

   518          0  test failed

   519          1  test passed

   520     """

   521 

   522     try:

   523         return runtest_inner(test, verbose, quiet, test_times,

   524                              testdir, huntrleaks)

   525     finally:

   526         cleanup_test_droppings(test, verbose)

   527 

   528 def runtest_inner(test, verbose, quiet, test_times,

   529                   testdir=None, huntrleaks=False):

   530     test_support.unload(test)

   531     if not testdir:

   532         testdir = findtestdir()

   533     if verbose:

   534         capture_stdout = None

   535     else:

   536         capture_stdout = cStringIO.StringIO()

   537 

   538     try:

   539         save_stdout = sys.stdout

   540         try:

   541             if capture_stdout:

   542                 sys.stdout = capture_stdout

   543             if test.startswith('test.'):

   544                 abstest = test

   545             else:

   546                 # Always import it from the test package

   547                 abstest = 'test.' + test

   548             start_time = time.time()

   549             the_package = __import__(abstest, globals(), locals(), [])

   550             the_module = getattr(the_package, test)

   551             # Old tests run to completion simply as a side-effect of

   552             # being imported.  For tests based on unittest or doctest,

   553             # explicitly invoke their test_main() function (if it exists).

   554             indirect_test = getattr(the_module, "test_main", None)

   555             if indirect_test is not None:

   556                 indirect_test()

   557             if huntrleaks:

   558                 dash_R(the_module, test, indirect_test, huntrleaks)

   559             test_time = time.time() - start_time

   560             test_times.append((test_time, test))

   561         finally:

   562             sys.stdout = save_stdout

   563     except test_support.ResourceDenied, msg:

   564         if not quiet:

   565             print test, "skipped --", msg

   566             sys.stdout.flush()

   567         return -2

   568     except (ImportError, test_support.TestSkipped), msg:

   569         if not quiet:

   570             print test, "skipped --", msg

   571             sys.stdout.flush()

   572         return -1

   573     except KeyboardInterrupt:

   574         raise

   575     except test_support.TestFailed, msg:

   576         print "test", test, "failed --", msg

   577         sys.stdout.flush()

   578         return 0

   579     except:

   580         type, value = sys.exc_info()[:2]

   581         print "test", test, "crashed --", str(type) + ":", value

   582         sys.stdout.flush()

   583         if verbose:

   584             traceback.print_exc(file=sys.stdout)

   585             sys.stdout.flush()

   586         return 0

   587     else:

   588         # Except in verbose mode, tests should not print anything

   589         if verbose or huntrleaks:

   590             return 1

   591         output = capture_stdout.getvalue()

   592         if not output:

   593             return 1

   594         print "test", test, "produced unexpected output:"

   595         print "*" * 70

   596         print output

   597         print "*" * 70

   598         sys.stdout.flush()

   599         return 0

   600 

   601 def cleanup_test_droppings(testname, verbose):

   602     import shutil

   603 

   604     # Try to clean up junk commonly left behind.  While tests shouldn't leave

   605     # any files or directories behind, when a test fails that can be tedious

   606     # for it to arrange.  The consequences can be especially nasty on Windows,

   607     # since if a test leaves a file open, it cannot be deleted by name (while

   608     # there's nothing we can do about that here either, we can display the

   609     # name of the offending test, which is a real help).

   610     for name in (test_support.TESTFN,

   611                  "db_home",

   612                 ):

   613         if not os.path.exists(name):

   614             continue

   615 

   616         if os.path.isdir(name):

   617             kind, nuker = "directory", shutil.rmtree

   618         elif os.path.isfile(name):

   619             kind, nuker = "file", os.unlink

   620         else:

   621             raise SystemError("os.path says %r exists but is neither "

   622                               "directory nor file" % name)

   623 

   624         if verbose:

   625             print "%r left behind %s %r" % (testname, kind, name)

   626         try:

   627             nuker(name)

   628         except Exception, msg:

   629             print >> sys.stderr, ("%r left behind %s %r and it couldn't be "

   630                 "removed: %s" % (testname, kind, name, msg))

   631 

   632 def dash_R(the_module, test, indirect_test, huntrleaks):

   633     # This code is hackish and inelegant, but it seems to do the job.

   634     import copy_reg, _abcoll, io

   635 

   636     if not hasattr(sys, 'gettotalrefcount'):

   637         raise Exception("Tracking reference leaks requires a debug build "

   638                         "of Python")

   639 

   640     # Save current values for dash_R_cleanup() to restore.

   641     fs = warnings.filters[:]

   642     ps = copy_reg.dispatch_table.copy()

   643     pic = sys.path_importer_cache.copy()

   644     abcs = {}

   645     modules = _abcoll, io

   646     for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]:

   647         # XXX isinstance(abc, ABCMeta) leads to infinite recursion

   648         if not hasattr(abc, '_abc_registry'):

   649             continue

   650         for obj in abc.__subclasses__() + [abc]:

   651             abcs[obj] = obj._abc_registry.copy()

   652 

   653     if indirect_test:

   654         def run_the_test():

   655             indirect_test()

   656     else:

   657         def run_the_test():

   658             reload(the_module)

   659 

   660     deltas = []

   661     nwarmup, ntracked, fname = huntrleaks

   662     repcount = nwarmup + ntracked

   663     print >> sys.stderr, "beginning", repcount, "repetitions"

   664     print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]

   665     dash_R_cleanup(fs, ps, pic, abcs)

   666     for i in range(repcount):

   667         rc = sys.gettotalrefcount()

   668         run_the_test()

   669         sys.stderr.write('.')

   670         dash_R_cleanup(fs, ps, pic, abcs)

   671         if i >= nwarmup:

   672             deltas.append(sys.gettotalrefcount() - rc - 2)

   673     print >> sys.stderr

   674     if any(deltas):

   675         msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas))

   676         print >> sys.stderr, msg

   677         refrep = open(fname, "a")

   678         print >> refrep, msg

   679         refrep.close()

   680 

   681 def dash_R_cleanup(fs, ps, pic, abcs):

   682     import gc, copy_reg

   683     import _strptime, linecache

   684     dircache = test_support.import_module('dircache', deprecated=True)

   685     import urlparse, urllib, urllib2, mimetypes, doctest

   686     import struct, filecmp

   687     from distutils.dir_util import _path_created

   688 

   689     # Clear the warnings registry, so they can be displayed again

   690     for mod in sys.modules.values():

   691         if hasattr(mod, '__warningregistry__'):

   692             del mod.__warningregistry__

   693 

   694     # Restore some original values.

   695     warnings.filters[:] = fs

   696     copy_reg.dispatch_table.clear()

   697     copy_reg.dispatch_table.update(ps)

   698     sys.path_importer_cache.clear()

   699     sys.path_importer_cache.update(pic)

   700 

   701     # clear type cache

   702     sys._clear_type_cache()

   703 

   704     # Clear ABC registries, restoring previously saved ABC registries.

   705     for abc, registry in abcs.items():

   706         abc._abc_registry = registry.copy()

   707         abc._abc_cache.clear()

   708         abc._abc_negative_cache.clear()

   709 

   710     # Clear assorted module caches.

   711     _path_created.clear()

   712     re.purge()

   713     _strptime._regex_cache.clear()

   714     urlparse.clear_cache()

   715     urllib.urlcleanup()

   716     urllib2.install_opener(None)

   717     dircache.reset()

   718     linecache.clearcache()

   719     mimetypes._default_mime_types()

   720     filecmp._cache.clear()

   721     struct._clearcache()

   722     doctest.master = None

   723 

   724     # Collect cyclic trash.

   725     gc.collect()

   726 

   727 def findtestdir():

   728     if __name__ == '__main__':

   729         file = sys.argv[0]

   730     else:

   731         file = __file__

   732     testdir = os.path.dirname(file) or os.curdir

   733     return testdir

   734 

   735 def removepy(name):

   736     if name.endswith(os.extsep + "py"):

   737         name = name[:-3]

   738     return name

   739 

   740 def count(n, word):

   741     if n == 1:

   742         return "%d %s" % (n, word)

   743     else:

   744         return "%d %ss" % (n, word)

   745 

   746 def printlist(x, width=70, indent=4):

   747     """Print the elements of iterable x to stdout.

   748 

   749     Optional arg width (default 70) is the maximum line length.

   750     Optional arg indent (default 4) is the number of blanks with which to

   751     begin each line.

   752     """

   753 

   754     from textwrap import fill

   755     blanks = ' ' * indent

   756     print fill(' '.join(map(str, x)), width,

   757                initial_indent=blanks, subsequent_indent=blanks)

   758 

   759 # Map sys.platform to a string containing the basenames of tests

   760 # expected to be skipped on that platform.

   761 #

   762 # Special cases:

   763 #     test_pep277

   764 #         The _ExpectedSkips constructor adds this to the set of expected

   765 #         skips if not os.path.supports_unicode_filenames.

   766 #     test_socket_ssl

   767 #         Controlled by test_socket_ssl.skip_expected.  Requires the network

   768 #         resource, and a socket module with ssl support.

   769 #     test_timeout

   770 #         Controlled by test_timeout.skip_expected.  Requires the network

   771 #         resource and a socket module.

   772 #

   773 # Tests that are expected to be skipped everywhere except on one platform

   774 # are also handled separately.

   775 

   776 _expectations = {

   777     'win32':

   778         """

   779         test__locale

   780         test_bsddb185

   781         test_bsddb3

   782         test_commands

   783         test_crypt

   784         test_curses

   785         test_dbm

   786         test_dl

   787         test_fcntl

   788         test_fork1

   789         test_epoll

   790         test_gdbm

   791         test_grp

   792         test_ioctl

   793         test_largefile

   794         test_kqueue

   795         test_mhlib

   796         test_openpty

   797         test_ossaudiodev

   798         test_pipes

   799         test_poll

   800         test_posix

   801         test_pty

   802         test_pwd

   803         test_resource

   804         test_signal

   805         test_threadsignals

   806         test_timing

   807         test_wait3

   808         test_wait4

   809         """,

   810     'linux2':

   811         """

   812         test_bsddb185

   813         test_curses

   814         test_dl

   815         test_largefile

   816         test_kqueue

   817         test_ossaudiodev

   818         """,

   819    'mac':

   820         """

   821         test_atexit

   822         test_bsddb

   823         test_bsddb185

   824         test_bsddb3

   825         test_bz2

   826         test_commands

   827         test_crypt

   828         test_curses

   829         test_dbm

   830         test_dl

   831         test_fcntl

   832         test_fork1

   833         test_epoll

   834         test_grp

   835         test_ioctl

   836         test_largefile

   837         test_locale

   838         test_kqueue

   839         test_mmap

   840         test_openpty

   841         test_ossaudiodev

   842         test_poll

   843         test_popen

   844         test_popen2

   845         test_posix

   846         test_pty

   847         test_pwd

   848         test_resource

   849         test_signal

   850         test_sundry

   851         test_tarfile

   852         test_timing

   853         """,

   854     'unixware7':

   855         """

   856         test_bsddb

   857         test_bsddb185

   858         test_dl

   859         test_epoll

   860         test_largefile

   861         test_kqueue

   862         test_minidom

   863         test_openpty

   864         test_pyexpat

   865         test_sax

   866         test_sundry

   867         """,

   868     'openunix8':

   869         """

   870         test_bsddb

   871         test_bsddb185

   872         test_dl

   873         test_epoll

   874         test_largefile

   875         test_kqueue

   876         test_minidom

   877         test_openpty

   878         test_pyexpat

   879         test_sax

   880         test_sundry

   881         """,

   882     'sco_sv3':

   883         """

   884         test_asynchat

   885         test_bsddb

   886         test_bsddb185

   887         test_dl

   888         test_fork1

   889         test_epoll

   890         test_gettext

   891         test_largefile

   892         test_locale

   893         test_kqueue

   894         test_minidom

   895         test_openpty

   896         test_pyexpat

   897         test_queue

   898         test_sax

   899         test_sundry

   900         test_thread

   901         test_threaded_import

   902         test_threadedtempfile

   903         test_threading

   904         """,

   905     'riscos':

   906         """

   907         test_asynchat

   908         test_atexit

   909         test_bsddb

   910         test_bsddb185

   911         test_bsddb3

   912         test_commands

   913         test_crypt

   914         test_dbm

   915         test_dl

   916         test_fcntl

   917         test_fork1

   918         test_epoll

   919         test_gdbm

   920         test_grp

   921         test_largefile

   922         test_locale

   923         test_kqueue

   924         test_mmap

   925         test_openpty

   926         test_poll

   927         test_popen2

   928         test_pty

   929         test_pwd

   930         test_strop

   931         test_sundry

   932         test_thread

   933         test_threaded_import

   934         test_threadedtempfile

   935         test_threading

   936         test_timing

   937         """,

   938     'darwin':

   939         """

   940         test__locale

   941         test_bsddb

   942         test_bsddb3

   943         test_curses

   944         test_epoll

   945         test_gdbm

   946         test_largefile

   947         test_locale

   948         test_kqueue

   949         test_minidom

   950         test_ossaudiodev

   951         test_poll

   952         """,

   953     'sunos5':

   954         """

   955         test_bsddb

   956         test_bsddb185

   957         test_curses

   958         test_dbm

   959         test_epoll

   960         test_kqueue

   961         test_gdbm

   962         test_gzip

   963         test_openpty

   964         test_zipfile

   965         test_zlib

   966         """,

   967     'hp-ux11':

   968         """

   969         test_bsddb

   970         test_bsddb185

   971         test_curses

   972         test_dl

   973         test_epoll

   974         test_gdbm

   975         test_gzip

   976         test_largefile

   977         test_locale

   978         test_kqueue

   979         test_minidom

   980         test_openpty

   981         test_pyexpat

   982         test_sax

   983         test_zipfile

   984         test_zlib

   985         """,

   986     'atheos':

   987         """

   988         test_bsddb185

   989         test_curses

   990         test_dl

   991         test_gdbm

   992         test_epoll

   993         test_largefile

   994         test_locale

   995         test_kqueue

   996         test_mhlib

   997         test_mmap

   998         test_poll

   999         test_popen2

  1000         test_resource

  1001         """,

  1002     'cygwin':

  1003         """

  1004         test_bsddb185

  1005         test_bsddb3

  1006         test_curses

  1007         test_dbm

  1008         test_epoll

  1009         test_ioctl

  1010         test_kqueue

  1011         test_largefile

  1012         test_locale

  1013         test_ossaudiodev

  1014         test_socketserver

  1015         """,

  1016     'os2emx':

  1017         """

  1018         test_audioop

  1019         test_bsddb185

  1020         test_bsddb3

  1021         test_commands

  1022         test_curses

  1023         test_dl

  1024         test_epoll

  1025         test_kqueue

  1026         test_largefile

  1027         test_mhlib

  1028         test_mmap

  1029         test_openpty

  1030         test_ossaudiodev

  1031         test_pty

  1032         test_resource

  1033         test_signal

  1034         """,

  1035     'freebsd4':

  1036         """

  1037         test_bsddb

  1038         test_bsddb3

  1039         test_epoll

  1040         test_gdbm

  1041         test_locale

  1042         test_ossaudiodev

  1043         test_pep277

  1044         test_pty

  1045         test_socket_ssl

  1046         test_socketserver

  1047         test_tcl

  1048         test_timeout

  1049         test_urllibnet

  1050         test_multiprocessing

  1051         """,

  1052     'aix5':

  1053         """

  1054         test_bsddb

  1055         test_bsddb185

  1056         test_bsddb3

  1057         test_bz2

  1058         test_dl

  1059         test_epoll

  1060         test_gdbm

  1061         test_gzip

  1062         test_kqueue

  1063         test_ossaudiodev

  1064         test_tcl

  1065         test_zipimport

  1066         test_zlib

  1067         """,

  1068     'openbsd3':

  1069         """

  1070         test_bsddb

  1071         test_bsddb3

  1072         test_ctypes

  1073         test_dl

  1074         test_epoll

  1075         test_gdbm

  1076         test_locale

  1077         test_normalization

  1078         test_ossaudiodev

  1079         test_pep277

  1080         test_tcl

  1081         test_multiprocessing

  1082         """,

  1083     'netbsd3':

  1084         """

  1085         test_bsddb

  1086         test_bsddb185

  1087         test_bsddb3

  1088         test_ctypes

  1089         test_curses

  1090         test_dl

  1091         test_epoll

  1092         test_gdbm

  1093         test_locale

  1094         test_ossaudiodev

  1095         test_pep277

  1096         test_tcl

  1097         test_multiprocessing

  1098         """,

  1099 }

  1100 _expectations['freebsd5'] = _expectations['freebsd4']

  1101 _expectations['freebsd6'] = _expectations['freebsd4']

  1102 _expectations['freebsd7'] = _expectations['freebsd4']

  1103 _expectations['freebsd8'] = _expectations['freebsd4']

  1104 

  1105 class _ExpectedSkips:

  1106     def __init__(self):

  1107         import os.path

  1108         from test import test_timeout

  1109 

  1110         self.valid = False

  1111         if sys.platform in _expectations:

  1112             s = _expectations[sys.platform]

  1113             self.expected = set(s.split())

  1114 

  1115             # expected to be skipped on every platform, even Linux

  1116             self.expected.add('test_linuxaudiodev')

  1117 

  1118             if not os.path.supports_unicode_filenames:

  1119                 self.expected.add('test_pep277')

  1120 

  1121             try:

  1122                 from test import test_socket_ssl

  1123             except ImportError:

  1124                 pass

  1125             else:

  1126                 if test_socket_ssl.skip_expected:

  1127                     self.expected.add('test_socket_ssl')

  1128 

  1129             if test_timeout.skip_expected:

  1130                 self.expected.add('test_timeout')

  1131 

  1132             if sys.maxint == 9223372036854775807L:

  1133                 self.expected.add('test_imageop')

  1134 

  1135             if not sys.platform in ("mac", "darwin"):