1 #! /usr/bin/env python

     2 

     3 """Regression test.

     4 

     5 This will find all modules whose name is "test_*" in the test

     6 directory, and run them.  Various command line options provide

     7 additional facilities.

     8 

     9 Command line options:

    10 

    11 -v: verbose    -- run tests in verbose mode with output to stdout

    12 -w: verbose2   -- re-run failed tests in verbose mode

    13 -q: quiet      -- don't print anything except if a test fails

    14 -g: generate   -- write the output file for a test instead of comparing it

    15 -x: exclude    -- arguments are tests to *exclude*

    16 -s: single     -- run only a single test (see below)

    17 -r: random     -- randomize test execution order

    18 -f: fromfile   -- read names of tests to run from a file (see below)

    19 -l: findleaks  -- if GC is available detect tests that leak memory

    20 -u: use        -- specify which special resource intensive tests to run

    21 -h: help       -- print this text and exit

    22 -t: threshold  -- call gc.set_threshold(N)

    23 -T: coverage   -- turn on code coverage using the trace module

    24 -D: coverdir   -- Directory where coverage files are put

    25 -N: nocoverdir -- Put coverage files alongside modules

    26 -L: runleaks   -- run the leaks(1) command just before exit

    27 -R: huntrleaks -- search for reference leaks (needs debug build, v. slow)

    28 -M: memlimit   -- run very large memory-consuming tests

    29 

    30 If non-option arguments are present, they are names for tests to run,

    31 unless -x is given, in which case they are names for tests not to run.

    32 If no test names are given, all tests are run.

    33 

    34 -v is incompatible with -g and does not compare test output files.

    35 

    36 -T turns on code coverage tracing with the trace module.

    37 

    38 -D specifies the directory where coverage files are put.

    39 

    40 -N Put coverage files alongside modules.

    41 

    42 -s means to run only a single test and exit.  This is useful when

    43 doing memory analysis on the Python interpreter (which tend to consume

    44 too many resources to run the full regression test non-stop).  The

    45 file /tmp/pynexttest is read to find the next test to run.  If this

    46 file is missing, the first test_*.py file in testdir or on the command

    47 line is used.  (actually tempfile.gettempdir() is used instead of

    48 /tmp).

    49 

    50 -f reads the names of tests from the file given as f's argument, one

    51 or more test names per line.  Whitespace is ignored.  Blank lines and

    52 lines beginning with '#' are ignored.  This is especially useful for

    53 whittling down failures involving interactions among tests.

    54 

    55 -L causes the leaks(1) command to be run just before exit if it exists.

    56 leaks(1) is available on Mac OS X and presumably on some other

    57 FreeBSD-derived systems.

    58 

    59 -R runs each test several times and examines sys.gettotalrefcount() to

    60 see if the test appears to be leaking references.  The argument should

    61 be of the form stab:run:fname where 'stab' is the number of times the

    62 test is run to let gettotalrefcount settle down, 'run' is the number

    63 of times further it is run and 'fname' is the name of the file the

    64 reports are written to.  These parameters all have defaults (5, 4 and

    65 "reflog.txt" respectively), so the minimal invocation is '-R ::'.

    66 

    67 -M runs tests that require an exorbitant amount of memory. These tests

    68 typically try to ascertain containers keep working when containing more than

    69 2 billion objects, which only works on 64-bit systems. There are also some

    70 tests that try to exhaust the address space of the process, which only makes

    71 sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,

    72 which is a string in the form of '2.5Gb', determines howmuch memory the

    73 tests will limit themselves to (but they may go slightly over.) The number

    74 shouldn't be more memory than the machine has (including swap memory). You

    75 should also keep in mind that swap memory is generally much, much slower

    76 than RAM, and setting memlimit to all available RAM or higher will heavily

    77 tax the machine. On the other hand, it is no use running these tests with a

    78 limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect

    79 to use more than memlimit memory will be skipped. The big-memory tests

    80 generally run very, very long.

    81 

    82 -u is used to specify which special resource intensive tests to run,

    83 such as those requiring large file support or network connectivity.

    84 The argument is a comma-separated list of words indicating the

    85 resources to test.  Currently only the following are defined:

    86 

    87     all -       Enable all special resources.

    88 

    89     audio -     Tests that use the audio device.  (There are known

    90                 cases of broken audio drivers that can crash Python or

    91                 even the Linux kernel.)

    92 

    93     curses -    Tests that use curses and will modify the terminal's

    94                 state and output modes.

    95 

    96     largefile - It is okay to run some test that may create huge

    97                 files.  These tests can take a long time and may

    98                 consume >2GB of disk space temporarily.

    99 

   100     network -   It is okay to run tests that use external network

   101                 resource, e.g. testing SSL support for sockets.

   102 

   103     bsddb -     It is okay to run the bsddb testsuite, which takes

   104                 a long time to complete.

   105 

   106     decimal -   Test the decimal module against a large suite that

   107                 verifies compliance with standards.

   108 

   109     compiler -  Test the compiler package by compiling all the source

   110                 in the standard library and test suite.  This takes

   111                 a long time.  Enabling this resource also allows

   112                 test_tokenize to verify round-trip lexing on every

   113                 file in the test library.

   114 

   115     subprocess  Run all tests for the subprocess module.

   116 

   117     urlfetch -  It is okay to download files required on testing.

   118 

   119 To enable all resources except one, use '-uall,-<resource>'.  For

   120 example, to run all the tests except for the bsddb tests, give the

   121 option '-uall,-bsddb'.

   122 """

   123 

   124 import os

   125 import sys

   126 import getopt

   127 import random

   128 import warnings

   129 import re

   130 import cStringIO

   131 import traceback

   132 

   133 # I see no other way to suppress these warnings;

   134 # putting them in test_grammar.py has no effect:

   135 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,

   136                         ".*test.test_grammar$")

   137 if sys.maxint > 0x7fffffff:

   138     # Also suppress them in <string>, because for 64-bit platforms,

   139     # that's where test_grammar.py hides them.

   140     warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,

   141                             "<string>")

   142 

   143 # Ignore ImportWarnings that only occur in the source tree,

   144 # (because of modules with the same name as source-directories in Modules/)

   145 for mod in ("ctypes", "gzip", "zipfile", "tarfile", "encodings.zlib_codec",

   146             "test.test_zipimport", "test.test_zlib", "test.test_zipfile",

   147             "test.test_codecs", "test.string_tests"):

   148     warnings.filterwarnings(module=".*%s$" % (mod,),

   149                             action="ignore", category=ImportWarning)

   150 

   151 # MacOSX (a.k.a. Darwin) has a default stack size that is too small

   152 # for deeply recursive regular expressions.  We see this as crashes in

   153 # the Python test suite when running test_re.py and test_sre.py.  The

   154 # fix is to set the stack limit to 2048.

   155 # This approach may also be useful for other Unixy platforms that

   156 # suffer from small default stack limits.

   157 if sys.platform == 'darwin':

   158     try:

   159         import resource

   160     except ImportError:

   161         pass

   162     else:

   163         soft, hard = resource.getrlimit(resource.RLIMIT_STACK)

   164         newsoft = min(hard, max(soft, 1024*2048))

   165         resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))

   166 

   167 from test import test_support

   168 

   169 RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',

   170                   'decimal', 'compiler', 'subprocess', 'urlfetch')

   171 

   172 

   173 def usage(code, msg=''):

   174     print __doc__

   175     if msg: print msg

   176     sys.exit(code)

   177 

   178 

   179 def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,

   180          exclude=False, single=False, randomize=False, fromfile=None,

   181          findleaks=False, use_resources=None, trace=False, coverdir='coverage',

   182          runleaks=False, huntrleaks=False, verbose2=False):

   183     """Execute a test suite.

   184 

   185     This also parses command-line options and modifies its behavior

   186     accordingly.

   187 

   188     tests -- a list of strings containing test names (optional)

   189     testdir -- the directory in which to look for tests (optional)

   190 

   191     Users other than the Python test suite will certainly want to

   192     specify testdir; if it's omitted, the directory containing the

   193     Python test suite is searched for.

   194 

   195     If the tests argument is omitted, the tests listed on the

   196     command-line will be used.  If that's empty, too, then all *.py

   197     files beginning with test_ will be used.

   198 

   199     The other default arguments (verbose, quiet, generate, exclude, single,

   200     randomize, findleaks, use_resources, trace and coverdir) allow programmers

   201     calling main() directly to set the values that would normally be set by

   202     flags on the command line.

   203     """

   204 

   205     test_support.record_original_stdout(sys.stdout)

   206     try:

   207         opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TD:NLR:wM:',

   208                                    ['help', 'verbose', 'quiet', 'generate',

   209                                     'exclude', 'single', 'random', 'fromfile',

   210                                     'findleaks', 'use=', 'threshold=', 'trace',

   211                                     'coverdir=', 'nocoverdir', 'runleaks',

   212                                     'huntrleaks=', 'verbose2', 'memlimit=',

   213                                     ])

   214     except getopt.error, msg:

   215         usage(2, msg)

   216 

   217     # Defaults

   218     if use_resources is None:

   219         use_resources = []

   220     for o, a in opts:

   221         if o in ('-h', '--help'):

   222             usage(0)

   223         elif o in ('-v', '--verbose'):

   224             verbose += 1

   225         elif o in ('-w', '--verbose2'):

   226             verbose2 = True

   227         elif o in ('-q', '--quiet'):

   228             quiet = True;

   229             verbose = 0

   230         elif o in ('-g', '--generate'):

   231             generate = True

   232         elif o in ('-x', '--exclude'):

   233             exclude = True

   234         elif o in ('-s', '--single'):

   235             single = True

   236         elif o in ('-r', '--randomize'):

   237             randomize = True

   238         elif o in ('-f', '--fromfile'):

   239             fromfile = a

   240         elif o in ('-l', '--findleaks'):

   241             findleaks = True

   242         elif o in ('-L', '--runleaks'):

   243             runleaks = True

   244         elif o in ('-t', '--threshold'):

   245             import gc

   246             gc.set_threshold(int(a))

   247         elif o in ('-T', '--coverage'):

   248             trace = True

   249         elif o in ('-D', '--coverdir'):

   250             coverdir = os.path.join(os.getcwd(), a)

   251         elif o in ('-N', '--nocoverdir'):

   252             coverdir = None

   253         elif o in ('-R', '--huntrleaks'):

   254             huntrleaks = a.split(':')

   255             if len(huntrleaks) != 3:

   256                 print a, huntrleaks

   257                 usage(2, '-R takes three colon-separated arguments')

   258             if len(huntrleaks[0]) == 0:

   259                 huntrleaks[0] = 5

   260             else:

   261                 huntrleaks[0] = int(huntrleaks[0])

   262             if len(huntrleaks[1]) == 0:

   263                 huntrleaks[1] = 4

   264             else:

   265                 huntrleaks[1] = int(huntrleaks[1])

   266             if len(huntrleaks[2]) == 0:

   267                 huntrleaks[2] = "reflog.txt"

   268         elif o in ('-M', '--memlimit'):

   269             test_support.set_memlimit(a)

   270         elif o in ('-u', '--use'):

   271             u = [x.lower() for x in a.split(',')]

   272             for r in u:

   273                 if r == 'all':

   274                     use_resources[:] = RESOURCE_NAMES

   275                     continue

   276                 remove = False

   277                 if r[0] == '-':

   278                     remove = True

   279                     r = r[1:]

   280                 if r not in RESOURCE_NAMES:

   281                     usage(1, 'Invalid -u/--use option: ' + a)

   282                 if remove:

   283                     if r in use_resources:

   284                         use_resources.remove(r)

   285                 elif r not in use_resources:

   286                     use_resources.append(r)

   287     if generate and verbose:

   288         usage(2, "-g and -v don't go together!")

   289     if single and fromfile:

   290         usage(2, "-s and -f don't go together!")

   291 

   292     good = []

   293     bad = []

   294     skipped = []

   295     resource_denieds = []

   296 

   297     if findleaks:

   298         try:

   299             import gc

   300         except ImportError:

   301             print 'No GC available, disabling findleaks.'

   302             findleaks = False

   303         else:

   304             # Uncomment the line below to report garbage that is not

   305             # freeable by reference counting alone.  By default only

   306             # garbage that is not collectable by the GC is reported.

   307             #gc.set_debug(gc.DEBUG_SAVEALL)

   308             found_garbage = []

   309 

   310     if single:

   311         from tempfile import gettempdir

   312         filename = os.path.join(gettempdir(), 'pynexttest')

   313         try:

   314             fp = open(filename, 'r')

   315             next = fp.read().strip()

   316             tests = [next]

   317             fp.close()

   318         except IOError:

   319             pass

   320 

   321     if fromfile:

   322         tests = []

   323         fp = open(fromfile)

   324         for line in fp:

   325             guts = line.split() # assuming no test has whitespace in its name

   326             if guts and not guts[0].startswith('#'):

   327                 tests.extend(guts)

   328         fp.close()

   329 

   330     # Strip .py extensions.

   331     if args:

   332         args = map(removepy, args)

   333     if tests:

   334         tests = map(removepy, tests)

   335 

   336     stdtests = STDTESTS[:]

   337     nottests = NOTTESTS[:]

   338     if exclude:

   339         for arg in args:

   340             if arg in stdtests:

   341                 stdtests.remove(arg)

   342         nottests[:0] = args

   343         args = []

   344     tests = tests or args or findtests(testdir, stdtests, nottests)

   345     if single:

   346         tests = tests[:1]

   347     if randomize:

   348         random.shuffle(tests)

   349     if trace:

   350         import trace

   351         tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],

   352                              trace=False, count=True)

   353     test_support.verbose = verbose      # Tell tests to be moderately quiet

   354     test_support.use_resources = use_resources

   355     save_modules = sys.modules.keys()

   356     for test in tests:

   357         if not quiet:

   358             print test

   359             sys.stdout.flush()

   360         if trace:

   361             # If we're tracing code coverage, then we don't exit with status

   362             # if on a false return value from main.

   363             tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',

   364                           globals=globals(), locals=vars())

   365         else:

   366             try:

   367                 ok = runtest(test, generate, verbose, quiet, testdir,

   368                              huntrleaks)

   369             except KeyboardInterrupt:

   370                 # print a newline separate from the ^C

   371                 print

   372                 break

   373             except:

   374                 raise

   375             if ok > 0:

   376                 good.append(test)

   377             elif ok == 0:

   378                 bad.append(test)

   379             else:

   380                 skipped.append(test)

   381                 if ok == -2:

   382                     resource_denieds.append(test)

   383         if findleaks:

   384             gc.collect()

   385             if gc.garbage:

   386                 print "Warning: test created", len(gc.garbage),

   387                 print "uncollectable object(s)."

   388                 # move the uncollectable objects somewhere so we don't see

   389                 # them again

   390                 found_garbage.extend(gc.garbage)

   391                 del gc.garbage[:]

   392         # Unload the newly imported modules (best effort finalization)

   393         for module in sys.modules.keys():

   394             if module not in save_modules and module.startswith("test."):

   395                 test_support.unload(module)

   396 

   397     # The lists won't be sorted if running with -r

   398     good.sort()

   399     bad.sort()

   400     skipped.sort()

   401 

   402     if good and not quiet:

   403         if not bad and not skipped and len(good) > 1:

   404             print "All",

   405         print count(len(good), "test"), "OK."

   406         if verbose:

   407             print "CAUTION:  stdout isn't compared in verbose mode:"

   408             print "a test that passes in verbose mode may fail without it."

   409     if bad:

   410         print count(len(bad), "test"), "failed:"

   411         printlist(bad)

   412     if skipped and not quiet:

   413         print count(len(skipped), "test"), "skipped:"

   414         printlist(skipped)

   415 

   416         e = _ExpectedSkips()

   417         plat = sys.platform

   418         if e.isvalid():

   419             surprise = set(skipped) - e.getexpected() - set(resource_denieds)

   420             if surprise:

   421                 print count(len(surprise), "skip"), \

   422                       "unexpected on", plat + ":"

   423                 printlist(surprise)

   424             else:

   425                 print "Those skips are all expected on", plat + "."

   426         else:

   427             print "Ask someone to teach regrtest.py about which tests are"

   428             print "expected to get skipped on", plat + "."

   429 

   430     if verbose2 and bad:

   431         print "Re-running failed tests in verbose mode"

   432         for test in bad:

   433             print "Re-running test %r in verbose mode" % test

   434             sys.stdout.flush()

   435             try:

   436                 test_support.verbose = 1

   437                 ok = runtest(test, generate, 1, quiet, testdir,

   438                              huntrleaks)

   439             except KeyboardInterrupt:

   440                 # print a newline separate from the ^C

   441                 print

   442                 break

   443             except:

   444                 raise

   445 

   446     if single:

   447         alltests = findtests(testdir, stdtests, nottests)

   448         for i in range(len(alltests)):

   449             if tests[0] == alltests[i]:

   450                 if i == len(alltests) - 1:

   451                     os.unlink(filename)

   452                 else:

   453                     fp = open(filename, 'w')

   454                     fp.write(alltests[i+1] + '\n')

   455                     fp.close()

   456                 break

   457         else:

   458             os.unlink(filename)

   459 

   460     if trace:

   461         r = tracer.results()

   462         r.write_results(show_missing=True, summary=True, coverdir=coverdir)

   463 

   464     if runleaks:

   465         os.system("leaks %d" % os.getpid())

   466 

   467     sys.exit(len(bad) > 0)

   468 

   469 

   470 STDTESTS = [

   471     'test_grammar',

   472     'test_opcodes',

   473     'test_operations',

   474     'test_builtin',

   475     'test_exceptions',

   476     'test_types',

   477     'test_unittest',

   478     'test_doctest',

   479     'test_doctest2',

   480    ]

   481 

   482 NOTTESTS = [

   483     'test_support',

   484     'test_future1',

   485     'test_future2',

   486     'test_future3',

   487     ]

   488 

   489 def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):

   490     """Return a list of all applicable test modules."""

   491     if not testdir: testdir = findtestdir()

   492     names = os.listdir(testdir)

   493     tests = []

   494     for name in names:

   495         if name[:5] == "test_" and name[-3:] == os.extsep+"py":

   496             modname = name[:-3]

   497             if modname not in stdtests and modname not in nottests:

   498                 tests.append(modname)

   499     tests.sort()

   500     return stdtests + tests

   501 

   502 def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False):

   503     """Run a single test.

   504 

   505     test -- the name of the test

   506     generate -- if true, generate output, instead of running the test

   507                 and comparing it to a previously created output file

   508     verbose -- if true, print more messages

   509     quiet -- if true, don't print 'skipped' messages (probably redundant)

   510     testdir -- test directory

   511     huntrleaks -- run multiple times to test for leaks; requires a debug

   512                   build; a triple corresponding to -R's three arguments

   513     Return:

   514         -2  test skipped because resource denied

   515         -1  test skipped for some other reason

   516          0  test failed

   517          1  test passed

   518     """

   519 

   520     try:

   521         return runtest_inner(test, generate, verbose, quiet, testdir,

   522                              huntrleaks)

   523     finally:

   524         cleanup_test_droppings(test, verbose)

   525 

   526 def runtest_inner(test, generate, verbose, quiet,

   527                      testdir=None, huntrleaks=False):

   528     test_support.unload(test)

   529     if not testdir:

   530         testdir = findtestdir()

   531     outputdir = os.path.join(testdir, "output")

   532     outputfile = os.path.join(outputdir, test)

   533     if verbose:

   534         cfp = None

   535     else:

   536         cfp = cStringIO.StringIO()

   537 

   538     try:

   539         save_stdout = sys.stdout

   540         try:

   541             if cfp:

   542                 sys.stdout = cfp

   543                 print test              # Output file starts with test name

   544             if test.startswith('test.'):

   545                 abstest = test

   546             else:

   547                 # Always import it from the test package

   548                 abstest = 'test.' + test

   549             the_package = __import__(abstest, globals(), locals(), [])

   550             the_module = getattr(the_package, test)

   551             # Most tests run to completion simply as a side-effect of

   552             # being imported.  For the benefit of tests that can't run

   553             # that way (like test_threaded_import), explicitly invoke

   554             # their test_main() function (if it exists).

   555             indirect_test = getattr(the_module, "test_main", None)

   556             if indirect_test is not None:

   557                 indirect_test()

   558             if huntrleaks:

   559                 dash_R(the_module, test, indirect_test, huntrleaks)

   560         finally:

   561             sys.stdout = save_stdout

   562     except test_support.ResourceDenied, msg:

   563         if not quiet:

   564             print test, "skipped --", msg

   565             sys.stdout.flush()

   566         return -2

   567     except (ImportError, test_support.TestSkipped), msg:

   568         if not quiet:

   569             print test, "skipped --", msg

   570             sys.stdout.flush()

   571         return -1

   572     except KeyboardInterrupt:

   573         raise

   574     except test_support.TestFailed, msg:

   575         print "test", test, "failed --", msg

   576         sys.stdout.flush()

   577         return 0

   578     except:

   579         type, value = sys.exc_info()[:2]

   580         print "test", test, "crashed --", str(type) + ":", value

   581         sys.stdout.flush()

   582         if verbose:

   583             traceback.print_exc(file=sys.stdout)

   584             sys.stdout.flush()

   585         return 0

   586     else:

   587         if not cfp:

   588             return 1

   589         output = cfp.getvalue()

   590         if generate:

   591             if output == test + "\n":

   592                 if os.path.exists(outputfile):

   593                     # Write it since it already exists (and the contents

   594                     # may have changed), but let the user know it isn't

   595                     # needed:

   596                     print "output file", outputfile, \

   597                           "is no longer needed; consider removing it"

   598                 else:

   599                     # We don't need it, so don't create it.

   600                     return 1

   601             fp = open(outputfile, "w")

   602             fp.write(output)

   603             fp.close()

   604             return 1

   605         if os.path.exists(outputfile):

   606             fp = open(outputfile, "r")

   607             expected = fp.read()

   608             fp.close()

   609         else:

   610             expected = test + "\n"

   611         if output == expected or huntrleaks:

   612             return 1

   613         print "test", test, "produced unexpected output:"

   614         sys.stdout.flush()

   615         reportdiff(expected, output)

   616         sys.stdout.flush()

   617         return 0

   618 

   619 def cleanup_test_droppings(testname, verbose):

   620     import shutil

   621 

   622     # Try to clean up junk commonly left behind.  While tests shouldn't leave

   623     # any files or directories behind, when a test fails that can be tedious

   624     # for it to arrange.  The consequences can be especially nasty on Windows,

   625     # since if a test leaves a file open, it cannot be deleted by name (while

   626     # there's nothing we can do about that here either, we can display the

   627     # name of the offending test, which is a real help).

   628     for name in (test_support.TESTFN,

   629                  "db_home",

   630                 ):

   631         if not os.path.exists(name):

   632             continue

   633 

   634         if os.path.isdir(name):

   635             kind, nuker = "directory", shutil.rmtree

   636         elif os.path.isfile(name):

   637             kind, nuker = "file", os.unlink

   638         else:

   639             raise SystemError("os.path says %r exists but is neither "

   640                               "directory nor file" % name)

   641 

   642         if verbose:

   643             print "%r left behind %s %r" % (testname, kind, name)

   644         try:

   645             nuker(name)

   646         except Exception, msg:

   647             print >> sys.stderr, ("%r left behind %s %r and it couldn't be "

   648                 "removed: %s" % (testname, kind, name, msg))

   649 

   650 def dash_R(the_module, test, indirect_test, huntrleaks):

   651     # This code is hackish and inelegant, but it seems to do the job.

   652     import copy_reg

   653 

   654     if not hasattr(sys, 'gettotalrefcount'):

   655         raise Exception("Tracking reference leaks requires a debug build "

   656                         "of Python")

   657 

   658     # Save current values for dash_R_cleanup() to restore.

   659     fs = warnings.filters[:]

   660     ps = copy_reg.dispatch_table.copy()

   661     pic = sys.path_importer_cache.copy()

   662 

   663     if indirect_test:

   664         def run_the_test():

   665             indirect_test()

   666     else:

   667         def run_the_test():

   668             reload(the_module)

   669 

   670     deltas = []

   671     nwarmup, ntracked, fname = huntrleaks

   672     repcount = nwarmup + ntracked

   673     print >> sys.stderr, "beginning", repcount, "repetitions"

   674     print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]

   675     dash_R_cleanup(fs, ps, pic)

   676     for i in range(repcount):

   677         rc = sys.gettotalrefcount()

   678         run_the_test()

   679         sys.stderr.write('.')

   680         dash_R_cleanup(fs, ps, pic)

   681         if i >= nwarmup:

   682             deltas.append(sys.gettotalrefcount() - rc - 2)

   683     print >> sys.stderr

   684     if any(deltas):

   685         print >> sys.stderr, test, 'leaked', deltas, 'references'

   686         refrep = open(fname, "a")

   687         print >> refrep, test, 'leaked', deltas, 'references'

   688         refrep.close()

   689 

   690 def dash_R_cleanup(fs, ps, pic):

   691     import gc, copy_reg

   692     import _strptime, linecache, dircache

   693     import urlparse, urllib, urllib2, mimetypes, doctest

   694     import struct, filecmp

   695     from distutils.dir_util import _path_created

   696 

   697     # Restore some original values.

   698     warnings.filters[:] = fs

   699     copy_reg.dispatch_table.clear()

   700     copy_reg.dispatch_table.update(ps)

   701     sys.path_importer_cache.clear()

   702     sys.path_importer_cache.update(pic)

   703 

   704     # Clear assorted module caches.

   705     _path_created.clear()

   706     re.purge()

   707     _strptime._regex_cache.clear()

   708     urlparse.clear_cache()

   709     urllib.urlcleanup()

   710     urllib2.install_opener(None)

   711     dircache.reset()

   712     linecache.clearcache()

   713     mimetypes._default_mime_types()

   714     struct._cache.clear()

   715     filecmp._cache.clear()

   716     doctest.master = None

   717 

   718     # Collect cyclic trash.

   719     gc.collect()

   720 

   721 def reportdiff(expected, output):

   722     import difflib

   723     print "*" * 70

   724     a = expected.splitlines(1)

   725     b = output.splitlines(1)

   726     sm = difflib.SequenceMatcher(a=a, b=b)

   727     tuples = sm.get_opcodes()

   728 

   729     def pair(x0, x1):

   730         # x0:x1 are 0-based slice indices; convert to 1-based line indices.

   731         x0 += 1

   732         if x0 >= x1:

   733             return "line " + str(x0)

   734         else:

   735             return "lines %d-%d" % (x0, x1)

   736 

   737     for op, a0, a1, b0, b1 in tuples:

   738         if op == 'equal':

   739             pass

   740 

   741         elif op == 'delete':

   742             print "***", pair(a0, a1), "of expected output missing:"

   743             for line in a[a0:a1]:

   744                 print "-", line,

   745 

   746         elif op == 'replace':

   747             print "*** mismatch between", pair(a0, a1), "of expected", \

   748                   "output and", pair(b0, b1), "of actual output:"

   749             for line in difflib.ndiff(a[a0:a1], b[b0:b1]):

   750                 print line,

   751 

   752         elif op == 'insert':

   753             print "***", pair(b0, b1), "of actual output doesn't appear", \

   754                   "in expected output after line", str(a1)+":"

   755             for line in b[b0:b1]:

   756                 print "+", line,

   757 

   758         else:

   759             print "get_opcodes() returned bad tuple?!?!", (op, a0, a1, b0, b1)

   760 

   761     print "*" * 70

   762 

   763 def findtestdir():

   764     if __name__ == '__main__':

   765         file = sys.argv[0]

   766     else:

   767         file = __file__

   768     testdir = os.path.dirname(file) or os.curdir

   769     return testdir

   770 

   771 def removepy(name):

   772     if name.endswith(os.extsep + "py"):

   773         name = name[:-3]

   774     return name

   775 

   776 def count(n, word):

   777     if n == 1:

   778         return "%d %s" % (n, word)

   779     else:

   780         return "%d %ss" % (n, word)

   781 

   782 def printlist(x, width=70, indent=4):

   783     """Print the elements of iterable x to stdout.

   784 

   785     Optional arg width (default 70) is the maximum line length.

   786     Optional arg indent (default 4) is the number of blanks with which to

   787     begin each line.

   788     """

   789 

   790     from textwrap import fill

   791     blanks = ' ' * indent

   792     print fill(' '.join(map(str, x)), width,

   793                initial_indent=blanks, subsequent_indent=blanks)

   794 

   795 # Map sys.platform to a string containing the basenames of tests

   796 # expected to be skipped on that platform.

   797 #

   798 # Special cases:

   799 #     test_pep277

   800 #         The _ExpectedSkips constructor adds this to the set of expected

   801 #         skips if not os.path.supports_unicode_filenames.

   802 #     test_socket_ssl

   803 #         Controlled by test_socket_ssl.skip_expected.  Requires the network

   804 #         resource, and a socket module with ssl support.

   805 #     test_timeout

   806 #         Controlled by test_timeout.skip_expected.  Requires the network

   807 #         resource and a socket module.

   808 

   809 _expectations = {

   810     'win32':

   811         """

   812         test__locale

   813         test_applesingle

   814         test_al

   815         test_bsddb185

   816         test_bsddb3

   817         test_cd

   818         test_cl

   819         test_commands

   820         test_crypt

   821         test_curses

   822         test_dbm

   823         test_dl

   824         test_fcntl

   825         test_fork1

   826         test_gdbm

   827         test_gl

   828         test_grp

   829         test_imgfile

   830         test_ioctl

   831         test_largefile

   832         test_linuxaudiodev

   833         test_mhlib

   834         test_nis

   835         test_openpty

   836         test_ossaudiodev

   837         test_poll

   838         test_posix

   839         test_pty

   840         test_pwd

   841         test_resource

   842         test_signal

   843         test_sunaudiodev

   844         test_threadsignals

   845         test_timing

   846         test_wait3

   847         test_wait4

   848         """,

   849     'linux2':

   850         """

   851         test_al

   852         test_applesingle

   853         test_bsddb185

   854         test_cd

   855         test_cl

   856         test_curses

   857         test_dl

   858         test_gl

   859         test_imgfile

   860         test_largefile

   861         test_linuxaudiodev

   862         test_nis

   863         test_ntpath

   864         test_ossaudiodev

   865         test_sqlite

   866         test_startfile

   867         test_sunaudiodev

   868         """,

   869    'mac':

   870         """

   871         test_al

   872         test_atexit

   873         test_bsddb

   874         test_bsddb185

   875         test_bsddb3

   876         test_bz2

   877         test_cd

   878         test_cl

   879         test_commands

   880         test_crypt

   881         test_curses

   882         test_dbm

   883         test_dl

   884         test_fcntl

   885         test_fork1

   886         test_gl

   887         test_grp

   888         test_ioctl

   889         test_imgfile

   890         test_largefile

   891         test_linuxaudiodev

   892         test_locale

   893         test_mmap

   894         test_nis

   895         test_ntpath

   896         test_openpty

   897         test_ossaudiodev

   898         test_poll

   899         test_popen

   900         test_popen2

   901         test_posix

   902         test_pty

   903         test_pwd

   904         test_resource

   905         test_signal

   906         test_sqlite

   907         test_startfile

   908         test_sunaudiodev

   909         test_sundry

   910         test_tarfile

   911         test_timing

   912         """,

   913     'unixware7':

   914         """

   915         test_al

   916         test_applesingle

   917         test_bsddb

   918         test_bsddb185

   919         test_cd

   920         test_cl

   921         test_dl

   922         test_gl

   923         test_imgfile

   924         test_largefile

   925         test_linuxaudiodev

   926         test_minidom

   927         test_nis

   928         test_ntpath

   929         test_openpty

   930         test_pyexpat

   931         test_sax

   932         test_startfile

   933         test_sqlite

   934         test_sunaudiodev

   935         test_sundry

   936         """,

   937     'openunix8':

   938         """

   939         test_al

   940         test_applesingle

   941         test_bsddb

   942         test_bsddb185

   943         test_cd

   944         test_cl

   945         test_dl

   946         test_gl

   947         test_imgfile

   948         test_largefile

   949         test_linuxaudiodev

   950         test_minidom

   951         test_nis

   952         test_ntpath

   953         test_openpty

   954         test_pyexpat

   955         test_sax

   956         test_sqlite

   957         test_startfile

   958         test_sunaudiodev

   959         test_sundry

   960         """,

   961     'sco_sv3':

   962         """

   963         test_al

   964         test_applesingle

   965         test_asynchat

   966         test_bsddb

   967         test_bsddb185

   968         test_cd

   969         test_cl

   970         test_dl

   971         test_fork1

   972         test_gettext

   973         test_gl

   974         test_imgfile

   975         test_largefile

   976         test_linuxaudiodev

   977         test_locale

   978         test_minidom

   979         test_nis

   980         test_ntpath

   981         test_openpty

   982         test_pyexpat

   983         test_queue

   984         test_sax

   985         test_sqlite

   986         test_startfile

   987         test_sunaudiodev

   988         test_sundry

   989         test_thread

   990         test_threaded_import

   991         test_threadedtempfile

   992         test_threading

   993         """,

   994     'riscos':

   995         """

   996         test_al

   997         test_applesingle

   998         test_asynchat

   999         test_atexit

  1000         test_bsddb

  1001         test_bsddb185

  1002         test_bsddb3

  1003         test_cd

  1004         test_cl

  1005         test_commands

  1006         test_crypt

  1007         test_dbm

  1008         test_dl

  1009         test_fcntl

  1010         test_fork1

  1011         test_gdbm

  1012         test_gl

  1013         test_grp

  1014         test_imgfile

  1015         test_largefile

  1016         test_linuxaudiodev

  1017         test_locale

  1018         test_mmap

  1019         test_nis

  1020         test_ntpath

  1021         test_openpty

  1022         test_poll

  1023         test_popen2

  1024         test_pty

  1025         test_pwd

  1026         test_strop

  1027         test_sqlite

  1028         test_startfile

  1029         test_sunaudiodev

  1030         test_sundry

  1031         test_thread

  1032         test_threaded_import

  1033         test_threadedtempfile

  1034         test_threading

  1035         test_timing

  1036         """,

  1037     'darwin':

  1038         """

  1039         test__locale

  1040         test_al

  1041         test_bsddb

  1042         test_bsddb3

  1043         test_cd

  1044         test_cl

  1045         test_curses

  1046         test_gdbm

  1047         test_gl

  1048         test_imgfile

  1049         test_largefile

  1050         test_linuxaudiodev

  1051         test_locale

  1052         test_minidom

  1053         test_nis

  1054         test_ntpath

  1055         test_ossaudiodev

  1056         test_poll

  1057         test_sqlite

  1058         test_startfile

  1059         test_sunaudiodev

  1060         """,

  1061     'sunos5':

  1062         """

  1063         test_al

  1064         test_applesingle

  1065         test_bsddb

  1066         test_bsddb185

  1067         test_cd

  1068         test_cl

  1069         test_curses

  1070         test_dbm

  1071         test_gdbm

  1072         test_gl

  1073         test_gzip

  1074         test_imgfile

  1075         test_linuxaudiodev

  1076         test_openpty

  1077         test_sqlite

  1078         test_startfile

  1079         test_zipfile

  1080         test_zlib

  1081         """,

  1082     'hp-ux11':

  1083         """

  1084         test_al

  1085         test_applesingle

  1086         test_bsddb

  1087         test_bsddb185

  1088         test_cd

  1089         test_cl

  1090         test_curses

  1091         test_dl

  1092         test_gdbm

  1093         test_gl

  1094         test_gzip

  1095         test_imgfile

  1096         test_largefile

  1097         test_linuxaudiodev

  1098         test_locale

  1099         test_minidom

  1100         test_nis

  1101         test_ntpath

  1102         test_openpty

  1103         test_pyexpat

  1104         test_sax

  1105         test_sqlite

  1106         test_startfile

  1107         test_sunaudiodev

  1108         test_zipfile

  1109         test_zlib

  1110         """,

  1111     'atheos':

  1112         """

  1113         test_al

  1114         test_applesingle

  1115         test_bsddb185

  1116         test_cd

  1117         test_cl

  1118         test_curses

  1119         test_dl

  1120         test_gdbm

  1121         test_gl

  1122         test_imgfile

  1123         test_largefile

  1124         test_linuxaudiodev

  1125         test_locale

  1126         test_mhlib

  1127         test_mmap

  1128         test_nis

  1129         test_poll

  1130         test_popen2

  1131         test_resource

  1132         test_sqlite

  1133         test_startfile

  1134         test_sunaudiodev

  1135         """,