1 

     2 :mod:`unittest` --- Unit testing framework

     3 ==========================================

     4 

     5 .. module:: unittest

     6    :synopsis: Unit testing framework for Python.

     7 .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>

     8 .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>

     9 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>

    10 .. sectionauthor:: Raymond Hettinger <python@rcn.com>

    11 

    12 

    13 .. versionadded:: 2.1

    14 

    15 The Python unit testing framework, sometimes referred to as "PyUnit," is a

    16 Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in

    17 turn, a Java version of Kent's Smalltalk testing framework.  Each is the de

    18 facto standard unit testing framework for its respective language.

    19 

    20 :mod:`unittest` supports test automation, sharing of setup and shutdown code for

    21 tests, aggregation of tests into collections, and independence of the tests from

    22 the reporting framework.  The :mod:`unittest` module provides classes that make

    23 it easy to support these qualities for a set of tests.

    24 

    25 To achieve this, :mod:`unittest` supports some important concepts:

    26 

    27 test fixture

    28    A :dfn:`test fixture` represents the preparation needed to perform one or more

    29    tests, and any associate cleanup actions.  This may involve, for example,

    30    creating temporary or proxy databases, directories, or starting a server

    31    process.

    32 

    33 test case

    34    A :dfn:`test case` is the smallest unit of testing.  It checks for a specific

    35    response to a particular set of inputs.  :mod:`unittest` provides a base class,

    36    :class:`TestCase`, which may be used to create new test cases.

    37 

    38 test suite

    39    A :dfn:`test suite` is a collection of test cases, test suites, or both.  It is

    40    used to aggregate tests that should be executed together.

    41 

    42 test runner

    43    A :dfn:`test runner` is a component which orchestrates the execution of tests

    44    and provides the outcome to the user.  The runner may use a graphical interface,

    45    a textual interface, or return a special value to indicate the results of

    46    executing the tests.

    47 

    48 The test case and test fixture concepts are supported through the

    49 :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be

    50 used when creating new tests, and the latter can be used when integrating

    51 existing test code with a :mod:`unittest`\ -driven framework. When building test

    52 fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods

    53 can be overridden to provide initialization and cleanup for the fixture.  With

    54 :class:`FunctionTestCase`, existing functions can be passed to the constructor

    55 for these purposes.  When the test is run, the fixture initialization is run

    56 first; if it succeeds, the cleanup method is run after the test has been

    57 executed, regardless of the outcome of the test.  Each instance of the

    58 :class:`TestCase` will only be used to run a single test method, so a new

    59 fixture is created for each test.

    60 

    61 Test suites are implemented by the :class:`TestSuite` class.  This class allows

    62 individual tests and test suites to be aggregated; when the suite is executed,

    63 all tests added directly to the suite and in "child" test suites are run.

    64 

    65 A test runner is an object that provides a single method, :meth:`run`, which

    66 accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and

    67 returns a result object.  The class :class:`TestResult` is provided for use as

    68 the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an

    69 example test runner which reports test results on the standard error stream by

    70 default.  Alternate runners can be implemented for other environments (such as

    71 graphical environments) without any need to derive from a specific class.

    72 

    73 

    74 .. seealso::

    75 

    76    Module :mod:`doctest`

    77       Another test-support module with a very different flavor.

    78 

    79    `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_

    80       Kent Beck's original paper on testing frameworks using the pattern shared by

    81       :mod:`unittest`.

    82 

    83 

    84 .. _unittest-minimal-example:

    85 

    86 Basic example

    87 -------------

    88 

    89 The :mod:`unittest` module provides a rich set of tools for constructing and

    90 running tests.  This section demonstrates that a small subset of the tools

    91 suffice to meet the needs of most users.

    92 

    93 Here is a short script to test three functions from the :mod:`random` module::

    94 

    95    import random

    96    import unittest

    97 

    98    class TestSequenceFunctions(unittest.TestCase):

    99 

   100        def setUp(self):

   101            self.seq = range(10)

   102 

   103        def testshuffle(self):

   104            # make sure the shuffled sequence does not lose any elements

   105            random.shuffle(self.seq)

   106            self.seq.sort()

   107            self.assertEqual(self.seq, range(10))

   108 

   109        def testchoice(self):

   110            element = random.choice(self.seq)

   111            self.assert_(element in self.seq)

   112 

   113        def testsample(self):

   114            self.assertRaises(ValueError, random.sample, self.seq, 20)

   115            for element in random.sample(self.seq, 5):

   116                self.assert_(element in self.seq)

   117 

   118    if __name__ == '__main__':

   119        unittest.main()

   120 

   121 A testcase is created by subclassing :class:`unittest.TestCase`. The three

   122 individual tests are defined with methods whose names start with the letters

   123 ``test``.  This naming convention informs the test runner about which methods

   124 represent tests.

   125 

   126 The crux of each test is a call to :meth:`assertEqual` to check for an expected

   127 result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify

   128 that an expected exception gets raised.  These methods are used instead of the

   129 :keyword:`assert` statement so the test runner can accumulate all test results

   130 and produce a report.

   131 

   132 When a :meth:`setUp` method is defined, the test runner will run that method

   133 prior to each test.  Likewise, if a :meth:`tearDown` method is defined, the test

   134 runner will invoke that method after each test.  In the example, :meth:`setUp`

   135 was used to create a fresh sequence for each test.

   136 

   137 The final block shows a simple way to run the tests. :func:`unittest.main`

   138 provides a command line interface to the test script.  When run from the command

   139 line, the above script produces an output that looks like this::

   140 

   141    ...

   142    ----------------------------------------------------------------------

   143    Ran 3 tests in 0.000s

   144 

   145    OK

   146 

   147 Instead of :func:`unittest.main`, there are other ways to run the tests with a

   148 finer level of control, less terse output, and no requirement to be run from the

   149 command line.  For example, the last two lines may be replaced with::

   150 

   151    suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)

   152    unittest.TextTestRunner(verbosity=2).run(suite)

   153 

   154 Running the revised script from the interpreter or another script produces the

   155 following output::

   156 

   157    testchoice (__main__.TestSequenceFunctions) ... ok

   158    testsample (__main__.TestSequenceFunctions) ... ok

   159    testshuffle (__main__.TestSequenceFunctions) ... ok

   160 

   161    ----------------------------------------------------------------------

   162    Ran 3 tests in 0.110s

   163 

   164    OK

   165 

   166 The above examples show the most commonly used :mod:`unittest` features which

   167 are sufficient to meet many everyday testing needs.  The remainder of the

   168 documentation explores the full feature set from first principles.

   169 

   170 

   171 .. _organizing-tests:

   172 

   173 Organizing test code

   174 --------------------

   175 

   176 The basic building blocks of unit testing are :dfn:`test cases` --- single

   177 scenarios that must be set up and checked for correctness.  In :mod:`unittest`,

   178 test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`

   179 class. To make your own test cases you must write subclasses of

   180 :class:`TestCase`, or use :class:`FunctionTestCase`.

   181 

   182 An instance of a :class:`TestCase`\ -derived class is an object that can

   183 completely run a single test method, together with optional set-up and tidy-up

   184 code.

   185 

   186 The testing code of a :class:`TestCase` instance should be entirely self

   187 contained, such that it can be run either in isolation or in arbitrary

   188 combination with any number of other test cases.

   189 

   190 The simplest :class:`TestCase` subclass will simply override the :meth:`runTest`

   191 method in order to perform specific testing code::

   192 

   193    import unittest

   194 

   195    class DefaultWidgetSizeTestCase(unittest.TestCase):

   196        def runTest(self):

   197            widget = Widget('The widget')

   198            self.assertEqual(widget.size(), (50, 50), 'incorrect default size')

   199 

   200 Note that in order to test something, we use the one of the :meth:`assert\*` or

   201 :meth:`fail\*` methods provided by the :class:`TestCase` base class.  If the

   202 test fails, an exception will be raised, and :mod:`unittest` will identify the

   203 test case as a :dfn:`failure`.  Any other exceptions will be treated as

   204 :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are

   205 caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are

   206 caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect

   207 function call.

   208 

   209 The way to run a test case will be described later.  For now, note that to

   210 construct an instance of such a test case, we call its constructor without

   211 arguments::

   212 

   213    testCase = DefaultWidgetSizeTestCase()

   214 

   215 Now, such test cases can be numerous, and their set-up can be repetitive.  In

   216 the above case, constructing a :class:`Widget` in each of 100 Widget test case

   217 subclasses would mean unsightly duplication.

   218 

   219 Luckily, we can factor out such set-up code by implementing a method called

   220 :meth:`setUp`, which the testing framework will automatically call for us when

   221 we run the test::

   222 

   223    import unittest

   224 

   225    class SimpleWidgetTestCase(unittest.TestCase):

   226        def setUp(self):

   227            self.widget = Widget('The widget')

   228 

   229    class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):

   230        def runTest(self):

   231            self.failUnless(self.widget.size() == (50,50),

   232                            'incorrect default size')

   233 

   234    class WidgetResizeTestCase(SimpleWidgetTestCase):

   235        def runTest(self):

   236            self.widget.resize(100,150)

   237            self.failUnless(self.widget.size() == (100,150),

   238                            'wrong size after resize')

   239 

   240 If the :meth:`setUp` method raises an exception while the test is running, the

   241 framework will consider the test to have suffered an error, and the

   242 :meth:`runTest` method will not be executed.

   243 

   244 Similarly, we can provide a :meth:`tearDown` method that tidies up after the

   245 :meth:`runTest` method has been run::

   246 

   247    import unittest

   248 

   249    class SimpleWidgetTestCase(unittest.TestCase):

   250        def setUp(self):

   251            self.widget = Widget('The widget')

   252 

   253        def tearDown(self):

   254            self.widget.dispose()

   255            self.widget = None

   256 

   257 If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether

   258 :meth:`runTest` succeeded or not.

   259 

   260 Such a working environment for the testing code is called a :dfn:`fixture`.

   261 

   262 Often, many small test cases will use the same fixture.  In this case, we would

   263 end up subclassing :class:`SimpleWidgetTestCase` into many small one-method

   264 classes such as :class:`DefaultWidgetSizeTestCase`.  This is time-consuming and

   265 discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler

   266 mechanism::

   267 

   268    import unittest

   269 

   270    class WidgetTestCase(unittest.TestCase):

   271        def setUp(self):

   272            self.widget = Widget('The widget')

   273 

   274        def tearDown(self):

   275            self.widget.dispose()

   276            self.widget = None

   277 

   278        def testDefaultSize(self):

   279            self.failUnless(self.widget.size() == (50,50),

   280                            'incorrect default size')

   281 

   282        def testResize(self):

   283            self.widget.resize(100,150)

   284            self.failUnless(self.widget.size() == (100,150),

   285                            'wrong size after resize')

   286 

   287 Here we have not provided a :meth:`runTest` method, but have instead provided

   288 two different test methods.  Class instances will now each run one of the

   289 :meth:`test\*`  methods, with ``self.widget`` created and destroyed separately

   290 for each instance.  When creating an instance we must specify the test method it

   291 is to run.  We do this by passing the method name in the constructor::

   292 

   293    defaultSizeTestCase = WidgetTestCase('testDefaultSize')

   294    resizeTestCase = WidgetTestCase('testResize')

   295 

   296 Test case instances are grouped together according to the features they test.

   297 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,

   298 represented by :mod:`unittest`'s :class:`TestSuite` class::

   299 

   300    widgetTestSuite = unittest.TestSuite()

   301    widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))

   302    widgetTestSuite.addTest(WidgetTestCase('testResize'))

   303 

   304 For the ease of running tests, as we will see later, it is a good idea to

   305 provide in each test module a callable object that returns a pre-built test

   306 suite::

   307 

   308    def suite():

   309        suite = unittest.TestSuite()

   310        suite.addTest(WidgetTestCase('testDefaultSize'))

   311        suite.addTest(WidgetTestCase('testResize'))

   312        return suite

   313 

   314 or even::

   315 

   316    def suite():

   317        tests = ['testDefaultSize', 'testResize']

   318 

   319        return unittest.TestSuite(map(WidgetTestCase, tests))

   320 

   321 Since it is a common pattern to create a :class:`TestCase` subclass with many

   322 similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`

   323 class that can be used to automate the process of creating a test suite and

   324 populating it with individual tests. For example, ::

   325 

   326    suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)

   327 

   328 will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and

   329 ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method

   330 name prefix to identify test methods automatically.

   331 

   332 Note that the order in which the various test cases will be run is determined by

   333 sorting the test function names with the built-in :func:`cmp` function.

   334 

   335 Often it is desirable to group suites of test cases together, so as to run tests

   336 for the whole system at once.  This is easy, since :class:`TestSuite` instances

   337 can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be

   338 added to a :class:`TestSuite`::

   339 

   340    suite1 = module1.TheTestSuite()

   341    suite2 = module2.TheTestSuite()

   342    alltests = unittest.TestSuite([suite1, suite2])

   343 

   344 You can place the definitions of test cases and test suites in the same modules

   345 as the code they are to test (such as :file:`widget.py`), but there are several

   346 advantages to placing the test code in a separate module, such as

   347 :file:`test_widget.py`:

   348 

   349 * The test module can be run standalone from the command line.

   350 

   351 * The test code can more easily be separated from shipped code.

   352 

   353 * There is less temptation to change test code to fit the code it tests without

   354   a good reason.

   355 

   356 * Test code should be modified much less frequently than the code it tests.

   357 

   358 * Tested code can be refactored more easily.

   359 

   360 * Tests for modules written in C must be in separate modules anyway, so why not

   361   be consistent?

   362 

   363 * If the testing strategy changes, there is no need to change the source code.

   364 

   365 

   366 .. _legacy-unit-tests:

   367 

   368 Re-using old test code

   369 ----------------------

   370 

   371 Some users will find that they have existing test code that they would like to

   372 run from :mod:`unittest`, without converting every old test function to a

   373 :class:`TestCase` subclass.

   374 

   375 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.

   376 This subclass of :class:`TestCase` can be used to wrap an existing test

   377 function.  Set-up and tear-down functions can also be provided.

   378 

   379 Given the following test function::

   380 

   381    def testSomething():

   382        something = makeSomething()

   383        assert something.name is not None

   384        # ...

   385 

   386 one can create an equivalent test case instance as follows::

   387 

   388    testcase = unittest.FunctionTestCase(testSomething)

   389 

   390 If there are additional set-up and tear-down methods that should be called as

   391 part of the test case's operation, they can also be provided like so::

   392 

   393    testcase = unittest.FunctionTestCase(testSomething,

   394                                         setUp=makeSomethingDB,

   395                                         tearDown=deleteSomethingDB)

   396 

   397 To make migrating existing test suites easier, :mod:`unittest` supports tests

   398 raising :exc:`AssertionError` to indicate test failure. However, it is

   399 recommended that you use the explicit :meth:`TestCase.fail\*` and

   400 :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`

   401 may treat :exc:`AssertionError` differently.

   402 

   403 .. note::

   404 

   405    Even though :class:`FunctionTestCase` can be used to quickly convert an existing

   406    test base over to a :mod:`unittest`\ -based system, this approach is not

   407    recommended.  Taking the time to set up proper :class:`TestCase` subclasses will

   408    make future test refactorings infinitely easier.

   409 

   410 

   411 .. _unittest-contents:

   412 

   413 Classes and functions

   414 ---------------------

   415 

   416 

   417 .. class:: TestCase([methodName])

   418 

   419    Instances of the :class:`TestCase` class represent the smallest testable units

   420    in the :mod:`unittest` universe.  This class is intended to be used as a base

   421    class, with specific tests being implemented by concrete subclasses.  This class

   422    implements the interface needed by the test runner to allow it to drive the

   423    test, and methods that the test code can use to check for and report various

   424    kinds of failure.

   425 

   426    Each instance of :class:`TestCase` will run a single test method: the method

   427    named *methodName*.  If you remember, we had an earlier example that went

   428    something like this::

   429 

   430       def suite():

   431           suite = unittest.TestSuite()

   432           suite.addTest(WidgetTestCase('testDefaultSize'))

   433           suite.addTest(WidgetTestCase('testResize'))

   434           return suite

   435 

   436    Here, we create two instances of :class:`WidgetTestCase`, each of which runs a

   437    single test.

   438 

   439    *methodName* defaults to ``'runTest'``.

   440 

   441 

   442 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])

   443 

   444    This class implements the portion of the :class:`TestCase` interface which

   445    allows the test runner to drive the test, but does not provide the methods which

   446    test code can use to check and report errors. This is used to create test cases

   447    using legacy test code, allowing it to be integrated into a :mod:`unittest`\

   448    -based test framework.

   449 

   450 

   451 .. class:: TestSuite([tests])

   452 

   453    This class represents an aggregation of individual tests cases and test suites.

   454    The class presents the interface needed by the test runner to allow it to be run

   455    as any other test case.  Running a :class:`TestSuite` instance is the same as

   456    iterating over the suite, running each test individually.

   457 

   458    If *tests* is given, it must be an iterable of individual test cases or other

   459    test suites that will be used to build the suite initially. Additional methods

   460    are provided to add test cases and suites to the collection later on.

   461 

   462 

   463 .. class:: TestLoader()

   464 

   465    This class is responsible for loading tests according to various criteria and

   466    returning them wrapped in a :class:`TestSuite`. It can load all tests within a

   467    given module or :class:`TestCase` subclass.

   468 

   469 

   470 .. class:: TestResult()

   471 

   472    This class is used to compile information about which tests have succeeded and

   473    which have failed.

   474 

   475 

   476 .. data:: defaultTestLoader

   477 

   478    Instance of the :class:`TestLoader` class intended to be shared.  If no

   479    customization of the :class:`TestLoader` is needed, this instance can be used

   480    instead of repeatedly creating new instances.

   481 

   482 

   483 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])

   484 

   485    A basic test runner implementation which prints results on standard error.  It

   486    has a few configurable parameters, but is essentially very simple.  Graphical

   487    applications which run test suites should provide alternate implementations.

   488 

   489 

   490 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])

   491 

   492    A command-line program that runs a set of tests; this is primarily for making

   493    test modules conveniently executable.  The simplest use for this function is to

   494    include the following line at the end of a test script::

   495 

   496       if __name__ == '__main__':

   497           unittest.main()

   498 

   499    The *testRunner* argument can either be a test runner class or an already

   500    created instance of it.

   501 

   502 In some cases, the existing tests may have been written using the :mod:`doctest`

   503 module.  If so, that module provides a  :class:`DocTestSuite` class that can

   504 automatically build :class:`unittest.TestSuite` instances from the existing

   505 :mod:`doctest`\ -based tests.

   506 

   507 .. versionadded:: 2.3

   508 

   509 

   510 .. _testcase-objects:

   511 

   512 TestCase Objects

   513 ----------------

   514 

   515 Each :class:`TestCase` instance represents a single test, but each concrete

   516 subclass may be used to define multiple tests --- the concrete class represents

   517 a single test fixture.  The fixture is created and cleaned up for each test

   518 case.

   519 

   520 :class:`TestCase` instances provide three groups of methods: one group used to

   521 run the test, another used by the test implementation to check conditions and

   522 report failures, and some inquiry methods allowing information about the test

   523 itself to be gathered.

   524 

   525 Methods in the first group (running the test) are:

   526 

   527 

   528 .. method:: TestCase.setUp()

   529 

   530    Method called to prepare the test fixture.  This is called immediately before

   531    calling the test method; any exception raised by this method will be considered

   532    an error rather than a test failure. The default implementation does nothing.

   533 

   534 

   535 .. method:: TestCase.tearDown()

   536 

   537    Method called immediately after the test method has been called and the result

   538    recorded.  This is called even if the test method raised an exception, so the

   539    implementation in subclasses may need to be particularly careful about checking

   540    internal state.  Any exception raised by this method will be considered an error

   541    rather than a test failure.  This method will only be called if the

   542    :meth:`setUp` succeeds, regardless of the outcome of the test method. The

   543    default implementation does nothing.

   544 

   545 

   546 .. method:: TestCase.run([result])

   547 

   548    Run the test, collecting the result into the test result object passed as

   549    *result*.  If *result* is omitted or :const:`None`, a temporary result object is

   550    created (by calling the :meth:`defaultTestCase` method) and used; this result

   551    object is not returned to :meth:`run`'s caller.

   552 

   553    The same effect may be had by simply calling the :class:`TestCase` instance.

   554 

   555 

   556 .. method:: TestCase.debug()

   557 

   558    Run the test without collecting the result.  This allows exceptions raised by

   559    the test to be propagated to the caller, and can be used to support running

   560    tests under a debugger.

   561 

   562 The test code can use any of the following methods to check for and report

   563 failures.

   564 

   565 

   566 .. method:: TestCase.assert_(expr[, msg])

   567             TestCase.failUnless(expr[, msg])

   568             TestCase.assertTrue(expr[, msg])

   569 

   570    Signal a test failure if *expr* is false; the explanation for the error will be

   571    *msg* if given, otherwise it will be :const:`None`.

   572 

   573 

   574 .. method:: TestCase.assertEqual(first, second[, msg])

   575             TestCase.failUnlessEqual(first, second[, msg])

   576 

   577    Test that *first* and *second* are equal.  If the values do not compare equal,

   578    the test will fail with the explanation given by *msg*, or :const:`None`.  Note

   579    that using :meth:`failUnlessEqual` improves upon doing the comparison as the

   580    first parameter to :meth:`failUnless`:  the default value for *msg* can be

   581    computed to include representations of both *first* and *second*.

   582 

   583 

   584 .. method:: TestCase.assertNotEqual(first, second[, msg])

   585             TestCase.failIfEqual(first, second[, msg])

   586 

   587    Test that *first* and *second* are not equal.  If the values do compare equal,

   588    the test will fail with the explanation given by *msg*, or :const:`None`.  Note

   589    that using :meth:`failIfEqual` improves upon doing the comparison as the first

   590    parameter to :meth:`failUnless` is that the default value for *msg* can be

   591    computed to include representations of both *first* and *second*.

   592 

   593 

   594 .. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]])

   595             TestCase.failUnlessAlmostEqual(first, second[, places[, msg]])

   596 

   597    Test that *first* and *second* are approximately equal by computing the

   598    difference, rounding to the given number of decimal *places* (default 7), 

   599    and comparing to zero.

   600    Note that comparing a given number of decimal places is not the same as

   601    comparing a given number of significant digits. If the values do not compare

   602    equal, the test will fail with the explanation given by *msg*, or :const:`None`.

   603 

   604 

   605 .. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]])

   606             TestCase.failIfAlmostEqual(first, second[, places[, msg]])

   607 

   608    Test that *first* and *second* are not approximately equal by computing the

   609    difference, rounding to the given number of decimal *places* (default 7), 

   610    and comparing to zero.

   611    Note that comparing a given number of decimal places is not the same as

   612    comparing a given number of significant digits. If the values do not compare

   613    equal, the test will fail with the explanation given by *msg*, or :const:`None`.

   614 

   615 

   616 .. method:: TestCase.assertRaises(exception, callable, ...)

   617             TestCase.failUnlessRaises(exception, callable, ...)

   618 

   619    Test that an exception is raised when *callable* is called with any positional

   620    or keyword arguments that are also passed to :meth:`assertRaises`.  The test

   621    passes if *exception* is raised, is an error if another exception is raised, or

   622    fails if no exception is raised.  To catch any of a group of exceptions, a tuple

   623    containing the exception classes may be passed as *exception*.

   624 

   625 

   626 .. method:: TestCase.failIf(expr[, msg])

   627             TestCase.assertFalse(expr[, msg])

   628 

   629    The inverse of the :meth:`failUnless` method is the :meth:`failIf` method.  This

   630    signals a test failure if *expr* is true, with *msg* or :const:`None` for the

   631    error message.

   632 

   633 

   634 .. method:: TestCase.fail([msg])

   635 

   636    Signals a test failure unconditionally, with *msg* or :const:`None` for the

   637    error message.

   638 

   639 

   640 .. attribute:: TestCase.failureException

   641 

   642    This class attribute gives the exception raised by the :meth:`test` method.  If

   643    a test framework needs to use a specialized exception, possibly to carry

   644    additional information, it must subclass this exception in order to "play fair"

   645    with the framework.  The initial value of this attribute is

   646    :exc:`AssertionError`.

   647 

   648 Testing frameworks can use the following methods to collect information on the

   649 test:

   650 

   651 

   652 .. method:: TestCase.countTestCases()

   653 

   654    Return the number of tests represented by this test object.  For

   655    :class:`TestCase` instances, this will always be ``1``.

   656 

   657 

   658 .. method:: TestCase.defaultTestResult()

   659 

   660    Return an instance of the test result class that should be used for this test

   661    case class (if no other result instance is provided to the :meth:`run` method).

   662 

   663    For :class:`TestCase` instances, this will always be an instance of

   664    :class:`TestResult`;  subclasses of :class:`TestCase` should override this as

   665    necessary.

   666 

   667 

   668 .. method:: TestCase.id()

   669 

   670    Return a string identifying the specific test case.  This is usually the full

   671    name of the test method, including the module and class name.

   672 

   673 

   674 .. method:: TestCase.shortDescription()

   675 

   676    Returns a one-line description of the test, or :const:`None` if no description

   677    has been provided.  The default implementation of this method returns the first

   678    line of the test method's docstring, if available, or :const:`None`.

   679 

   680 

   681 .. _testsuite-objects:

   682 

   683 TestSuite Objects

   684 -----------------

   685 

   686 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except

   687 they do not actually implement a test.  Instead, they are used to aggregate

   688 tests into groups of tests that should be run together. Some additional methods

   689 are available to add tests to :class:`TestSuite` instances:

   690 

   691 

   692 .. method:: TestSuite.addTest(test)

   693 

   694    Add a :class:`TestCase` or :class:`TestSuite` to the suite.

   695 

   696 

   697 .. method:: TestSuite.addTests(tests)

   698 

   699    Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`

   700    instances to this test suite.

   701 

   702    This is equivalent to iterating over *tests*, calling :meth:`addTest` for each

   703    element.

   704 

   705 :class:`TestSuite` shares the following methods with :class:`TestCase`:

   706 

   707 

   708 .. method:: TestSuite.run(result)

   709 

   710    Run the tests associated with this suite, collecting the result into the test

   711    result object passed as *result*.  Note that unlike :meth:`TestCase.run`,

   712    :meth:`TestSuite.run` requires the result object to be passed in.

   713 

   714 

   715 .. method:: TestSuite.debug()

   716 

   717    Run the tests associated with this suite without collecting the result. This

   718    allows exceptions raised by the test to be propagated to the caller and can be

   719    used to support running tests under a debugger.

   720 

   721 

   722 .. method:: TestSuite.countTestCases()

   723 

   724    Return the number of tests represented by this test object, including all

   725    individual tests and sub-suites.

   726 

   727 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is

   728 invoked by a :class:`TestRunner` rather than by the end-user test harness.

   729 

   730 

   731 .. _testresult-objects:

   732 

   733 TestResult Objects

   734 ------------------

   735 

   736 A :class:`TestResult` object stores the results of a set of tests.  The

   737 :class:`TestCase` and :class:`TestSuite` classes ensure that results are

   738 properly recorded; test authors do not need to worry about recording the outcome

   739 of tests.

   740 

   741 Testing frameworks built on top of :mod:`unittest` may want access to the

   742 :class:`TestResult` object generated by running a set of tests for reporting

   743 purposes; a :class:`TestResult` instance is returned by the

   744 :meth:`TestRunner.run` method for this purpose.

   745 

   746 :class:`TestResult` instances have the following attributes that will be of

   747 interest when inspecting the results of running a set of tests:

   748 

   749 

   750 .. attribute:: TestResult.errors

   751 

   752    A list containing 2-tuples of :class:`TestCase` instances and strings holding

   753    formatted tracebacks. Each tuple represents a test which raised an unexpected

   754    exception.

   755 

   756    .. versionchanged:: 2.2

   757       Contains formatted tracebacks instead of :func:`sys.exc_info` results.

   758 

   759 

   760 .. attribute:: TestResult.failures

   761 

   762    A list containing 2-tuples of :class:`TestCase` instances and strings holding

   763    formatted tracebacks. Each tuple represents a test where a failure was

   764    explicitly signalled using the :meth:`TestCase.fail\*` or

   765    :meth:`TestCase.assert\*` methods.

   766 

   767    .. versionchanged:: 2.2

   768       Contains formatted tracebacks instead of :func:`sys.exc_info` results.

   769 

   770 

   771 .. attribute:: TestResult.testsRun

   772 

   773    The total number of tests run so far.

   774 

   775 

   776 .. method:: TestResult.wasSuccessful()

   777 

   778    Returns :const:`True` if all tests run so far have passed, otherwise returns

   779    :const:`False`.

   780 

   781 

   782 .. method:: TestResult.stop()

   783 

   784    This method can be called to signal that the set of tests being run should be

   785    aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to

   786    :const:`True`.  :class:`TestRunner` objects should respect this flag and return

   787    without running any additional tests.

   788 

   789    For example, this feature is used by the :class:`TextTestRunner` class to stop

   790    the test framework when the user signals an interrupt from the keyboard.

   791    Interactive tools which provide :class:`TestRunner` implementations can use this

   792    in a similar manner.

   793 

   794 The following methods of the :class:`TestResult` class are used to maintain the

   795 internal data structures, and may be extended in subclasses to support

   796 additional reporting requirements.  This is particularly useful in building

   797 tools which support interactive reporting while tests are being run.

   798 

   799 

   800 .. method:: TestResult.startTest(test)

   801 

   802    Called when the test case *test* is about to be run.

   803 

   804    The default implementation simply increments the instance's ``testsRun``

   805    counter.

   806 

   807 

   808 .. method:: TestResult.stopTest(test)

   809 

   810    Called after the test case *test* has been executed, regardless of the outcome.

   811 

   812    The default implementation does nothing.

   813 

   814 

   815 .. method:: TestResult.addError(test, err)

   816 

   817    Called when the test case *test* raises an unexpected exception *err* is a tuple

   818    of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.

   819 

   820    The default implementation appends a tuple ``(test, formatted_err)`` to the

   821    instance's ``errors`` attribute, where *formatted_err* is a formatted

   822    traceback derived from *err*.

   823 

   824 

   825 .. method:: TestResult.addFailure(test, err)

   826 

   827    Called when the test case *test* signals a failure. *err* is a tuple of the form

   828    returned by :func:`sys.exc_info`:  ``(type, value, traceback)``.

   829 

   830    The default implementation appends a tuple ``(test, formatted_err)`` to the

   831    instance's ``failures`` attribute, where *formatted_err* is a formatted

   832    traceback derived from *err*.

   833 

   834 

   835 .. method:: TestResult.addSuccess(test)

   836 

   837    Called when the test case *test* succeeds.

   838 

   839    The default implementation does nothing.

   840 

   841 

   842 .. _testloader-objects:

   843 

   844 TestLoader Objects

   845 ------------------

   846 

   847 The :class:`TestLoader` class is used to create test suites from classes and

   848 modules.  Normally, there is no need to create an instance of this class; the

   849 :mod:`unittest` module provides an instance that can be shared as

   850 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows

   851 customization of some configurable properties.

   852 

   853 :class:`TestLoader` objects have the following methods:

   854 

   855 

   856 .. method:: TestLoader.loadTestsFromTestCase(testCaseClass)

   857 

   858    Return a suite of all tests cases contained in the :class:`TestCase`\ -derived

   859    :class:`testCaseClass`.

   860 

   861 

   862 .. method:: TestLoader.loadTestsFromModule(module)

   863 

   864    Return a suite of all tests cases contained in the given module. This method

   865    searches *module* for classes derived from :class:`TestCase` and creates an

   866    instance of the class for each test method defined for the class.

   867 

   868    .. warning::

   869 

   870       While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient

   871       in sharing fixtures and helper functions, defining test methods on base classes

   872       that are not intended to be instantiated directly does not play well with this

   873       method.  Doing so, however, can be useful when the fixtures are different and

   874       defined in subclasses.

   875 

   876 

   877 .. method:: TestLoader.loadTestsFromName(name[, module])

   878 

   879    Return a suite of all tests cases given a string specifier.

   880 

   881    The specifier *name* is a "dotted name" that may resolve either to a module, a

   882    test case class, a test method within a test case class, a :class:`TestSuite`

   883    instance, or a callable object which returns a :class:`TestCase` or

   884    :class:`TestSuite` instance.  These checks are applied in the order listed here;

   885    that is, a method on a possible test case class will be picked up as "a test

   886    method within a test case class", rather than "a callable object".

   887 

   888    For example, if you have a module :mod:`SampleTests` containing a

   889    :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test

   890    methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the

   891    specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a

   892    suite which will run all three test methods.  Using the specifier

   893    ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite

   894    which will run only the :meth:`test_two` test method.  The specifier can refer

   895    to modules and packages which have not been imported; they will be imported as a

   896    side-effect.

   897 

   898    The method optionally resolves *name* relative to the given *module*.

   899 

   900 

   901 .. method:: TestLoader.loadTestsFromNames(names[, module])

   902 

   903    Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than

   904    a single name.  The return value is a test suite which supports all the tests

   905    defined for each name.

   906 

   907 

   908 .. method:: TestLoader.getTestCaseNames(testCaseClass)

   909 

   910    Return a sorted sequence of method names found within *testCaseClass*; this

   911    should be a subclass of :class:`TestCase`.

   912 

   913 The following attributes of a :class:`TestLoader` can be configured either by

   914 subclassing or assignment on an instance:

   915 

   916 

   917 .. attribute:: TestLoader.testMethodPrefix

   918 

   919    String giving the prefix of method names which will be interpreted as test

   920    methods.  The default value is ``'test'``.

   921 

   922    This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`

   923    methods.

   924 

   925 

   926 .. attribute:: TestLoader.sortTestMethodsUsing

   927 

   928    Function to be used to compare method names when sorting them in

   929    :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The

   930    default value is the built-in :func:`cmp` function; the attribute can also be

   931    set to :const:`None` to disable the sort.

   932 

   933 

   934 .. attribute:: TestLoader.suiteClass

   935 

   936    Callable object that constructs a test suite from a list of tests. No methods on

   937    the resulting object are needed.  The default value is the :class:`TestSuite`

   938    class.

   939 

   940    This affects all the :meth:`loadTestsFrom\*` methods.

   941