Posted by: John Bresnahan | September 22, 2012

Generated Nosetests in Classes

The Topic
This entry is about creating tests for the python testing framework nosetests.  Specifically it focuses on generating methods in code for Test objects such that each method becomes a single test recognized by nose as tho it were manually typed in.

With nosetests you can create a module of tests such that each test is a function in the module (file), for example:

def test_two_plus_two():
    x = 2 + 2
    assert(x == 4)

or you can create a unittest.TestCase object, like this:

import unittest

class TestMath(unittest.TestCase):

    def test_two_plus_two(self):
        x = 2 + 2
        self.assertTrue(x == 4)

In this blog we will focus on the second type.

Generating Tests
Often it is convenient to generate tests. Imaging you have a test that takes an integer as parameter. Each value of that integer makes for a new and distinct test worth of its own setUp() and teardown() call. If the integer value can range from 0 to 10000, the last thing you want to do is manually write a test for each value. What you want to do is generate each test.

As an example say you have a system for which you wrote a very basic test. The test always passes when things are good, but you want to run that test while introducing several distinct different types of chaos. The test code that you are calling remains the same (and it could be fairly complicated when including setUp and teardown), but the type of chaos varies over 1000 different types. Here is what that test could look like.

import unittest

class TestChaos(unittest.TestCase):

    def test_simple_system_with_chaos(self):
        start_chaos(chaos_type=1)
        do_the_simple_test()

Here is how you turn that one simple test into 1000 distinct tests:

import unittest

class TestChaos(unittest.TestCase):

    def _simple_system_with_chaos(self, n):
        start_chaos(chaos_type=n)
        do_the_simple_test()


def create_em(n):
    def doit(self):
        self._simple_system_with_chaos(n)
    return doit

for n in range(0, 1000):
    method = create_create(n)
    method.__name__ = 'test_simple_system_with_chaos_type_%d" % (n)
    setattr(TestChaos, method.__name__, method)

del method

Here is how it works. nosetests opens up the file and loads it as a module. As the module is loaded the top level code, which includes the last range() loop is executed. This loop dynamically creates functions and attaches them to the defined class TestChaos.

Once that is complete, and nosetests has fully loaded the module, It calls dir() on it and looks for any class that has the string “Test” in it. For everyone that it finds it looks for any method with the string “test” in its name. When the module was loaded our source code created 1000 of them which now appear to nosetests as if we simply typed them in and processing proceeds as such.

Open Question
Something to note: The very last line of the example is del method. This is needed, because when nosetests does the dir() on the module it not only looks for classes but also for functions. When the loop is done iterating it leaves a “method” in the module which appears to the dir() call to be a function, and thus it trys to call it as a test, only because it is not attached to a class it does not receive the self argument and the extra and unwanted test fails with a strange error. I am not sure why this happens, I would have thought it would go out of scope but empirical testing on python 2.7 shows this is not the case. If anyone knows, please tell me. In any case the del method statement solves it.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: