Test classes, singular or plural?

Sunday 7 February 2010

A minor hiccup in writing unit tests is how to name the classes that contain them. The jUnit style of test class, which has been adopted by virtually everyone, including Python's unittest module, is that tests are methods of a class. The class is instantiated once for each test method, then three methods are called: setUp, the test method, and tearDown.

As a result, you end up with test classes that look like this:

# Tests for double_it, and no, no one would write them this way...
class DoubleItTests(unittest.TestCase):
    def test_ten(self):
        assert double_it(10) == 20
    def test_twenty(self):
        assert double_it(20) == 40

Here I've named the class DoubleItTests, plural. That's because I can see that it's a container for a number of tests. This feels right if you think about the class simply as a namespace for the test methods.

But what is instantiated from the class? Only single tests. In this case, the class will be instantiated twice, once to run test_ten, and once to run test_twenty. The class' name should really be the name of the objects. No one would name their user class Users under the theory that the class encompasses a number of users.

So the test class should really be called DoubleItTest, which I guess fits in with the unittest.TestCase base class it derives from. But somehow it just looks wrong.

This is reminiscent of the SQL table naming dilemma. Is it the CUSTOMER table, or the CUSTOMERS table? How you feel about it seems to come down to whether you think natively in SQL, or whether it's just a backing store for your ORM.

I'm getting used to the singular test class name, but it still doesn't come naturally, I have to remind myself to leave off those tempting plurals.


Doug Latornell 9:17 PM on 7 Feb 2010

Interesting observation, Ned. I'd never thought much about that aspect of test class naming because I learned to write my test suites by following the example of folks who named their test classes like TestDoubleIt. So, for me Test in the class name is a verb, not a noun, and the plural/singular issue (that would have bugged me too) never arose.

Thanks for pointing out the blinders I've been wearing... I think...

Casey Duncan 9:17 PM on 7 Feb 2010

I've come to the conclusion that suffixing these classes "TestCase" is the least confusing to me. It also makes sense since you are subclassing unittest.TestCase anyhow. Plus I often find myself creating "mock" classes containing the word "Test" so it helps to differentiate the scaffolding. I know I could name them MockSomethingOrOther, but I don't like to, but not for any logical reason. It just sounds weird to me.

Ben Finney 1:57 AM on 8 Feb 2010

As you point out, a subclass of TestCase is not a container, so shouldn't be plural. I use the same convention as Casey: a class defining test cases for DoubleIt would be DoubleIt_TestCase. (PEP 8 gets bent, here, because I want the name of the thing to be tested to pop out visually.)

As for naming entities in SQL? Relations aren't containers either. I try to make my databases relational, even where SQL's non-relational warts make that difficult; so all the relations are named in the singular.

Eli 5:06 AM on 8 Feb 2010

I just use "test" as a verb and not as a noun. Thus:

class Test_DoubleIt(unittest.TestCase):
    def test_ten(self):
        assert double_it(10) == 20
    def test_twenty(self):
        assert double_it(20) == 40
All my test classes start with Test_, so I don't have to think about singular or plural. Moreover, it blends nicely with the test themselves starting with test_.

Nate Finch 7:16 AM on 8 Feb 2010

Our test classes are all plural and I think that's exactly the right thing to do. I think the Users analogy is spurious - if you actually had a class like

class Users:
    def GetBob():
        return Bob
    def GetJill():
        return Jill
Then calling it plural would be correct. It really is just a container, just like the unit test container... even though you only use it for one test/user at a time.

This is especially apt for the testcases, because programmers aren't ever going to actually instantiate the class themselves. You'll never see

testcase = DoubleItTests()

It's the nature of the JUnit style that it's all automated, and the classes really *are* just a namespace... so don't fight it, let it be plural... since most of the time you're talking about them, you're talking about them in the plural "hey - one of the DoubleItTests failed". "Can you add that as a new test to the DoubleItTests?" "Where's that test again? Oh right, in DoubleItTests".

Brandon Craig Rhodes 8:37 AM on 8 Feb 2010

First, I think that Nate Finch is on to something, and that the plural form has a lot to recommend it.

Second, has it ever occurred to you that this naming problem might be a symptom of something deeper — namely, that it's a semantic confusion to wrap tests in a class in the first place, when they're not behaviors of a persistent object (which is what the "class" concept was invented to express in the first place)? Precisely this kind of semantic overhead is the reason that I encourage everyone to try out "nose" or "py.test", and simply write their test cases as functions, and see whether the semantics flatten out into something easier to manage. A test class might make sense if setUp() sets several instance variables that the tests then use to avoid creating a test database over and over again — in which case ExampleDatabaseTests becomes a reasonable thing to have a name for! But in other cases the class is just cognitive overhead. And given that "nose" and "py.test" have ways to have setUp and tearDown as simple functions, I'm not even sure whether a class is ever useful!

Casey Duncan 11:11 AM on 8 Feb 2010

@Brandon Craig Rhodes I find it is usually the case that when I have trouble naming something it usually points to a deeper problem. It's definitely one of those "gut feel" things that can seem trivial, but is almost always worth taking a step back and examining. The unittest module brings with it a lot of Java baggage to be sure. The only time I really find the TestCase class construct useful is when I want to apply the same "abstract" tests to several concrete implementations. Then it is handy to create a base TestCase class (subclassing only object), then mix-in unittest.TestCase for the individual implementations, overriding setUp() and adding implementation-specific test methods. This capability is very handy when you need it, but it's pretty rare.

More commonly, I just use the TestCase subclasses as a way to organize the tests. For example, one class being tested typically gets one TestCase. A module under test gets one test script/module. This of course breaks down when testing functions or doing integration testing though.

At the test module level, I name things moduleundertest_test.py. Mostly because having a directory full of test scripts prefixed with the word "test" is really annoying when you want to run just one from the shell. But that's a different matter.

A nitpick I have with unittest is the spelling of setUp(). Even after using unittest for many many years, I'm constantly leaving the "U" lower-case, because it's one word damn it all! Even ignoring pep8 it's still wrong. Sometimes it's the littlest things than can slowly drive us crazy. 8^)

AdSR 3:35 PM on 8 Feb 2010

Interestingly, one of the (inconsistently used) practices in Ruby is to name test classes TestSomething - TestDoubleIt in your case. However, this creates a verb which is against the tradition of class names being nouns.

Add a comment:

Ignore this:
not displayed and no spam.
Leave this empty:
not searched.
Name and either email or www are required.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.