assertRaisesMsg

Tuesday 5 September 2006This is 18 years old. Be careful.

I use the standard unittest module for testing, despite the community’s general protestations about it not being pythonic. It does what I need, except for one thing. It has an assertRaises method that tests that a callable raises an exception of the expected class, but the message that comes with the exception is not checked. So I usually mix this method into my test case base class:

def assertRaisesMsg(self, excClass, msg, callableObj, *args, **kwargs):
    """ Just like unittest.TestCase.assertRaises,
        but checks that the message is right too.
    """
    try:
        callableObj(*args, **kwargs)
    except excClass, exc:
        excMsg = str(exc)
        if not msg:
            # No message provided: any message is fine.
            return
        elif excMsg == msg:
            # Message provided, and we got the right message: it passes.
            return
        else:
            # Message provided, and it didn't match: fail!
            raise self.failureException(
                "Right exception, wrong message: got '%s' expected '%s'" % 
                (excMsg, msg)
                )
    else:
        if hasattr(excClass, '__name__'):
            excName = excClass.__name__
        else:
            excName = str(excClass)
        raise self.failureException(
            "Expected to raise %s, didn't get an exception at all" % 
            excName
            )

I can call it (for example) like this:

self.assertRaisesMsg(MyException, "Exception message", my_function, (arg1, arg2))

This test line will call my_function(arg1, arg2) and succeed if it raises a MyException with the message “Exception message”. It will fail in all other cases.

Comments

[gravatar]
This looks like a good test case.

We use unittest at work. We've customised it a bit, but it's very easy to add new tests. I think unittest gets more flak than it deserves. :-)
[gravatar]
Michael Chermside 10:01 AM on 5 Sep 2006
While I have no problem with using this class, I think that in 95%+ of the cases I would NOT want to test the message. The exact text of error messages is rarely something that I want to enforce (via tests). What I did once when I was extending a testing framework (in another language, long ago) was to have the test pass if the string provided by the test appeared as a SUBSTRING someplace within the exception message. That way I could do a test that (for instance) verified that the filename appeared someplace within the FileNotFoundException without hard-coding (in the tests) the exact wording of the message.
[gravatar]
Michael: you are right, substring is a more useful implementation. If I want to check the whole message, substring works fine, and if I'm only interested in part, it also works fine.
[gravatar]
Ned,

I added an "assertRaisesEx" to the Python Cookbook a while back that adds "(1) the ability to assert the raised exception's args; (2) the ability to test that the stringified exception matches a given regular expression; and (3) much better failure messages."

Cheers
[gravatar]
I will definately be pulling this! I have lots of cases for using it, and it illustrates a weak point of the python exceptions that no one seems to agree on: they are not descriptive enough!

I want an AttributeError to have obj and name attributes, so I know what name was not found on what object. I want to see exactly what caused the error without looking at a message, so I can handle it properly.
[gravatar]
No sooner am I on-board with substrings, when Mick ups the stakes (and the value) by going to regexes!
[gravatar]
What about internationalization?
[gravatar]
This is ease to do with Py.test
http://codespeak.net/py/current/doc/test.html#how-to-write-assertions-about-exceptions

Something like:
assert py.test.raises(Exception, func, *args, **kwargs).value.args == msg
[gravatar]
Michael Chermside 8:15 PM on 5 Sep 2006
Good point Mick... regexes are indeed better than substrings. As to Glyph's point, I would argue that you should NOT be using internationalization in your exception messages. Internationalization is reserved for user-displayed strings, NOT for program internals. And exception message contents shouldn't be user-visible content.
[gravatar]
Michael: I believe there are arguments for both sides in that debate, both internationalizing and englishifying exception messages. However, it's rather academic, as Python has an established convention, which would be difficult to ignore in real-world tests:

>>> import locale
>>> import os
>>> def reallysetlocale(loc):
os.environ['LANG'] = loc
os.environ['LANGUAGE'] = loc
locale.setlocale(locale.LC_ALL, loc)

>>> file("test", "r")
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or directory: 'test'
>>> reallysetlocale('fr_FR.UTF-8')
>>> file("test", "r")
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] Aucun fichier ou r������©pertoire de ce type: 'test'

By the way, I don't have an answer to this problem. Personally I've been ignoring internationalization for some time, and I suspect that one day it's going to bite me really hard.

Unfortunately most software ignores more than just that - internationalization and encoding rules. Just look at what this blog software did to my error message! :)
[gravatar]
(Sorry, Glyph. This blog code isn't much good at I18N stuff)

Thanks, everyone! This has been a richer discussion than I anticipated, and has moved on to the next blog post...
[gravatar]
I just wanted to point out that if exceptions gave more information in a programmably accessable way, as I mentioned previously, the I18N problem goes away pretty quickly.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.