Re-throwing exceptions in Python

Tuesday 13 November 2007This is nearly 17 years old. Be careful.

When dealing seriously with error handling, an important technique is to be able to manipulate exceptions in ways other than simply throwing and catching them. One of these is to re-throw exceptions.

The simplest way to do this is if you need to perform a little work after the catch, but then immediately re-throw. This can be done with a simple raise statement:

try:
    do_something_dangerous()
except:
    do_something_to_apologize()
    raise

Here the raise statement means, “throw the exception last caught”. This is a simple case, and I probably didn’t need to remind you of it. But a more sophisticated technique is to catch an exception in one place, and raise it again in another.

For example, you may have a worker thread pre-fetching data from slow storage, and then on the main thread, the consumer of the data either gets the data or sees the exception that prevented him from getting the data.

Here’s the simple implementation:

1class DelayedResult:
2    def __init__(self):
3        self.e = None
4        self.result = None
5        
6    def do_work(self):
7        try:
8            self.result = self.do_something_dangerous()
9        except Exception, e:
10            self.e = e
11
12    def do_something_dangerous(self):
13        raise Exception("Boo!")
14
15    def get_result(self):
16        if self.e:
17            raise self.e
18        return self.result
19    
20dr = DelayedResult()
21dr.do_work()
22dr.get_result()

We store an exception in the object, and when retrieving the result, if there’s an exception, we raise it. It works:

$ python delayed.py
Traceback (most recent call last):
  File "C:\lab\delayed.py", line 22, in ?
    dr.get_result()
  File "C:\lab\delayed.py", line 17, in get_result
    raise self.e
Exception: Boo!

The only problem is, the traceback for the exception shows the problem starting in get_result. When debugging problems, it’s enormously helpful to know their real origin.

To solve that problem, we’ll store more than the exception, we’ll also store the traceback at the time of the original problem, and in get_results, we’ll use the full three-argument form of the raise statement to use the original traceback:

1class DelayedResult:
2    def __init__(self):
3        self.exc_info = None
4        self.result = None
5        
6    def do_work(self):
7        try:
8            self.result = self.do_something_dangerous()
9        except Exception, e:
10            import sys
11            self.exc_info = sys.exc_info()
12    
13    def do_something_dangerous(self):
14        raise Exception("Boo!")
15
16    def get_result(self):
17        if self.exc_info:
18            raise self.exc_info[1], None, self.exc_info[2]
19        return self.result
20    
21dr = DelayedResult()
22dr.do_work()
23dr.get_result()

Now when we run it, the traceback points to do_something_dangerous, called from do_work, as the real culprit:

$ python delayed.py
Traceback (most recent call last):
  File "C:\lab\delayed.py", line 23, in ?
    dr.get_result()
  File "C:\lab\delayed.py", line 8, in do_work
    self.result = self.do_something_dangerous()
  File "C:\lab\delayed.py", line 14, in do_something_dangerous
    raise Exception("Boo!")
Exception: Boo!

The three-argument raise statement is a little odd, owing to its heritage from the old days of Python when exceptions could be things other than instances of subclasses of Exception. This accounts for the odd tuple-dance we do on the saved exc_info.

It’s easy to write code that does the right thing when everything is going well. It’s much harder to write code that does a good job when things go wrong. Properly manipulating exceptions helps.

Comments

[gravatar]
The amount of times I've seen this in C# code is baffling:

try
{
DoSomethingDangerous();
}
catch (Exception ex)
{
throw ex;
}

So if DoSomethingDangerous() throws an exception (worse, if one of its sub-methods!) that isn't caught, then I have almost no idea where the exception came from.
[gravatar]
There's two explanations:

1) There used to be something else in the catch clause, like a print statement, or they wanted a place to put a breakpoint.

2) They just aren't used to the idea of exceptions flowing through their code, and want to somehow control it. Of course, they aren't adding anything here but a stop-off to throw away valuable information.

Either way, it's bad.
[gravatar]
Neil,

You're right, the proper C# way should be:

try
{
ThrowSomeExceptionHere();
}
catch(Exception ex)
{
DoSomeLoggingAndNotification();
throw;
}

That will properly re-throw the exception and keep the stack trace intact. And of course, if you're not logging or doing something with the exception, why is there a catch block in the first place?
[gravatar]
i often catch an exception because in order to add info about the context to the exception and reraise it, and i found this to be helpful: http://blog.ianbicking.org/2007/09/12/re-raising-exceptions/
[gravatar]
Here is the Java code for re-raising exceptions without losing any information:

try{
doSomethingDangerous();
}
catch(OneExceptionType e){
throw new AnotherExceptionType(e); // just pass the original exception to the constructor

Yup, that's it. As a bonus, you get both the original stack trace and the stack trace from re-raising. Compared to this, the 'proper' Python way is an ugly hack.
[gravatar]
I think in the future the tb object (that holds the stack trace) will be attached to the exception directly, so re-raising exceptions will become easier. Maybe in Python 2.6?
[gravatar]
Great article, but i can see new Python programmers falling to the trap of catching an exception but instead of re-raising it they raise another, which is bad for debugging.
[gravatar]
I stumbled upon a great way to rethrow exception with a little more info (python of course):

try:
do_something_dangerous()
except Exception, e:
e.args = "Some smart and helpful message. The original message:\n%s" % e.args
raise

It adds some info without changing the traceback, so it's both a bit more helpful and it doesn't change the location from which it was thrown to the except location. Oh and it keeps the type of the exception, and doesn't convert it to Exception.
[gravatar]
Rafal Dowgird:

I duno, I seem to remember finding problems with this in Java. I'm not sure why... I at least remember finding code like "throw new SomeException(e.getMessage())", which absolutely drove me nuts.

hmm....

java programmers should still at least think about this kind of issue. I'm writing a little test program and can't seem to get it wrong, but I know I've seen it badly messed up, and had to fix a lot of code to be able to get real stack traces.
[gravatar]
Old principle: if you don't know how to handle an exception, don't catch it
[gravatar]
Helped me in 2015, thanks :)
[gravatar]
This seems to be fixed in Python 3 (I checked using 3.5.2). Great post though.
[gravatar]
Edward Natanzon 6:13 AM on 31 Oct 2018
Thank you!
[gravatar]
In Python 3 (at least from 3.5 onward) the original exception context and traceback are preserved, so this works as expected:
class DelayedResult:
    def __init__(self):
        self.exc = None
        self.result = None

    def do_work(self):
        try:
            self.result = self.do_something_dangerous()
        except Exception as e:
            self.exc = e

    def do_something_dangerous(self):
        raise Exception("Boo!")

    def get_result(self):
        if self.exc:
            raise self.exc
        return self.result


dr = DelayedResult()
dr.do_work()
dr.get_result()
The Java equivalent of throwing a new exception while preserving the old one is:
raise new_exc from original_exc
In c#, the general best way for exception re-throwing is in fact the following:
try {
    ExceptionThrowingMethodCall();
}
catch (Exception ex)
{
    ExceptionDispatchInfo.Capture(ex).Throw();
    throw;
}
A plain throw doesn't actually provide info on the exact line inside ExceptionThrowingMethodCall() that threw the exception. I think this has been fixed in later versions of .NET Core, but it hasn't been back-ported to .NET Framework yet. Still doesn't work perfectly though. Its main use is for preserving the exception context when rethrowing in a multi-threaded environment.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.