Interactive debugging in Python

Friday 2 September 2005

One of the standard Python tools I haven’t made enough use of is the built-in debugger, pdb. Jeremy Jones gives us a tutorial on it: Interactive Debugging in Python. I hope his assessment of the place of the debugger in a programmer’s toolbox is inaccurate, though:

Programmers typically place debuggers in the “uh oh” corner of their toolboxes, somewhere between the network packet sniffer and the disassembler. The only time we reach for a debugger is when something goes wrong or breaks and our standard debugging techniques such as print statements (or better yet, log messages) do not reveal the root of the problem.


That's funny. I only resort to print statements and log messages when I'm working on a platform where I can't use a debugger. (assuming he means adding extra printf/log statements to help debug a problem, not just looking at pre-existing ones)

It's usually 10x faster to simply step through the part of the code where the bug is likely to be located then to add a bunch of log statements, rebuild, run, look at the log statements, realize that you missed a spot, add a new one, rebuild, find the bug, remove the log statements....

The thing about the debugger is that it knows *everything*. To print out all the info a debugger has for every line of your code would take anywhere from a half dozen to 30 or more print statements (or one really long one).

Now maybe he's talking about Python programmers in general, in which case he may be right, I don't know. I haven't done enough in Python to really need a debugger.
It really depends on the code and the situation. For most code, I will certainly use a source level debugger wherever possible

I pretty much stick to log statement based debugging when dealing with network based systems. Not because I don't want to use a debugger, but because there may be 100s or 1000s of events before the broken one and because timing (and timeouts) often play such a critical role.

Using an intelligently designed, well categorized, logging system that can be configured at runtime also lays a foundation for some serious in-production debugging, monitoring and optimization capabilities.
I am working on a medium size C# project and due to the dissonance between the beta compiler and the iDE, the debugger basically does not work. So I use verbose logging. After a successful run of the thing (it is a wizardy configuration tool), I am looking at a 2MB text file. Sigh. Do I really need to log that a constructor was entered and exited when nothing was done?
Ok, I'll bite ... I generally don't use a debugger when writting C, and when I do the main reason is to get a backtrace (and python gives you that by default).

There are a couple of arguments for why debuggers aren't that great, the two main ones being the Heisenberg problem and the fact the debugger only tells you the last part of the problem ... not the root cause.

Saying that for those cases where I can't see what the hell is going wrong by looking at the source and the unit tests, a debugger is very useful.
What I really want for python is a debugger with a reverse button - it would be much more helpful than the regular debugger.

I don't want it enough to go out and *make* it, mind you, but it certainly would be neat and a nice feature to attract people.
Andrew - I'm curious what compiler you're using for C#. I've been using the C# Visual Studio Express 2005 beta 2, and its debugger works just fine. I honestly can't imagine trying to write a real program without a debugger.

re: omniscient debugging

Yes; I think it is practical, but only to maintain a limited history. Really, I just want to be able to set a breakpoint in the vicinity of where I know an error occurred, and if I can move a couple of steps backwards, that would be all I need. Instead of carefully assuring that my breakpoint is in the right place, and hitting "step over" one too many times, I could just plop it somewhere near where I wanted it.
I firmly believe that a debugger is a last resort tool, and the best way to debug it to let the software tell you what happened through error handling and reporting and logging. Software will eventually go into production, and a debugger will rarely be available to help the problem.

Another problem that debuggers can't solve but which error handling (btw, thanks for the excellent articles, Ned) and logging can solve is the reproduction of bugs. How often does a user say: the product crashed, and an error message came up but I can't remember what it said. A debugger is no help at all here, but logs are.

I can't remember where I saw this (a quick Google search didn't find anything), but there was an excellent article written about logging vs debugging, and I think it was written by either Kernigan and Ritchie or the Pragmatic Programming guys. Highly recommended.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.