Titus Brown doesn't like a title of mine. He claims "Flaws in coverage measurement" is sensational, but I don't see why. It's actually pretty bland, it almost sounds like the title of a scientific paper. It would only be controversial to someone who thought coverage testing was perfect, which Titus does not.

In fact, one of the central points of his original post that got me in hot water was:

High code coverage doesn't guarantee that your tests are complete.

He and I are in total agreement. I guess you could quibble that the flaws are not in coverage measurement, that to believe they are flaws is to over-estimate the extent of the power of coverage testing, to misunderstand what coverage testing is and how it should be applied. In that sense, it is a harmful title because it feeds into the notion that coverage testing is flawed, when it is people's over-reliance on it that is flawed.

So maybe I should have titled it "What coverage measurement can't tell you," or "Coverage measurement can only take you so far," or "Where coverage measurement runs out of gas." Ironically, I think each of these are both more accurate and more sensational that the original title!

I've certainly been guilty of writing purposefully sensational titles, like Joel Spolsky is a crotchety old man, or attempted eye-catchers like Dieting cavemen on twitter. Like all blog authors, I want people to read what I write, and a good title can help attract readers. As it happens, the title in question wasn't too good!

I sure hope I haven't harmed the cause of coverage testing, because I think it's a great thing and everyone should use it. Be sure to catch my PyCon 2009 talk, "Coverage testing, the good and the bad." Or should I call it, "Coverage testing is a very good thing, but it can't tell you everything you need to know to be sure your software is good, so please don't stop at 100%, there are other things you have to do"?

And yes, the title of this post is a joke...

tagged: , , » 4 reactions

Comments

[gravatar]
Nate 7:18 AM on 23 Feb 2009

Hilarious and awesome, and right as usual.

[gravatar]
Marius Gedminas 11:42 AM on 23 Feb 2009

How about "100% test coverage: necessary, but not sufficient"?

Personally, I find the "Flaws in coverage measurement" title to be non-sensational and accurate. I don't see an implied "therefore coverage is pointless" that Titus seems to have seen.

[gravatar]
Doug Napoleone 1:25 PM on 23 Feb 2009

You should have ended your with the proper quote:
"I said good day sir!"

[gravatar]
Anonymous 10:14 PM on 23 Feb 2009

Keep up the great posts.

Add a comment:

name
email
Ignore this:
not displayed and no spam.
Leave this empty:
www
not searched.
 
Name and either email or www are required.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.