v3.2b1: branch coverage!

Tuesday 10 November 2009 v3.2 beta 1 is available, and it’s got a big new feature: branch coverage. It’s been a long time coming, but I’m pretty pleased with the results. I’m very interested to hear whether this is useful, and what could be improved.

Branch coverage now supports branch coverage measurement. Where a line in your program could jump to more than one next line, tracks which of those destinations are actually visited, and flags lines that haven’t visited all of their possible destinations.

For example:

def my_partial_fn(x):       # line 1
    if x:                   #      2
        y = 10              #      3
    return y                #      4


In this code, the if on line 2 could branch to either line 3 or line 4. Statement coverage would show all lines of the function as executed. But the if is always true, so line 2 never jumps to line 4. Even though line 4 is executed, knows that it was never because of a branch from line 2.

Branch coverage would flag this code as not fully covered because of the missing jump from line 2 to line 4.

How to measure branch coverage

To measure branch coverage, run with the --branch flag:

coverage run --branch

When you report on the results with “coverage report” or “coverage html”, the percentage of branch possibilities taken will be included in the percentage covered total for each file. The coverage percentage for a file is the actual executions divided by the execution opportunities. Each line in the file is an execution opportunity, as is each branch destination.

Currently, only HTML reports give information about which lines had missing branches. Lines that were missing some branches are shown in yellow, with an annotation at the far right showing branch destination line numbers that were not exercised.

How it works

When measuring branches, collects pairs of line numbers, a source and destination for each transition from one line to another. Static analysis of the compiled bytecode provides a list of possible transitions. Comparing the measured to the possible indicates missing branches.

The idea of tracking how lines follow each other was from C. Titus Brown. Thanks, Titus!


Some Python constructs are difficult to measure properly. For example, an infinite loop will be marked as partially executed:

while True:                         # line 1
    if some_condition():            #      2
    body_of_loop()                  #      4

keep_working()                      #      6

Because the loop never terminates naturally (jumping from line 1 to 6), thinks the branch is partially executed.

Currently, if you exclude code from coverage testing, a branch into that code will still be considered executable, and may result in the branch being flagged.

A few other unexpected cases are described in branches.html, which also shows how partial branches are displayed in the HTML report.

The only way currently to initiate branch coverage is with the command-line interface. In particular, the nose coverage plugin has no way to use it.

Other work

One interesting side effect of tracking line transitions: we know where some exceptions happened because a transition happens that wasn’t predicted by the static analysis. Currently, I’m not doing anything with this information. Any ideas?


Wow Thank you Thank you Thank you!
Thanks Ned (and Titus) - This is great news
I do not think that except clauses create a three-way-branch: The throw clause creates a branch into the next available except handler (or, failing that, out of the program).
Very nice indeed!
I'm looking forward to a possible API-implementation in the future. :)
Thanks for one of the best tools I have in my toolkit. Coverage is fantastic.
Would it be possible to treat 'while True:' and 'if __name__ == "__main__":' as special cases so that they are not reported as partially executed? Otherwise a great tool: I already found several branches which were not covered by tests in my source code. should make it into the standard library.
@Uwe: I have to figure out what to do about "while True:", that is a real problem. The '__name__ == "__main__"' issue can already be dealt with by excluding that line from coverage reporting, either with an explicit pragma comment, or by matching it with an exclude regex.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.