Monday 12 August 2013 — This is more than 11 years old. Be careful.
Coverage.py is kind of stuck. I haven’t done much to it in the last six months or so. Well, I’ve fixed six bugs, but I haven’t released them because I want to fix more.
In particular, I want to fix the branch coverage bugs. When I wrote the branch coverage feature, I based it on bytecode analysis of the code. Coverage.py finds the possible branches by looking at the bytecode to decide which of them can jump to which . Some of those bugs are because I don’t fully understand CPython bytecode.
To help me learn all the ins and outs of the CPython bytecode, I started byterun, a pure-Python implementation of a Python bytecode runner, or VM. Actually, I didn’t start it, I found a thing written by Paul Swartz, and renovated and extended it. It works pretty well, but isn’t complete. In particular, some of the more complex bytecodes, dealing with the block stack, don’t work properly on Python 3. I would love some help finishing byterun.
Then I need to figure out if analyzing bytecode is a reasonable way to find branches in a program. I think when I started down this path, I thought that everything would be compiled to simple bytecodes like “if cond jump to x,” but that’s not what happens. Bytecodes like WITH_CLEANUP are complex beasts with sub-optimal documentation, and they behave slightly differently in Python 2 and Python 3.
Of course I would rather not rip out the bytecode approach and implement an AST approach, but I would like to know whether that’s the way to get branch coverage that truly works well.
I’m looking for collaborators to help get through this issue. I need to get past it so I can get back to implementing features people want, like gevent support, or faster reporting, or getting rid of pickles!
Comments
Also, I would like to point out that "branch coverage" is a fuzzy thing. You point out the context manager example: if a "with" block always ends up successfully, should it be counted as missing the exception case? How about try..finally pairs?
I think "branch coverage" may therefore be misguided and the only reasonably measurable thing is actualy code coverage (i.e. coverage of bytecode blocks).
I would still like to improve the feature where I can!
Add a comment: