A proof-of-concept tool for finding unneeded coverage.py exclusion pragmas
To answer a long-standing coverage.py feature request, I threw together an experiment: a tool to identify lines that have been excluded from coverage, but which were actually executed.
The program is a standalone file in the coverage.py repo. It is unsupported. I’d like people to try it to see what they think of the idea. Later we can decide what to do with it.
To try it: copy warn_executed.py from GitHub. Create a .toml file that looks something like this:
# Regexes that identify excluded lines:
warn-executed = [
"pragma: no cover",
"raise AssertionError",
"pragma: cant happen",
"pragma: never called",
]
# Regexes that identify partial branch lines:
warn-not-partial = [
"pragma: no branch",
]
These are exclusion regexes that you’ve used in your coverage runs. The program will print out any line identified by a pattern and that ran during your tests. It might be that you don’t need to exclude the line, because it ran.
In this file, none of your coverage settings or the default regexes are assumed: you need to explicitly specify all the patterns you want flagged.
Run the program with Python 3.11 or higher, giving the name of the coverage data file and the name of your new TOML configuration file. It will print the lines that might not need excluding:
$ python3.12 warn_executed.py .coverage warn.toml
The reason for a new list of patterns instead of just reading the existing coverage settings is that some exclusions are “don’t care” rather than “this will never happen.” For example, I exclude “def __repr__” because some __repr__’s are just to make my debugging easier. I don’t care if the test suite runs them or not. It might run them, so I don’t want it to be a warning that they actually ran.
This tool is not perfect. For example, I exclude “if TYPE_CHECKING:” because I want that entire clause excluded. But the if-line itself is actually run. If I include that pattern in the warn-executed list, it will flag all of those lines. Maybe I’m forgetting a way to do this: it would be good to have a way to exclude the body of the if clause while understanding that the if-line itself is executed.
Give warn_executed.py a try and comment on the issue about what you think of it.
Comments
Add a comment: