Coverage.py 5.0a4: the sys.path to hell

Sunday 25 November 2018

Another alpha of Coverage.py 5.0 is available: 5.0a4. This fixes a few problems with the new SQLite-based storage. Please give it a try, especially to experiment with dynamic contexts.

The challenge with this release was something that started as a seemingly simple fix. Coverage.py tries to emulate how Python runs programs, including how the first element of sys.path is set. A few people run coverage with sys.path fully configured, and coverage’s setting of sys.path[0] was breaking their stuff.

The proposal was simple: delete the one line of code that set sys.path[0]. I tried that, and it seemed to work. Fixed!

Not so fast: the Windows builds failed. This started a multi-week adventure of debugging and refactoring. The Windows builds were failing not because of Windows itself, but because on Windows, I don’t use pytest-xdist, which parallelizes tests into worker processes. With xdist, the tests were all passing. Without xdist, a few sys.path-related tests were failing.

It turns out that xdist manipulates sys.path itself, which was masking the fact that I had removed an important step from coverage.py. First thing to do was to adjust my test code so that even with xdist, my tests didn’t get xdist’s path changes.

Then I had to re-think how to adjust sys.path. That required refactoring how I ran the user’s Python code, so that I could apply the path changes a little earlier than I used to. That made me look at how I was testing that layer of code with mocks, and I changed it from explicit dependency injection to implicit mock patching.

A few more little fixes were needed here and there along the way. All told, the “one line” fix ended up being 14 files changed, 587 insertions, 427 deletions.

Comments

[gravatar]
Simon Charette 2:50 AM on 26 Nov 2018

Ahh that looks like an horrible yak shaving session! Thanks for the hardwork here Ned. The new dynamic context feature should really bring the usefulness of coverage data to the next level! Is there any public discussion (mailing list, Github issue) about the reporting part ideas so far?

[gravatar]
Ralph Heinkel 6:33 AM on 26 Nov 2018

Hi Ned, wow, sounds like a week of debugging Horror ;-) Thanks for putting all this effort into this project, I really appreciate it! Coverage.py is an absolute great project.

[gravatar]
Adam Johnson 11:28 AM on 26 Nov 2018

Your persistence is inspiring, thanks for putting all this effort in over time.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.