« | » Main « | »

Coverage.py 4.1

Friday 27 May 2016

Coverage.py 4.1 is out!

I'm not sure what else to say about it that I haven't said a few times in the last six months: branch coverage is completely rewritten, so it should be more accurate and more understandable.

The beta wasn't getting much response, so I shielded my eyes and released the final version a few days ago. No explosions, so it's seems to be OK!

Ben portrait

Sunday 15 May 2016

Often, when I am headed to bed, I stop in at my son Ben's room, to see what he's up to. He'll be working on some piece of art, and we'll chat for a moment about it.

The other night, he was working on a self-portrait. It was a realistic depiction, but in a style reminiscent of a Renaissance prince. We talked about the style, what parts looked just like him, and what parts might need tweaking.

I went to bed, and then in the morning this was on his Facebook:

Ben' portrait

I didn't get a story about why the realistic face was gone, and why the self-homunculus is in its place instead. This picture looks much less like him, but says much more about him. He changed it from a picture of him to a picture about him. Ben has always impressed me with his art, not just as a technical skill, but as an expression of deeper ideas.

As always, I am proud of him, and thrilled to see what he creates.

Generator comprehensions

Wednesday 11 May 2016

Python has a compact syntax for constructing a list with a loop and a condition, called a list comprehension:

my_list = [ f(x) for x in sequence if cond(x) ]

You can also build dictionaries with dictionary comprehensions, and sets with set comprehensions:

my_dict = { k(x): v(x) for x in sequence if cond(x) }
my_set = { f(x) for x in sequence if cond(x) }

(The syntax allows more complexity than these examples, let's not get distracted!)

Finally, you can make a generator with similar syntax:

my_generator = ( f(x) for x in sequence if cond(x) )

Unfortunately, this is called a generator expression, not a generator comprehension. Why not? If the first three are all comprehensions, why isn't this a comprehension?

PEP 289, Generator Expressions has detailed notes at the end which point out that Raymond Hettinger originally proposed "generator comprehensions," that they were then resurrected by Peter Norvig as "accumulation displays," and that Tim Peters suggested the name "generator expressions." It does not explain why the names changed along the way.

I made a query on Twitter:

OK, #python question I don’t know the answer to: why are they called “generator expressions” and not “generator comprehensions”?

Guido's reply gets at the heart of the matter:

Originally comprehension was part of the "literal display" notion. GenExprs are not displays.

Matt Boehm found the email where Tim Peters proposed "generator expression" that also has some details.

After reading that, I understand more. First, what's with the word "comprehension"? As Tim pointed out, the word comes from set theory's Axiom of Comprehension, which talks about sets formed by applying a predicate (condition) to elements of another set. This is very similar to lists formed by applying a condition to elements of another sequence.

As Guido's tweet points out, and the subject line of the email thread makes clear ("accumulator display syntax"), the designers at the time were thinking much more about displays than they were about conditions. The word "display" here means that the syntax for the code looks like the data structure it will create. A list display (list comprehension) looks like a list. Same for set and dictionary displays. But there is no generator literal syntax, so there's nothing for a generator display to look like, so there are no generator displays.

In that original email thread designing the feature, the word "comprehension" became synonymous with "display", and since generators couldn't have displays, they also couldn't have comprehensions.

But as Tim points out in his email, the interesting part of a comprehension is the condition. The heart of the Axiom of Comprehension is the predicate. Perhaps because the condition is optional in a Python comprehension, the focus shifted to the display aspect.

I think we should call them "generator comprehensions" again. We don't use the term "display" for these things. There's no reason to link "comprehension" to "display," and literal syntax.

The four different expressions (list comprehension, dict comprehension, set comprehension, and generator expressions) have an awful lot in common with each other. It would be a great shorthand to be able to discuss their similarities by talking about "comprehensions" and having it cover all four. Their similarities are more than their differences, so let's use the same word for all four.

Proposal: call them "generator comprehensions."

Coverage.py 4.1b3 and the truth about Coverage.py

Tuesday 10 May 2016

Work on Coverage.py 4.1 is continuing: beta 3 is available.

If you haven't used any of the 4.1 betas, the big change is that branch coverage has been completely rewritten. The new code produces much more reliable results, and has allowed me to implement things like better support for lambdas. Eleven bugs with branch coverage have been fixed.

The HTML report has a cool new feature, contributed by Dmitry Shishov, a map in the scrollbar of where the highlighted lines are, so you can quickly drag to where you need to look. (By the way, there are also keyboard shortcuts to do that, have been for a long time!)

One small backward-incompatibility: if you've been using the API, and calling the Coverage.report function, the default for the show_missing parameter has changed.

Try Coverage.py 4.1b3 and let me know what you think.

Now, the truth about Coverage.py: I think it could be much better. There are lots of things about the internals that I don't like. I think the classes could be refactored better. Too many of the tests are integration tests rather than unit tests. Too many real-world scenarios aren't covered by tests. I'm not good at staying on top of the pull requests and issues. If you think you could help with any of this, get in touch.

« | » Main « | »