Coverage.py 4.2 is done.

As I mentioned in the beta 1 announcement, this contains work from the sprint at PyCon 2016 in Portland.

The biggest change since 4.1 is the only incompatible change. The "coverage combine" command now will ignore an existing .coverage data file, rather than appending to it as it used to do. This new behavior makes more sense to people, and matches how "coverage run" works. If you've ever seen (or written!) a tox.ini file with an explicit coverage-clean step, you won't have to any more. There's also a new "--append" option on "coverage combine", so you can get the old behavior if you want it.

The multiprocessing support continues to get the polish it deserves:

  • Now the concurrency option can be multi-valued, so you can measure programs that use multiprocessing and another library like gevent.
  • Options on the command line weren't being passed to multiprocessing subprocesses. Now they still aren't, but instead of failing silently, you'll get an error explaining the situation.
  • If you're using a custom-named configuration file, multiprocessing processes now will use that same file, so that all the processes will be measured the same.
  • Enabling multiprocessing support now also enables parallel measurement, since there will be subprocesses. This reduces the possibility for error when configuring coverage.py.

Finally, the text report can be sorted by columns as you wish, making it more convenient.

The complete change history is in the source.

tagged: » 2 reactions

Coverage.py 4.2 beta 1 is available.

This contains a few things we worked on during a day of sprinting at PyCon 2016 in Portland. Thanks to my fellow sprinters: Dan Riti, Dan Wandschneider, Josh Williams, Matthew Boehm, Nathan Land, and Scott Belden. Each time I've sprinted on coverage.py, I've been surprised at the number of people willing to dive into the deep end to make something happen. It's really encouraging to see people step up like that.

What's changed? The biggest change is the only incompatible change. The "coverage combine" command now will ignore an existing .coverage data file, rather than appending to it as it used to do. This new behavior makes more sense to people, and matches how "coverage run" works. If you've ever seen (or written!) a tox.ini file with an explicit coverage-clean step, you won't have to any more. There's also a new "--append" option on "coverage combine", so you can get the old behavior if you want it.

A new option lets you control how the text report is sorted.

The concurrency option can now be multi-valued, if you are using multiprocessing and some other concurrency library, like gevent.

The complete change history is in the source.

This isn't going to be a long beta, so try it now!

tagged: » react

54 can be written as the sum of three squares in three different ways:

7² + 2² + 1² = 6² + 3² + 3² = 5² + 5² + 2² = 54

It is the smallest number with this property.

Also, a Rubik's cube has 54 colored squares.

tagged: , » 1 reaction

I just got back from PyCon 2016, and it was a giant summer camp love-fest as usual. But I've been thinking about a subtle and unfortunate dynamic that I saw a few times there.

In three different cases, I was with a group of people, and one person in particular had a disproportionate amount of air-time. They were different guys each time, but they just had a way of being the one doing more talking than listening, and more talking than others. In some cases, they were physically loud, but I don't always mean literally the loudest.

These weren't bad people. Sometimes, they were explicitly discussing the need to include others, or improve diversity, or other good impulses. They weren't trying to dominate the space, and they might even be surprised to hear that they were.

But I found myself cringing watching their interactions with others. Even when they thought they were being encouraging, I felt like they were subtly pushing others aside to do it. Keep in mind, this was at PyCon, one of the most explicitly inclusive places I frequent.

I'm a successful white guy, so I know it can be very easy to slip into the alpha male stance. Sometimes people expect it of me. It can be hard to tamp down the impulse to hold forth, letting others have the spotlight. But it's important, and a good exercise for yourself. It's fine to be able to be at the front of the room, but you should be able to turn it off when needed, which is more often than you would think.

Sometimes, this was in a men-only setting. It's great to be aware of the gender gap, but there are other kinds of gaps to consider also: non-native speakers, introverts, beginners, outsiders of various sorts. There are lots of reasons people might be quiet, and need a little room.

Ask questions instead of making statements. Stay quiet, and see what happens. Listen rather than speak. Even when it seems no one is going to say anything, wait longer than you are comfortable. See what happens. Leave space.

Next time you are in a group of people, look around and try to figure out who is the loudest guy in the room. If you aren't sure, then maybe it's you.

I gave a talk today at PyCon 2016: Machete-Mode Debugging. I went over time, but people really liked it. Amazingly, the video was available within six hours!

Gratifyingly, a talk later in the day referenced the phrase "machete-mode debugging," so maybe it will catch on.

tagged: » react

Coverage.py 4.1 is out!

I'm not sure what else to say about it that I haven't said a few times in the last six months: branch coverage is completely rewritten, so it should be more accurate and more understandable.

The beta wasn't getting much response, so I shielded my eyes and released the final version a few days ago. No explosions, so it's seems to be OK!

tagged: » react

Often, when I am headed to bed, I stop in at my son Ben's room, to see what he's up to. He'll be working on some piece of art, and we'll chat for a moment about it.

The other night, he was working on a self-portrait. It was a realistic depiction, but in a style reminiscent of a Renaissance prince. We talked about the style, what parts looked just like him, and what parts might need tweaking.

I went to bed, and then in the morning this was on his Facebook:

Ben' portrait

I didn't get a story about why the realistic face was gone, and why the self-homunculus is in its place instead. This picture looks much less like him, but says much more about him. He changed it from a picture of him to a picture about him. Ben has always impressed me with his art, not just as a technical skill, but as an expression of deeper ideas.

As always, I am proud of him, and thrilled to see what he creates.

Python has a compact syntax for constructing a list with a loop and a condition, called a list comprehension:

my_list = [ f(x) for x in sequence if cond(x) ]

You can also build dictionaries with dictionary comprehensions, and sets with set comprehensions:

my_dict = { k(x): v(x) for x in sequence if cond(x) }
my_set = { f(x) for x in sequence if cond(x) }

(The syntax allows more complexity than these examples, let's not get distracted!)

Finally, you can make a generator with similar syntax:

my_generator = ( f(x) for x in sequence if cond(x) )

Unfortunately, this is called a generator expression, not a generator comprehension. Why not? If the first three are all comprehensions, why isn't this a comprehension?

PEP 289, Generator Expressions has detailed notes at the end which point out that Raymond Hettinger originally proposed "generator comprehensions," that they were then resurrected by Peter Norvig as "accumulation displays," and that Tim Peters suggested the name "generator expressions." It does not explain why the names changed along the way.

I made a query on Twitter:

OK, #python question I don’t know the answer to: why are they called “generator expressions” and not “generator comprehensions”?

Guido's reply gets at the heart of the matter:

Originally comprehension was part of the "literal display" notion. GenExprs are not displays.

Matt Boehm found the email where Tim Peters proposed "generator expression" that also has some details.

After reading that, I understand more. First, what's with the word "comprehension"? As Tim pointed out, the word comes from set theory's Axiom of Comprehension, which talks about sets formed by applying a predicate (condition) to elements of another set. This is very similar to lists formed by applying a condition to elements of another sequence.

As Guido's tweet points out, and the subject line of the email thread makes clear ("accumulator display syntax"), the designers at the time were thinking much more about displays than they were about conditions. The word "display" here means that the syntax for the code looks like the data structure it will create. A list display (list comprehension) looks like a list. Same for set and dictionary displays. But there is no generator literal syntax, so there's nothing for a generator display to look like, so there are no generator displays.

In that original email thread designing the feature, the word "comprehension" became synonymous with "display", and since generators couldn't have displays, they also couldn't have comprehensions.

But as Tim points out in his email, the interesting part of a comprehension is the condition. The heart of the Axiom of Comprehension is the predicate. Perhaps because the condition is optional in a Python comprehension, the focus shifted to the display aspect.

I think we should call them "generator comprehensions" again. We don't use the term "display" for these things. There's no reason to link "comprehension" to "display," and literal syntax.

The four different expressions (list comprehension, dict comprehension, set comprehension, and generator expressions) have an awful lot in common with each other. It would be a great shorthand to be able to discuss their similarities by talking about "comprehensions" and having it cover all four. Their similarities are more than their differences, so let's use the same word for all four.

Proposal: call them "generator comprehensions."

Work on Coverage.py 4.1 is continuing: beta 3 is available.

If you haven't used any of the 4.1 betas, the big change is that branch coverage has been completely rewritten. The new code produces much more reliable results, and has allowed me to implement things like better support for lambdas. Eleven bugs with branch coverage have been fixed.

The HTML report has a cool new feature, contributed by Dmitry Shishov, a map in the scrollbar of where the highlighted lines are, so you can quickly drag to where you need to look. (By the way, there are also keyboard shortcuts to do that, have been for a long time!)

One small backward-incompatibility: if you've been using the API, and calling the Coverage.report function, the default for the show_missing parameter has changed.

Try Coverage.py 4.1b3 and let me know what you think.

Now, the truth about Coverage.py: I think it could be much better. There are lots of things about the internals that I don't like. I think the classes could be refactored better. Too many of the tests are integration tests rather than unit tests. Too many real-world scenarios aren't covered by tests. I'm not good at staying on top of the pull requests and issues. If you think you could help with any of this, get in touch.

tagged: » react

I have two great juggling videos to share: two jugglers, each great in his own way.

First: Alexander Koblikov. He is a great professional juggler, currently with the Big Apple Circus. He has a smooth evocative style with a small number of balls, starting with simple contact moves, but growing to flawless five-ball work. Then he can show off raw power with nine-ball multiplexes. A very impressive combination of both ends of the professional spectrum:

Kota Hayashi is very different. He's an amateur juggler, performing at the International Juggler's Association convention. He isn't wearing any particular costume, his act has no story. He's not a poker-faced artiste, and he only juggles three balls. He's a good juggler, but more importantly, he just obviously loves juggling. His enthusiasm is infectious. As you watch his act, it gets a bit ridiculous. You start to think, this is silly. But really, isn't juggling silly to begin with? Why do we throw objects around in fancy patterns? There's no point to it, other than our own amusement. Kota's act is a visible embodiment of the pure pleasure of mastering an absurd skill for its own sake.

Skip ahead to 1:05 where Kota starts:

And because I can't stop watching juggling videos, here are two bonus jugglers showing two more completely different styles:

  • Olivia Porter has charming Chaplinesque way of working with three balls that put her at their mercy in a baffling world.
  • Tony Pezzo is insanely talented, with his own style of twisty multiplexed numbers street juggling.

Glossary:

  • multiplex: throwing more than one ball at once from a hand
  • numbers: juggling more than three balls
  • contact: using a ball without leaving contact with your body, usually by rolling it
tagged: » react

A little over a week ago, a nephew-of-sorts of mine died in a fall. He was almost 19, a freshman at Tufts. It was tragic, and senseless, and horrifying. The funeral was Sunday, and he's been on my mind a lot.

To be honest, I didn't know Alex that much. We saw each other at most once a year, and usually less frequently. I learned more about Alex at his funeral than I had over the years of rote greetings at family gatherings. He was a smart, generous, energetic guy (boy? man?)

When I think about Alex's death, of course I think about him, and his too-short life, and his final hours. I think about his parents, and what they must be going through, and I wonder if I could handle such a loss.

I think about my own children and I think about parenthood: the enormous commitment, energy, love, and work that goes into shaping and guiding these new people. The pain and fear of sending them off into the world, away from your protective watch. It's difficult in the best of times.

Alex's death was painful not only because we lost Alex, but because it was a brutal reminder that we can lose anyone, at any time, with no notice. It's easy to imagine a nearby parallel universe where it was one of my sons instead of him.

At the funeral, Alex's dad recounted an uncle's similar loss. The uncle's wisdom was that we will not find a reason for Alex's death, but that we will find meaning in life.

I come back again to Kurt Vonnegut's son Mark, answering, and neatly side-stepping, the question of meaning: "We're here to get each other through this thing, whatever it is."

Take care of each other.

If you use Slack, or read docs on Read The Docs, you've seen Lato. It's a free high-quality font. I like it a lot, but it has a feature that bugs me a lot: the f-i ligature:

A sample of Lato

If you've never looked into this before, a ligature is a combination of letters that are designed as a new distinct glyph. In this case, there's an "fi" shape that is used when you have an "f" and an "i" next to each other.

Ligatures have a long tradition in Latin fonts, for a reason: some pairings of letters can have a jarring look. The bulb of the f and the dot of the i can clash, and it looks better to design a combined shape that shares the space better.

But Lato doesn't suffer from this problem. Ligatures are a solution to a problem, and here they are being used when there is no problem to solve. The Lato fi ligature is more jarring than the f and the i, because it looks like there's no dot for the i.

Here's a comparison of the fi ligature in some fonts. The first column is a plain f and i presented naturally, but forced to be individual, naively. Then the fi combination as the browser text renderer draws them, and then the Unicode fi ligature, U+FB01 (LATIN SMALL LIGATURE FI):

Ligatures in various fonts

The naive Lato f and i look fine together without any intervention. The ligature looks silly without the dot. The f is trying to reach over to join the dot, but it's too far to reach, so it doesn't get there, and the f has no bulb in the first place. It doesn't make any sense.

Constantia and Georgia demonstrate a good use of ligatures: the naive pairing shows how the bulb and the dot crowd into each other, and the ligatures shift things a little to resolve the clash.

(Somehow, Lato doesn't map its fi ligature to the U+FB01 code point, so we get the default font there instead.) If you want to experiment, here's the HTML file I used to make the image.

By the way, it was an interesting challenge to get the browsers to display the unligatured f-i pairs. In Firefox, I used a zero-width space (U+200B) between the characters. But Chrome substituted the ligature anyway, so I tried putting the f and the i in adjacent spans. This worked in Chrome, but Firefox used the ligature. So I combined both methods:

<span>f&#x200b;</span><span>i</span>

Older:

Even older...