|Ned Batchelder : Blog | Code | Text | Site|
I have two great juggling videos to share: two jugglers, each great in his own way.
First: Alexander Koblikov. He is a great professional juggler, currently with the Big Apple Circus. He has a smooth evocative style with a small number of balls, starting with simple contact moves, but growing to flawless five-ball work. Then he can show off raw power with nine-ball multiplexes. A very impressive combination of both ends of the professional spectrum:
Kota Hayashi is very different. He's an amateur juggler, performing at the International Juggler's Association convention. He isn't wearing any particular costume, his act has no story. He's not a poker-faced artiste, and he only juggles three balls. He's a good juggler, but more importantly, he just obviously loves juggling. His enthusiasm is infectious. As you watch his act, it gets a bit ridiculous. You start to think, this is silly. But really, isn't juggling silly to begin with? Why do we throw objects around in fancy patterns? There's no point to it, other than our own amusement. Kota's act is a visible embodiment of the pure pleasure of mastering an absurd skill for its own sake.
Skip ahead to 1:05 where Kota starts:
And because I can't stop watching juggling videos, here are two bonus jugglers showing two more completely different styles:
A little over a week ago, a nephew-of-sorts of mine died in a fall. He was almost 19, a freshman at Tufts. It was tragic, and senseless, and horrifying. The funeral was Sunday, and he's been on my mind a lot.
To be honest, I didn't know Alex that much. We saw each other at most once a year, and usually less frequently. I learned more about Alex at his funeral than I had over the years of rote greetings at family gatherings. He was a smart, generous, energetic guy (boy? man?)
When I think about Alex's death, of course I think about him, and his too-short life, and his final hours. I think about his parents, and what they must be going through, and I wonder if I could handle such a loss.
I think about my own children and I think about parenthood: the enormous commitment, energy, love, and work that goes into shaping and guiding these new people. The pain and fear of sending them off into the world, away from your protective watch. It's difficult in the best of times.
Alex's death was painful not only because we lost Alex, but because it was a brutal reminder that we can lose anyone, at any time, with no notice. It's easy to imagine a nearby parallel universe where it was one of my sons instead of him.
At the funeral, Alex's dad recounted an uncle's similar loss. The uncle's wisdom was that we will not find a reason for Alex's death, but that we will find meaning in life.
I come back again to Kurt Vonnegut's son Mark, answering, and neatly side-stepping, the question of meaning: "We're here to get each other through this thing, whatever it is."
Take care of each other.
If you use Slack, or read docs on Read The Docs, you've seen Lato. It's a free high-quality font. I like it a lot, but it has a feature that bugs me a lot: the f-i ligature:
If you've never looked into this before, a ligature is a combination of letters that are designed as a new distinct glyph. In this case, there's an "fi" shape that is used when you have an "f" and an "i" next to each other.
Ligatures have a long tradition in Latin fonts, for a reason: some pairings of letters can have a jarring look. The bulb of the f and the dot of the i can clash, and it looks better to design a combined shape that shares the space better.
But Lato doesn't suffer from this problem. Ligatures are a solution to a problem, and here they are being used when there is no problem to solve. The Lato fi ligature is more jarring than the f and the i, because it looks like there's no dot for the i.
Here's a comparison of the fi ligature in some fonts. The first column is a plain f and i presented naturally, but forced to be individual, naively. Then the fi combination as the browser text renderer draws them, and then the Unicode fi ligature, U+FB01 (LATIN SMALL LIGATURE FI):
The naive Lato f and i look fine together without any intervention. The ligature looks silly without the dot. The f is trying to reach over to join the dot, but it's too far to reach, so it doesn't get there, and the f has no bulb in the first place. It doesn't make any sense.
Constantia and Georgia demonstrate a good use of ligatures: the naive pairing shows how the bulb and the dot crowd into each other, and the ligatures shift things a little to resolve the clash.
(Somehow, Lato doesn't map its fi ligature to the U+FB01 code point, so we get the default font there instead.) If you want to experiment, here's the HTML file I used to make the image.
By the way, it was an interesting challenge to get the browsers to display the unligatured f-i pairs. In Firefox, I used a zero-width space (U+200B) between the characters. But Chrome substituted the ligature anyway, so I tried putting the f and the i in adjacent spans. This worked in Chrome, but Firefox used the ligature. So I combined both methods:
I got an email from a mom last week:
I told her I couldn't meet one-on-one, but suggested they attend the upcoming Boston Python project night. I didn't know what would come of it. Project night is completely unstructured, an opportunity to hang out with other Python people. It's a complete jumble of all kinds of people. There's no guarantee you'll find what you need there, but there's a good chance you will.
Last night was the project night, and there they were! They sat down at one of our beginning learner tables, and others joined them. I didn't have a chance to sit and talk with them at length, but I could see they had the attention of helpers, including John, one of the regulars. Each time I looked over, John was in deep discussion with the kid.
While talking to someone else last night who was interested in game programming, I looked up the game that originally got the mom's attention: I posted Nat's World to this site 13 years ago today!
The mom and the kid said goodbye to me when they had to go. She seemed pleased, and he did too, in his quiet but eager way. I told him, "That makes me happy."
As the night was winding down, I caught up with John, who was talking to a few others. "That kid was amazing!" he said. "I know, that was so cool," said someone else.
This is what makes local user groups so great. I don't know what John was expecting to do with his evening. I don't know what the mom and the kid were expecting when they decided to come. But they made a connection, got some help, and made an impression on each other. People across the room who didn't even talk to the mom or the kid came away with an unexpected picture of what the Python community can be like: broader and more diverse, more welcoming than the stereotype of a tech user group filled with brogrammers.
And this is also what is cool about making things and putting them online. Nat's World was a fun project when I made it. I haven't run it in years, but my family still remembers it fondly. When I first posted it, I had a few nibbles of interest from people, but it was only a little side project, I could have just as easily not put it on my site.
In the way of the internet, Nat's World had receded into the past, an old post unlikely to get any further attention. The code doesn't even run any more. But someone found it, and because of it they got in touch, and they got to a project night, and connected with other people, and who knows where it will all lead?
I was in the library yesterday, and wandered into the Brookline room, where books particular to Brookline are kept. They have annual town directories going back more than a century. I pulled one down and looked up my street address.
Of course, when we bought the house, the realtor had little stories about previous occupants. We were told it was built by a Cabot, and that he was a bachelor.
The town records gave me two names: John H. Cabot, and F. Ernest Cabot, living in the house in 1905. Googling around a little bit, I found a notice about John H's death:
Is it safe to say that today he would be out of the closet? It goes on:
I love that line: a distaste for retrospect!
"His chamber!" That's a room in our house! Of course living in an old house, you know that people have lived there before, had entire lives there, and felt as proprietary and private about the house as you do. But this sentence somehow made it much more real. This weekend as I have moved through my house, I've been more aware that others have preceded me.
A little more digging shows that John H. is buried in Mt. Auburn cemetery, along with some close family, so in the spring I may look them up...
My new favorite t-shirt:
It's a stellated icosahedron from Henry Segerman, who makes many interesting nerdy things, inspired by both math and juggling:
BTW: stellation is the process of creating new shapes by extending the faces of a polyhedron. The shirt is a stellation of a regular icosahedron (known in gaming circles as a D20). The logo for this site is a stellation of a regular dodecahedron.
Seems like testing and podcasts are in the air... First, I was interviewed on Brian Okken's Python Test podcast. I wasn't sure what to expect. The conversation went in a few different directions, and it was really nice to just chat with Brian for 45 minutes. We talked about coverage.py, testing, doing presentations, edX, and a few other things.
Then I see that Brian was himself a guest on Talk Python to Me, Michael Kennedy's podcast about all things Python.
On that episode, Brian does a good job arguing against some of the prevailing beliefs about testing. For example, he explains why unit tests are bad, and integration tests are good. His argument boils down to, you should test the promises you've made. Unit tests mostly deal with internal details that are not promises you've made to the outside world, so why focus on testing them? The important thing is whether your product behaves right from the outside.
I liked this argument, it made sense. But I don't think I agree with it. Or, I completely agree with it, and come to a different conclusion.
When I build a complex system, I can't deal with the whole thing at once. I need to think of it as a collection of smaller pieces. And the boundaries between those pieces need to remain somewhat stable. So they are promises, not to the outside world, but to myself. And since I have made those promises to myself, I want unit tests to be sure I'm keeping those promises.
Another value of unit tests is that they are a way to chop up combinatorial explosions. If my system has three main components, and each of them can be in ten different states, I'll need 1000 integration tests to cover all the possibilities. If I can test each component in isolation, then I only need 30 unit tests to cover the possibilities, plus a small number of integration tests to consider everything mashed together. Not to mention, the unit tests will be faster than the integration tests. Which would you rather have? 1000 slow tests, or 30 fast tests plus 20 slow tests?
Sure, it's possible to overdo unit testing. And it's really easy to have all your unit tests pass and still have a broken system. You need integration tests to be sure everything fits together properly. Finding the right balance is an art. I really like hearing Brian's take on it. Give it a listen.
Two good things in the Python testing world intersected this week.
Harry Percival wrote a great book called Test-Driven Development with Python. I should have written about it long ago. It's a step-by-step example of building real software (Django web applications) using Test-Driven Development.
Harry describes the philosophy, the methods, and the steps, of doing real TDD with Django. Even if you aren't using Django, this book shows the way to use TDD for serious projects. I'm not yet a TDD convert, but it was very helpful to see it in action and understand more about it.
The entire book is available to read online if you like. Taking the meta to a whole new level, Harry also has the source for the book, including tests on GitHub.
Brian Okken has been running a podcast devoted to Python testing, called Python Test Podcast.
His latest episode is an interview with Harry. People must have thought I was nuts driving to work the other day, I was nodding so much. It was a good conversation. Highly recommended.
Let's say I have a piece of software. In this case, it's some automation for installing and upgrading Open edX. I want to know how it is being used, for example, how many people in the last month used certain versions or switches.
To collect information like that, I can put together a URL in the program, and ping that URL. What's a good simple way to collect that information? What server or service is easy to use and can help me look at the data? Is this something I should use classic marketing web analytics for? Is there a more developer-centric service out there?
This is one of those things that seems easy enough to just do with bit.ly, or a dead-stupid web server with access logs, but I'm guessing there are better ways I don't yet know about.
When calling functions that are expensive, and expected to return the same results for the same input, lots of people like using an @memoize decorator. It uses a cache to quickly return the same results if they have been produced before. Here's a simplified one, adapted from a collection of @memoize implementations:
This is great, and does what we want: repeated calls to expensive_fn with the same arguments will use the cached values instead of actually invoking the function.
But there's a potential problem: the cache dictionary is a global. Don't be fooled by the fact that it isn't literally a global: it doesn't use the global keyword, and it isn't a module-level variable. But it is global in the sense that there is only one cache dictionary for expensive_fn for the entire process.
Globals can interfere with disciplined testing. One ideal of automated tests in a suite is that each test be isolated from all the others. What happens in test1 shouldn't affect test99. But here, if test1 and test99 both call expensive_fn with arguments (1, 2), then test1 will run the function, but test99 will get the cached value. Worse, if I run the complete suite, test99 gets a cached value, but if I run test99 alone, it runs the function.
This might not be a problem, if expensive_fn is truly a pure function with no side effects. But sometimes that's not the case.
I inherited a project that used @memoize to retrieve some fixed data from a web site. @memoize is great here because it means each resource will be fetched only once, no matter how the program uses them. The test suite used Betamax to fake the network access.
Betamax is great: it automatically monitors network access, and stores a "cassette" for each test case, which is a JSON record of what was requested and returned. The next time the tests are run, the cassette is used, and the network access is faked.
The problem is that test1's cassette will have the network request for the memoized resource, and test99's cassette will not, because it never requested the resource, because @memoize made the request unnecessary. Now if I run test99 by itself, it has no way to get the resource, and the test fails. Test1 and test99 weren't properly isolated, because they shared the global cache of memoized values.
My solution was to use an @memoize that I could clear between tests. Instead of writing my own, I used the lru_cache decorator from functools (or from the functools32 if you are still using Python 2.7). It offers a .cache_clear function that can be used to clear all the values from the hidden global cache. It's on each decorated function, so we have to keep a list of them:
Now an automatic fixture (for py.test) or a setUp function, can clear the cache before each test:
In truth, it might be better to distinguish between the various reasons for using @memoize. A pure function might be fine to cache between tests, who cares when the value is computed? But other uses clearly should be isolated. @memoize isn't magic, you have to think about what it is doing for you, and when you want to have more control.
I wasn't planning on any big features for Coverage.py 4.1, but I ended up making a big change. tl;dr: branch coverage is implemented differently, and as a result, your coverage totals are slightly different. Try it: Coverage.py 4.1 beta 1.
Because of Python 3.5's async and await keywords, the existing branch analysis based on bytecode was completely out of gas. The code had long felt jury-rigged, and there were long-standing bugs that seemed impossible to solve. The async features compiled very differently than their synchronous counterparts, and I didn't see a way to bring them into the bytecode fold.
So I ripped out the bytecode analysis and replaced it with AST (Abstract Syntax Tree) analysis. I like it much better: it's based on the structure of the code that people see and can reason about. Four old bugs were fixed as a result, along with the two or three new bugs reported on Python 3.5.
As a result though, coverage.py now calculates totals differently, because the full set of possible code paths is different. So your results will likely shift a little bit, especially if you are using branch measurement. They might be higher, they might be lower. For example, class docstrings now count as executable statements (because they are executable statements), and paths through "except" clauses probably were being overlooked.
Please try it out and let me know if you see anything wrong. I won't be surprised if there are complex code structures that are analyzed wrong somehow. Coverage.py 4.1b1: you know you want it...
When I look back on 2015, I'm happy about a number of things, but two that stand out have nothing to do with software or the virtual world.
This past year, partly inspired by watching co-workers learning to juggle three balls, I made a new concerted effort. I've made significant progress. Now I can routinely make ten throws, and cleanly catch all five balls. 15 or 20 throws is not unusual, and my record has increased to 30. More importantly, now every day that I practice for even 10 minutes feels like I'm getting slightly better.
This video by Niels Duinker helped to give me drills. The four-ball tosses were especially helpful, because they enabled me to hear how my throws were lop-sided. Practicing those until they evened out really opened up the way to move forward. Now it no longer feels like a crisis to have five balls in motion.
My old feeling when trying five balls was frustration. Now I feel the improvement, and it's encouraging and make it enjoyable to practice, which leads to more improvement. I'm looking forward to more progress in 2016.
My other physical activity is swimming. I set a goal for myself to swim a total of 150 miles in 2015. After a slow start to the year due to blizzards and colds, I caught up in the fall, and ended with a total of 154.02 miles. (The precision is illusory: I counted the distance of summer swims at the beach as very rough guesses.)
This year in the pool I figured out flip turns. I used to think I got dizzy trying to do them. That wasn't the problem, the problem was water up the nose, being too far from the wall, end up too deep, not breathing enough, etc. But I watched others, and practiced, and now they feel natural and easy. I like feeling confident in this new skill.
In the coming year, I want to add dolphin kicks to the repertoire, and then maybe consider the butterfly.
I spend a lot of time online, and writing software. That involves many kinds of abstract mental activities. It's great to be able to learn new skills and techniques in that world, and there's no shortage of things to learn. But having physical world challenges is satisfying in a very different way.