« | » Main « | »

Things I don't like about doctest

Sunday 30 November 2008

Python's doctest is very cool technology, letting you write a narrative in documentation that can then be executed directly to test the code. But as cool as it is, I don't like it very much:

  • You can't run a subset of the tests.
  • If a failure happens in the middle of the doctest, the whole thing stops.
  • The coding style is stylized, and has to have printable results.
  • Your code is executed in a special way, so it's harder to reason about how it will be executed, harder to add helpers, and harder to program around the tests.

Most importantly, though, doctest seems like an odd way to write tests. Docstrings, and the long sequence of code they encourage, may be good ways to explain what code does, but explaining and testing are two different tasks, and the code you write for each will be different. So why try to serve two masters at once?

Either your expository text will be cluttered with uninformative edge cases, or your tests will merely skim the surface of what your code can do.

I know that doctest can be used independently of the actual docstrings in the code, but then where's the great advantage? I'd rather use real software engineering tools to write my tests, and the idiomatic way doctest executes code and evaluates results don't help me.

I'm not the only one who feels this way. Andrew Bennetts has two posts with much more detail about these issues: Narrative tests are lousy tests and Tests are code, doctests aren't.

While I admire the ingenuity that went into doctest, I just don't find it a good tool for testing real code.

They want Sandy

Friday 28 November 2008

Rael Dornfest ran a company called Values of n which built two web services, stikkit and I Want Sandy, both of which seem to have been well designed and very well received. But the intellectual property of the company, and Rael himself, have been acquired by Twitter, and both services are shutting down with two weeks' notice. People are naturally upset (from a comment by ToddZ):

Chalk me up, too, as disappointed in this rather callous decision. Not only disappointed in the loss of Stikkit and Sandy, but disappointed that the Twitter folks would bring someone onto the team who is so willing to scuttle a boatful of passengers to leap unencumbered to the next lucrative opportunity.

Yeah yeah, these are free services - but they were made, allegedly, to benefit users. You invited us to make these services part of our day-to-day lives. That was their purpose. Now, having been successful at that, to so easily dispose of their users calls into question your motives.

No more trust from me. No more investment of my time, energy, and evangelism. I'll be canceling my Twitter account. Not going to get burned again.

There's more user reaction at the thread on getsatisfaction.com.

I understand this dilemma all too well. We have fought to keep Tabblo alive after the acquisition by Hewlett-Packard, and have not always done a good job of it. These web 2.0 companies all offer free services, and something has to pay the bills, so it isn't always possible to be altruistic forever.

But at the same time, we live in a web world where the culture is to offer services free or very cheap. None of these interesting technologies would have flourished if they had to be bought from day one. How many photos would Flickr currently have if you had to pay $25/year to upload anything? How many tweets would be sent if you needed a credit card and paid 2 cents to send one?

So it's a delicate balancing act. Closing down a service with two weeks notice seems pretty harsh to me, especially if one of those weeks is Thanksgiving when many people may not notice. How hard would it be for Twitter to keep those services running? For the sake of all of us that want to experiment with free services to try out cool new technology, everyone has to be careful not to be too cavalier with the trust users put in us. It's hard enough for a small team to build something people will invest their focus in — to shut it down as if it were no big deal is careless indeed.

As it is, I only heard of I Want Sandy because of this controversy. Will its legacy be that it was the Web 2.0 thing that burned a bunch of users? Rael and Twitter were remarkably clumsy with this transition. Forget about the technology, they are both in the business of building stuff that people want to use. That includes building trust that those services will stick around.

No amount of disclaimers now that the service was free will change the fact that everyone involved in building it really wanted users to invest their time, energy and trust in the service. If you want users to use your service, you have to treat them right. "It was fun, I'm on to the next thing" isn't good enough, and "it was free, what did you expect?" is insulting. No one starting a new service like this would say, "It's free, we'll keep it running until something else comes along," on their home page, so they shouldn't treat the shutdown of the service that way.

This is a black eye for the whole industry. People are already suspicious of using free in-the-cloud services for important data. Rael's handling of this shutdown seems uncaring. He and Twitter should have done a better job handling their most valuable possession: people's trust.

Print this file, your printer will jam

Monday 24 November 2008

Another story from printer-land of 20 years ago: this time about a seemingly impossible bug.

While working on the LPS-20 PostScript software, a bug was filed that said roughly,

Print the attached file. The LPS-20 will jam. You'll have to open the printer to remove the scrunched up paper.

We were no strangers to jammed printers, but a particular file that could jam the printer? Yeah, right. It was crazy.

I printed the file. The printer jammed! I cleared the jam, printed the file again, it jammed again. I printed a different file, it printed fine. I printed a third file, it printed fine. Printed the bug report file again, the printer jammed, WTF!? How can a file reliably cause a printer to jam?

The mystery in this case was solved by the hardware team, because while we software guys were working on the software that fed PostScript files to the printer, the hardware guys were still getting the kinks out of the printer itself.

A laser printer has a drum on which the image is formed, and then used to transfer the image to the paper. In a 20 ppm printer like the LPS-20, the drum rotates once every three seconds. With a PostScript printer, a page could take an arbitrary amount of time to render. If the page takes less than three seconds to render, then the drum will rotate at full speed, with no pauses.

But if the PostScript interpreter takes longer than three seconds to finish a page, then the drum has to stop and wait for the image to be ready. Starting and stopping the drum and all the associated feed machinery is not trivial to get right.

In the case of the bug report file, the page took longer than three seconds, but only slightly longer, so not only did the drum have to stop, but it hadn't quite totally stopped before it was started again. That mechanical edge case is what made the printer jam. The file had pages that took just the right amount of time to hit a bad timing window in the drive train firmware, and the printer jammed every time. Once the hardware guys adjusted the firmware, the problem went away.

Lessons:

  • Just because a bug seems impossible doesn't mean it is.
  • Abstractions are everywhere, and they can be broken. As a software guy, I believed that getting the paper into the output tray was a solved problem.

My oldest code still running

Sunday 23 November 2008

One of the odd things about working at Hewlett-Packard is that part of it is what's left of Digital Equipment, where I worked for seven years, from 1986 to 1993. Every once in a while, I'll stumble on some new artifact of that history. Lying next to a printer, I found this, a VMS banner page:

A VMS PostScript banner page

It took me back, because back in the late eighties, I worked on this software, meaning not just the printer software in VMS, but the PostScript code that drew this page.

This is probably the oldest software of mine still running. Funny to think that while I went off to pursue hundreds of other projects, this PostScript code was obediently drawing this simple page over and over for 20 years.

Back when we were first developing that banner page, it was being used on an LN03R, an 8 page-per-minute laser printer. People complained that the banner page was taking too long to print. I was given the job of toning down the PostScript complexity so that it would print more quickly. People felt like all those scalable fonts must be a bottleneck.

After doing a number of controlled experiments, and whittling down on the pages, it became clear that nothing was making it go faster. Finally I reduced the code to:

% Just print a blank page
showpage

and the blank page took just as long to come out of the printer as the "fancy" banner page with all the different font sizes on it.

What I learned was:

  • Just because a technology is new doesn't mean it's the culprit.
  • When waiting for your print job, any pages you don't want will seem too slow.

Judge orders five detainees released

Friday 21 November 2008

A federal judge has ordered five Guantanamo detainees released because the case against them is weak. I was disgusted to hear the details of this case. When the men were first captured, the Bosnian government thought the case was too weak against them, but handed them over to the U.S. anyway. We tossed them into Guantanamo, and argued that they shouldn't be allowed to challenge their detention. In June, the Supreme Court decided they had that right, and now the government drops the primary case against them, resorting to lesser charges of planning travel to Afghanistan to train as terrorists, but the judge rules that even that case is too weak to be supported. Meanwhile, they've been held at Guantanamo for seven years!

I'm glad these men are having their day in court. Too bad it's taken this long. We should be strong enough as a nation to allow anyone to plead their case.

Pathological backtracking

Wednesday 19 November 2008

At work we've been using the well-regarded feedparser module to parse RSS feeds, and it works great for the most part, but we'd occasionally get a stuck server process. The CPU would spike to 100%, and wouldn't make any progress.

We discovered a particular feed would cause a particular regular expression in the code to spin endlessly. The regex was intended to determine if a style attribute is valid CSS:

if not re.match("^(\s*[-\w]+\s*:\s*[^:;]*(;|$))*$", style):
    return ''

Breaking this out into verbose regex syntax shows how it matches valid CSS:

"""(?x)             # use verbose regex syntax
    ^(
                    # A single CSS clause is:
    \s*             #   leading whitespace
    [-\w]+          #   a dash-word, the property name
    \s*:\s*         #   space, colon, space
    [^:;]*          #   anything but :;, the value
    (;|$)           #   ends with a semi or the end of the string
    
    )*              # Valid CSS is any number of clauses
    $
"""

And here's the snippet discovered in the feed that spun us hard (with whitespace added for readability):

<var style="COLOR: #fffafe; coming: ; basket: ; philologist: ; gradually: ;
encyclic: ; whitechapel: ; left: ; albino: ; lamelliform: ; foment: ;
adjuvant: ; Room:  ; Milk:  ; buynow: ; wheelwork: ; unseal: ; reasons: ;
socalled: ; dazed: ; Brain:  ; Kaleidoscope:  ; hardheaded: ; asthenic: ;
preferred: ;  Barbecue:  ; Comet:  ; Nail:  ; lubberly: ; School:  ;
Mist:  ; undercurrent: ; intwine: ; isotonic: ; Chief:  ; miscellaneous: ;
Book:  ; Shoes:  ; Chocolates:  ; deuced: ; you: ; Man:  ; federalize: ;
Rainbow:  ; Satellite:  ; Printer:  ; amicus: ; tautophony: ; taking: ;
regrater: ; waggon: ; prescient: ; God:  ; prosing: ; Bank:  ; hariolation: ;
patriarchs: ; Pyramid:  ; Data Base:  ; PaintBrush:  ; ingenu: ; Rope:  ;
parenchyma: ; price: ; Alphabet:  ; Circle:  ; seeks: ; frankhearted: ;
vituperate: ; dysmeromorph: ; Shop:  ; firm: ;  imperforation: ; lane: ;
Gemstone:  ; slatternly: ; Fire:  ; impudence: ; Carrot:  ; Fan:  ;
inoccupation: ; uncover: ; Liquid:  ; drawee: ; Pocket:  ;barbacan: ;
fornicatress: ; chimes: ; Crystal:  ;innovation: ; years: ; untiring: ;
Freeway:  ;desertful: ; unreined: ; Compass:  ; Hose:  ;prelusive: ;
impenetrability: ; Fruit:  ; direct: ; "></var>

(yes, it's garbage, and yes, spam sucks.)

It's hard to see the problem here, but this is not valid CSS because they used "Data Base" as a property name about half-way through and spaces aren't allowed in property names.

The CPU spins because when the regex encounters the failure to match "Data Base", it backtracks to reconsider previous matches in the hopes that it can still make the regex work. In fact, it isn't in an infinite loop, just a very very very long one. Eventually this regex will finish and decide that the string doesn't match.

But we don't need it to backtrack: going back to re-match previous CSS clauses isn't going to help.

Some regex libraries offer solutions to this problem. Possessive quantifiers let you use *+ to mean, match as many as possible, and once matched, don't try matching fewer during backtracking. They're called possessive because once the operator claims part of the string, it won't give it back for other operators to match later.

But Python doesn't offer possessive quantifiers (yet yet). So we have to choose a different technique than trying to match the whole string in one large regex. In this case, since we don't need the match data, we're just checking that the whole string matches, so we can use re.sub to remove matching clauses and then check that there's nothing left over:

if re.sub("\s*[-\w]+\s*:\s*[^:;]*;?\s*", '', style):
    return ''

Because re.sub grabs matches, performs the replacement, and moves on, there's no needless backtracking to throw a wrench in the works. Now our crazy CSS spam is speedily dispatched as invalid.

As an interesting side effect, if the string is not empty, what remains is the invalid part of the string.

Presentation tools

Tuesday 18 November 2008

I've had presentations on my mind lately, and that has caused me to search out a good tool for doing them. I have a Windows laptop, so PowerPoint is always an option, but I wanted to try other alternatives first. Unfortunately, I found them all lacking.

I want a way to author and show slides, and then a way to export them for use in a web page. I've got a goal of putting the presentations online not just as a deck of images, but as a text transcript, illustrated with slides. I've never been able to watch other people's presentations online because I don't have the patience to watch an hour-long video in real time, and a slide deck with none of the actual talk behind it is pretty spare.

S5 (A Simple Standards-Based Slide Show System) is all HTML, CSS, and JavaScript, runs in the browser, and was created by Eric Meyer, a very nice pedigree. Remarkably, the thing I like least about it is that when I display the slides in a full-screen Firefox, they look horrible. The text is either too small or too large, and the line-spacing too tight, while the slide title overlaps the text in the wrong way, exposing some of the structure of the divs.

I would have hoped that a CSS-based slideshow by the king of CSS would be a shining example of how information could be cleanly authored and then sparklingly displayed. S5 seems to miss this mark, especially since there don't seem to be many themes available for it, another surprise given how CSS should have made it accessible to lots of designers. Also, although (or perhaps because) the format is native to the web, it's not possible to get the slides as illustrations.

I tried OpenOffice Impress, and was initially impressed. It's got a lot of presentation features, notably all sorts of animations. When I started using it, though, I ran into some problems: there's no way to create a text style, so any code samples have to be tediously formatted by hand. Although there are tons of slide animations, a bullet list that reveals one bullet at a time is not among them. I briefly considered the scripting capability, but it seems arcane and under-documented.

Also, I have to say that OpenOffice's themes are cheesy, and though I found one that looked good, when I installed it, it didn't work.

I briefly tried 280 Slides again, and it is very impressive, but too slow to run in my browser.

So I may be using PowerPoint...

Tabblo at the Cambridge Django meetup

Wednesday 12 November 2008

Tomorrow night, Thursday November 13, Dave St. Germain and I will be talking about Tabblo at the Cambridge Django meetup. We aren't that organized, so I hesitate to call it a presentation.

Tabblo.com is a fairly large site, but since being acquired by Hewlett-Packard, we've built a number of other sites running on the same code base. We'll be talking about some of the things we've had to do to keep flexible and light on our feet while building and maintaining our web experiences.

If you're interested, drop in.

Us on abcnews.com

Monday 10 November 2008

Back in June, Susan and I did an interview with abcnews.com for a feature they were doing on autism families. The piece is finally up on their site: Autism Voices and Views. We talked about Nat's involvement in Special Olympics, and they took pictures at a swim practice. I think they did a good job capturing us.

ABCnews Autism Voices and Views

In one of the shots, I am walking on the pool deck, holding my hand above Nat while he swims the back stroke. This is a swim coach hack I invented to try to get him to lower his head into the water. He naturally swims with his head held up out of the water, as if he's trying to see his feet. To get him to put his head back, I tried walking the deck ahead of him, telling him to look at my hand. It worked for a little while, until he realized there was nothing very interesting about my hand!

This is one of the interesting things about raising a special needs child: you come up with your own techniques, because even the specialists will need to invent stuff specially for each child, and you know yours better than anyone, so you've got a good shot of coming up with the successful tricks.

And then you realize that this is true of all your children!

President Obama

Wednesday 5 November 2008

I am very pleased that Obama has been elected president. It's been a very long election, and of course there have been fights and probably hard feelings. I hope Obama can truly unite people. Of all of the practices of the last eight years that I want Obama to reverse, the most important one is to lead so that 100% of the people feel like he is their leader, rather than just the 52% who voted for him.

I don't know if that is possible, since it will require effort from everyone, including the people. Reining in an overwhelmingly Democrat congress may be difficult, but it is crucial. Of all the reasons Obama won, the most important may be that people were punishing the Republicans for their excesses when they controlled all of government. If the Democrats are smart, they will learn from that. If they do not, we may just see another whipsaw election next time around.

Many have said that Obama will have a very hard time leading, given the economic crisis, and the war to deal with one way or another, and that is very true. I hope that he can do more than just run the ball a few yards down the field, I hope that he can genuinely lead.

Gear cube

Tuesday 4 November 2008

Haruki Nakamura has made a truly outstanding gear cube out of paper:

I want one of these!

Nakamura's site is unfortunately all Japanese, and doesn't offer instructions. It does have photos of other fascinating work, though...

Stackoverflow social dynamics

Tuesday 4 November 2008

Stackoverflow is a new programmer's question and answer site from Jeff Atwood and Joel Spolsky, and it's a very nice site: cleanly designed, with lots of dynamic page goodness.

The part that interests me about it, though, are the social dynamics. They've done a clever job with their reputation system: users on the site earn reputation points based on their participation, for example, based on how other community members vote up or down on answers. The points don't mean anything, you can't cash them in, but having them displayed next to your name everywhere on the site is a powerful motivator, especially for quantitative engineer-types.

In addition to reputation points, there's a large collection of badges you can earn, for example by asking a popular question, or doing a lot of re-tagging.

Put together, reputation and badges are powerful motivators. They turn the site into a game, where being useful members of the community gives you tangible (though virtual) points. Questions are answered within seconds of being asked, as eager developers try to pounce and be the first with the correct answer. Questions with widely-known answers frequently have a half-dozen correct responses within minutes.

I don't see much on the site that will engender a real community, though. In fact, the questions are strictly policed to be about programming, and any off-topic discussion is quickly curtailed. This might be a mistake: off-topic threads are an important part of building a larger sense of place. On the other hand, ensuring all discussion is "useful" will make the site much more appealing to new people who aren't won't get the in-jokes.

The primary goal of stackoverflow is to be a useful repository of answers to programming questions, and I think it will succeed at that. Time will tell if it goes in other directions and becomes something larger.

BTW, one other dynamic: since Jeff and Joel are Windows developers, their readership is very Windows-heavy, and as a result, the stackoverflow crowd is heavily tilted in that direction as well.

« | » Main « | »