« | » Main « | »

Simon's optfunc

Friday 29 May 2009

There's a Moore's Law effect in Python command-line processing. Every 18 months, it gets 50% easier or 50% more powerful. Simon Willison is keeping the effect going with optfunc, which introspects a function definition and creates the optparse parser needed to grab its arguments from the command line. Very clever.

Buying online and offline

Friday 29 May 2009

I recently replaced our home phones because the old ones weren't charging well, even with new batteries. I bought them online, had them delivered, set them up, and didn't like them. The ring is strident, and the keypad has an aggravating quarter-second delay between pushing the button and making the tone. The phones are just not a pleasure to use.

I kicked myself for buying online where all I could tell was how the phone looked, and quantitative facts like battery type and price. I should have gone to a store. So the next time I was in a Best Buy, I went to look at the phones. And then I remembered: you can't tell anything about the phones there either. First, none of them do anything, they are all non-working display models. Second, you can't even pick some of them up: they are glued into the base, or have a cable awkwardly attached to their back. I had gotten into a car, driven to a real brick-and-mortar store that Best Buy had expensively built and staffed, and I had pretty much the same experience as I had online.

I understand that Best Buy needs to prevent theft, but don't they also need to encourage purchases? I'm told that especially for laptops, trying the product physically is an important part of the purchasing process. Among many other reasons, Apple stores are part of the success of Apple laptops: customers are encouraging to touch and try the real thing.

For at least one of my complaints about the new phones, the online site could help me out: give me a usable emulation of the on-screen menu, and let me hear the noises the phone can make. For the keypad delay, there's not much they can do.

Phones (I almost said "cordless phones", but that would have dated me!) are a commodity these days, and price and appearance are probably the two most important considerations, so maybe I shouldn't be surprised. But I would expect physical retailers to take whatever advantages they could.

Mostly wrong

Monday 25 May 2009

Jame Iry sums up the history of programming languages: A Brief, Incomplete, and Mostly Wrong History of Programming Languages. It's hilarious and sort of true:

1964 - John Kemeny and Thomas Kurtz create BASIC, an unstructured programming language for non-computer scientists.

1965 - Kemeny and Kurtz go to 1964.

Predictably, the comments include contributions of other "milestones", and the usual debates over whether something is a "scripting language" or not.


Saturday 23 May 2009

I saw Objectified the other night. It's a Gary Hustwit (of Helvetica fame) documentary about industrial design, and its effect on our lives. I enjoyed the movie, it's great to see a little bit behind the scenes of how things are designed, and the people he interviewed were thoughtful and intelligent on the subject.

Early on in the film, he makes the point that every man-made object has been designed, nothing is accidental. But then later, you see stores like Ikea and Target which explicitly sell you on the idea of "Design for the masses". Which made me realize that when someone talks about "design", they mean "good design", or at worst, "self-conscious design".

But I thought the most telling part of the film was near the end when the journalist Rob Walker proposes a thought experiment: imagine a hurricane will strike your house in 20 minutes, what do you grab? And the answer is that it has nothing to do with the design of things. It's only about your emotional connection to them, usually based on what you see through and beyond them (photos or heirlooms, for example). It seemed to be a coda to the film that basically said, "As cool as this intentional design is, it doesn't really matter." Good design gets products in the door, it may be why you buy something in the first place, but it won't be what gets it onto your 20-minute grab list.

The designers throughout the film talked about that, about needing to make a connection with people, about understanding them and designing an object well enough to touch them at a deep level, but I don't know if there's anything a designer can do to get his work onto the grab list. Good design can surround us with beauty and function and comfort, but in the end, it's the Rosebuds in our lives that really matter.

Mona Tweeta

Wednesday 20 May 2009

Mario Klingemann undertook a challenge: to somehow represent an image in a tweet. Not by pasting a URL to the image, but by actually cramming the data into 140 characters. The result is better than you think, and involved all sorts of technical craftiness.

First, he used Chinese characters, to get 210 bytes of data into 140 characters. Then he used clever color and point coordinate representations to optimize the use of the handful of bytes available.

One of my first jobs was at General Computer Corporation, which made games for the Atari 2600. The 2600 system had only 256 bytes of RAM, so the designers of games had to cram all of the mutable data, including the program stack, into 256 bytes. When it was time to plan a new feature for a game, the developers would bring out the memory map, a sheet of paper detailing what each bit of those 256 bytes did, and find a bit that was unused.

And now here we are on multi-gigabyte machines, trying to squeeze something interesting into 210 bytes!

It seems every time we think technology has moved beyond some irritating limit, new tech comes in and re-introduces it to us. No more 8-bit video, unless you're on a VNC connection. Screens are big, unless you're in a phone. Network is always connected, unless you're in a phone. Actually, most of the limitations come back once you're in a phone.

Or on twitter!

Coverage v3.0 beta 3

Saturday 16 May 2009

I hadn't planned on doing a third beta of Coverage.py, but two things happened:

  • I changed a number of details about the programmatic interface, and so wanted to get a little more verification that it's right.
  • I discovered that the beta 2 kits weren't complete: the brand-new HTML reporting feature wouldn't work because the files were missing from the kit. Ugh.

So, beta 3 it is. Kits are available as source or as Windows installers from the coverage.py page on PyPI. The code is also available from the repository on bitbucket.

Feedback is welcome in any form you like, but particularly good ways are tickets on bitbucket, or email on the testing-in-python mailing list.

Significant changes in coverage.py since v2.x:

  • HTML reports and annotation of source files: use the new -b (browser) switch. Thanks to George Song for code, inspiration and guidance.
  • The trace function is implemented in C for speed. Coverage runs are now much faster. Thanks to David Christian for productive micro-sprints and other encouragement.
  • Code in the Python standard library is not measured by default. If you need to measure standard library code, use the -L switch during execution.

Log in as a user

Friday 15 May 2009

This s a question that pops up every once in a while in web site technology discussion forums: if I'm running a web site, what's the best way to log in as a particular user so that I can see the site as they would see it?

The simplistic thing to do is to somehow get the user's password, and just log in as them. But this is bad because you should never need to know your users' passwords, and ideally, you don't even have access to them.

The better solution is to interpret the classic username and password fields liberally. The username field is always interpreted as the desired username. The clever part comes with the password field: it can either be the user's password, or it can be a combination of a super-user username and the super-user's password. It's the online equivalent of getting access to Joe's stuff either by showing your identity card proving you are Joe, or by being a policeman and showing your badge.

To make it concrete, imagine authentication is implemented in a function called who_am_i that takes the username and password, and returns the authenticated username, or None. You have an is_password function that can tell you whether a username and password match, and an is_superuser function that can tell you whether a username is a super-user.

Here's the classic username/password authentication:

def who_am_i(username, password):
    """Determine what user these credentials represent."""
    if is_password(username, password):
        return username
    return None

Here's the extended version. An @-sign is the separator in the password field for the super-user's name and password:

def who_am_i(username, password):
    """Determine what user these credentials represent."""
    if is_password(username, password):
        return username
    if '@' in password:
        super_name, super_pass = password.split('@', 1)
        if is_super(super_name):
            if is_password(super_name, super_pass):
                return username
    return None

I like this solution because it doesn't require any extra UI, but gives super-users what they need: a way to log in any user they want, without knowing the user's password. If you like, you can strengthen the super-user path by requiring white-listed IP addresses, or special privilege cookies, etc.

BTW, stackoverflow has more discussion on the same topic, including other (bad) ideas about how to do it (change the user's password??)

Web font embedding

Sunday 10 May 2009

There's a lot of talk of web font embedding these days. Dave Shea covers the technology and debate pretty well, as well as showing one (flawed) technique for accomplishing it today. Mark Pilgrim has a much more radical opinion, which also serves as a jumping off point for a very spirited debate in the comments.

Basically, it comes down to this: web designers want to be able to use more than just the web-safe fonts in their pages, which means somehow linking to interesting fonts. Font designers want to keep their fonts from becoming free-ware, so want some form of control over where the product of their sweat ends up. In some ways, it's a classic DRM battle, but over typefaces rather than songs or films.

There are some good free fonts out there, (also here and here), but not many. The high-quality faces, which require not only good design but good implementation in the form of hinting, cost real money, which money funds their development.

So there's plenty of discussion about this, but no clear resolution. The browsers are just beginning to support @font-face, so it will become more of an issue in the coming year. Hopefully we can resolve this like civilized people, without the battles the music and movie industries have been relishing.

Running a Python file as main, take 2

Saturday 9 May 2009

Recently, I tried to simulate the way Python runs programs, but I was worried that there was a gotcha. It turns out there is one: when imp.load_module loads a Python source file, it compiles it to bytecode, and saves it as a file.

Python uses a very simplistic way of naming the compiled file: it adds a 'c' to the end of the source file name. This works well when the file is "foo.py" and the compiled file is "foo.pyc". But if you try to run a script in this way, the file may not have a .py extension. So for example, if the python file is named "zzz", then load_module will create a compiled file named "zzzc". This is bad.

There was no reasonable way to prevent this compiled file creation, so I went back to the execfile technique, instead setting up more of the proper sys structures to get the proper execution context. If you're interested in the details, the code is in execfile.py, with tests in test_execfile.py.

BTW: This was basically Fredrik's advice from the first post, and he was right. Marius suggested I look at runpy in the standard library, and it's similar to what I'm trying to do here, but it takes a module name rather than a file, so it isn't quite right.

OS truce?

Wednesday 6 May 2009

Tell you what: I'll be careful not to create Python kits with \r\n in the source files, if you Mac guys will stop including files like ._setup.py in your tarballs. OK, thanks. Resume coding...

Hg ftw

Tuesday 5 May 2009

These are good days for Mercurial, the Python-based distributed version control system. First Python chooses Mercurial. Then Google Code announces support for Mercurial, the first DVCS it supports. Now ActiveState comes out with Workspace, its project hosting product, which supports Subversion, git, and you guessed it, Mercurial.

I don't understand the reasons why Mercurial is getting all the headlines these days over Bazaar, which has a number of high-profile users of its own.

But I've liked Mercurial since choosing it and Bitbucket for hosting Coverage.py. I'm glad that a clear leader is emerging among the non-git DVCS systems. It makes decisions easier.

« | » Main « | »