« | » Main « | »

Recent tweets

Saturday 30 October 2010

Recent tweets of potential interest:

¶  Bye MS: "[Fitting a square peg into a round hole] can be done but only at major cost to both the peg and the hole." - http://hugunin.net/microsoft_farewell.html (Oct 22)

¶  Anyone know the answer to my oddball French comics question: what's up with the tire and two sticks? Answer: http://qr.ae/776w (Oct 03)

¶  So true: RT @pheinberg: "There are only two hard problems in Computer Science: cache invalidation, naming things, and off-by-one errors." (Sep 30)

¶  Gotta love CSS humor: RT @sgalineau: Hot. Even works in IE6! RT @zeldman The 997 Grid System http://j.mp/c7trUD #css #grid (Sep 24)

¶  RT @duganesque: RT @bob_hancock: A Java programer meets Django. http://rheide.wordpress.com/2010/09/05/java-versus-python/ (Sep 06)

¶  RT @r1chardj0n3s: Penn and Teller take on vaccines - http://www.google.com/buzz/r1chardj0n3s/i9Mqbrn9fM4/Penn-and-Teller-take-on-vaccines-Bad-Astronomy <= Excellent stuff. Should be required viewing. (Sep 01)

¶  RT @dweinberger: Incredible 3D lightshow. A building becomes a toon. http://www.hyperorg.com/blogger/2010/08/29/a-building-becomes-a-toon/ (Aug 29)

¶  No arms, no legs, no worries: http://www.wimp.com/noworries/ (Aug 28)

¶  No Rubik's cube needs more than 20 moves to solve, God's number = 20: http://www.cube20.org/ (Aug 10)

¶  Detailed analysis of the real-world cave locations in the original Colossal Cave text adventure: http://www.digitalhumanities.org/dhq/vol/001/2/000009/000009.html (Jul 31)

¶  Whoa: play until it explodes: http://wonderfl.net/c/tNGi/fullscreen (Jul 28)

¶  Oh the humanity... RT @sgalineau: likes ! RT @happycog Swashes and flourishes? Say hello to 2010 OpenType Comic Sans. http://cot.ag/a9XOA4 (Jul 22)

Juan Williams is a bigot

Saturday 23 October 2010

NPR fired Juan Williams because he said this:

I'm not a bigot. You know the kind of books I've written about the civil rights movement in this country. But when I get on the plane, I got to tell you, if I see people who are in Muslim garb and I think, you know, they are identifying themselves first and foremost as Muslims, I get worried. I get nervous.

When I read this, I see a classic example of bigotry. Juan saw a person who was different from him, chose the most unfamiliar aspect of that person, decided it was the defining characteristic of that person, and then used that characteristic to group him into a negative stereotype. Can anyone explain to me how that isn't bigotry?

Juan defends himself against the bigot label by pointing out that he has written books about the civil rights movement. Isn't this just an updated version of "some of my best friends are black?"

Bigotry isn't a binary attribute, where either you are a bigot or you aren't. Juan may be the most open, loving, welcoming, tolerant person there is. He can also be a bigot when it comes to Muslims. Just because he writes books about the civil rights movement doesn't mean he isn't unfairly pre-judging Muslims. He can do both at once, it's now apparent that he does.

The most telling part of his quote is "they are identifying themselves first and foremost as Muslims." No, they aren't. Juan Williams is identifying them first and foremost as Muslims. They might also have been American citizens, or doctors, or fathers, or football fans, or Rotary club members, or even 9/11 survivors. There are many things that they are, and they weren't given a chance to identify themselves. Because they were wearing Muslim garb, Juan Williams decided they were first and foremost Muslim, and that Muslim equated with dangerous.

And the ultimate irony in this is that the Muslims who have actually been dangerous on planes weren't wearing Muslim garb. In all likelihood, the people Juan encountered were the least likely to be dangerous, because if they were planning to cause trouble, they wouldn't want to stand out.

I don't know if NPR should have fired Juan Williams. It sounds like NPR might have overreacted because of the Fox News connection.

Others are jumping on this story as an example of how we are bending over backwards to accommodate fundamentalism, but that's nonsense: simply being a Muslim is not fundamentalism, and assuming all Muslims are dangerous is not a rational response to the radical Islamic threat.

Others are saying that Juan Williams simply expressed what many are feeling. That may be true, I'm sure many people these days are wary of people that "look Muslim." But it's still bigotry, and isn't right. It's a reaction we need to resist.

What is this thing?

Tuesday 19 October 2010

Hewlett-Packard is closing its facility in Marlborough Massachusetts. It's the old Digital Equipment MRO building, and many of the people here are Digital-vintage employees. In the process of cleaning out offices to prepare for the move, people find stuff they've had for a long time and no longer want. It goes into dumpsters or otherwise set out for others to consider. Most of it is true trash, empty pendaflex folders, boxes of floppies, manuals for products no one even remembers.

But occasionally there's something truly interesting. Here's one I found. I don't know what it is, and am hoping someone has an idea.

It's not large (5.5 inches square), but heavier than you'd expect:

The mystery thing

The connectors on the side seem made for heavy-duty current, but are connected to fine traces in the ribbon material wrapped over the edges. The gold area between the chips is etched with perpendicular lines, but they don't seem like circuitry, just a texture:

Showing more detail

The "chips" seem permanently placed, and eight of them are stamped "Mechanical Sample", with a Motorola logo:

Closer still

Underneath looks like a heat sink, though it isn't aluminum, with a hex nut for each of the nine chips on top:

The underside of the mystery thing

Anyone know what this is? Can you explain its mysteries?

Engineers as leaders

Saturday 16 October 2010

I attended the MassTLC Innovation Unconference on Thursday, and it was very interesting. One session I sat in on was about engineers as leaders, and how to help them. I made a few observations there that have been rolling in my head since:

In my experience, there are two particular stumbling blocks for engineers becoming leaders or managers. The first is that they discover that other engineers can behave irrationally. That is, engineers are people. Engineers understand that the whole world doesn't work they way they do. They know that if they have to deal with Sales or Marketing people, that there will be some cultural friction. But they are surprised to learn that once they become a manager, the engineer in the cubicle next to them is actually kind of messed up, and has emotional issues to deal with, and will not behave like a nice linear white box.

Engineers are most comfortable with predictable systems that they can hack to acheive a desired outcome. People are difficult to model this way. It isn't impossible, but it's a lot more involved than most engineers assume at first. Lots of engineers as managers are really good when the people under them don't need any management. I probably fall into this category more than I'd like to admit. Once the problems crop up, the manager doesn't understand why "we can't all be adults," or where this "soap opera" came from. Tough luck, dude, you hired people instead of robots. They're complicated.

The second difficulty is with indirect work. As a manager, you often aren't building something yourself, you are enabling someone else to build something. You're setting direction, triaging bugs, coordinating groups, resolving disputes, and so on. All of that is necessary to the group building something, but you don't feel like you are building anything yourself. A common complaint for an engineer-turned-manager is "I didn't do anything today."

Of course, you did a lot today, you just have to value it. It can be hard to see the effect your actions have, because they are at a distance. As an engineer, we're taught to value direct creation. A clear sign of progress is dirt under your fingernails and a new gizmo sitting on the workbench. Learning to appreciate the subtler effect you can have on a group of people building gizmos is harder.

While we're on the subject of engineers as leaders, a complementary topic is engineers as followers. Too often, engineers are solo creators. They are used to doing things themselves. It can be hard for them to ask for help. This could be because they don't want to seem incapable, or because they don't want to bother others, or just because they are fascinated by a problem and want to figure it out for themselves. Teaching engineers to ask others when they need it is a valuable skill that you won't find on any engineering curriculum.

I've encouraged my developers to become "lazy weasels." Why build something when you can find it ready made, why struggle through reverse-engineering a dense morass of old code when the guy who wrote it is two doors down? Industry and diligence are virtues, "shortcut" is a bad word, sure. But using the appropriate tool is always a good thing, and what if the best tool for the job is someone sitting next to you, or your manager?

G4G Martin Gardner Celebration of Mind

Tuesday 12 October 2010

In celebration of what would have been Martin Gardner's 96th birthday, the Gathering for Gardner is having parties around the world on October 21st: Celebration of Mind.

It should be interesting: local recreational math enthusiasts gathering for an evening to honor Martin Gardner. Click around on the map to find the event near you.

The Boston party is at the Brookline Senior Center, co-hosted by Eureka Puzzles.

Checking Javascript syntax in Django

Sunday 10 October 2010

Older versions of Internet Explorer have a problem with some legal Javascript syntax, namely, a comma appearing after the last element of a list:

var things = [
    "thing 1",
    "thing 2",
    ];

We happen to have a number of data tables like this that have to be maintained by hand. We are pretty good about testing real code changes in a number of browsers, but we've had problems where we "just" changed data, and skipped IE. When moving lines around in these tables, it's very easy to leave a trailing comma, and most browsers don't care. But IE completely gives up on the rest of the file, and when multiple files are concatenated and compressed together, that means giving up on much of our Javascript code, and things don't work right.

Finding problems like this is the kind of thing computers are good at, so why not let them. One idea was a checkin hook, but I figured we could detect it earlier than that. By using Django middleware, we could check the files for problems as soon as the next request to the developer's server, and the developer wouldn't be able to continue with their work until the problem was fixed.

Django middleware has a very useful feature that I've exploited a few times recently: the __init__ is run before any requests are handled, and it can raise MiddlewareNotUsed, which will remove the middleware for good after that. This makes it a good way to do something once on startup.

Here's a middleware implementation to check a set of Javascript files for this particular error:

class CheckJavascriptMiddleware(object):
    """Find annoying Javascript errors."""

    # Text that would trigger the alarm, but are actually ok.
    dont_alarm_patterns = [
        r"\{\d+,\}",        # in base.js:           {20,}
        r"\[\\\.,]",        # in jquery-1.4.2.js:   [\.,]
        ]

    def __init__(self):
        self.errors = []

        # Get the file names from the django-compress setting.
        for specs in settings.COMPRESS_JS.values():
            for filename in specs['source_filenames']:
                self.check_file(filename)

        if not self.errors:
            # No errors, so get rid of this middleware.
            raise MiddlewareNotUsed()

    def check_file(self, filename):
        js = open(filename, "r").read()
        # Remove any false alarm text.
        for pat in self.dont_alarm_patterns:
            js = re.sub(pat, "", js)
        # See if any problems remain.
        m = re.search(r",\s*[\]\}]", js)
        if m:
            self.errors.append((filename, "trailing comma"))

    def process_request(self, request):
        # This won't get called if there were no errors, 
        # because the middleware will be removed.
        html = "<h1>Please fix these errors</h1>" 
        for filename, problem in self.errors:
            html += "<p>%s in %s</p>" % (problem, filename)
        return HttpResponse(html)

Note here that we only raise the exception to remove the middleware if there were no errors. If there were, we leave the middleware in place, and obnoxiously return a "Please fix" message for every request. I figure that should be noticeable enough that the developer won't check in extra commas, or if they do, will earn the ire of every other developer, a strong deterrent for the future.

We use django-compress to concatenate and compress our Javascript files, so this middleware can simply use its settings to find the files to check. You may need to change the code to get your source file names some other way.

Of course, the Javascript "parsing" here is ridiculously simplistic, and the list of patterns not to alarm on is also simplistic, but it gets the job done for us. A smarter parser or other form of linter could be incorporated into similar machinery. Even if it is simplistic, I like the idea of my code checking my code for me.

Surprising __getattr__ recursion

Thursday 7 October 2010

A bug was filed against coverage.py this morning, and digging into it revealed a number of details about Python's inner workings. The bug boiled down to this code:

import copy

class Tricky(object):
    def __init__(self):
        self.special = ["foo"]

    def __getattr__(self, name):
        if name in self.special:
            return "yes"
        raise AttributeError()

t1 = Tricky()
assert t1.foo == "yes"
t2 = copy.copy(t1)
assert t2.foo == "yes"
print "This runs, but isn't covered."

The code runs just fine, but coverage.py claims that the last two lines aren't executed. They clearly are, because the print statement produces output during the run.

It turns out that coverage fails because there's an infinite recursion here, and when the Python interpreter unwinds the recursion, it doesn't report it to the trace function, so its bookkeeping gets out of whack.

But where's the recursion? It's well-known that you have to be careful in __getattr__ not to use an attribute that might be missing. That would cause an infinite recursion. But here, the only attribute used in __getattr__ is self.special, and that's created in __init__, so it should always be present, right?

The answer lies in how copy.copy works. When it copies an object, it doesn't invoke its __init__ method. It makes a new empty object, then copies attributes from the old to the new. In order to implement custom copying, the object can provide functions to do the copying, so the copy module looks for those attributes on the object. This naturally invokes __getattr__.

If we add a bit of logging to __getattr__ like this:

def __getattr__(self, name):
    print name
    if name in self.special:
        return "yes"
    raise AttributeError()

then we see the recursion:

foo
__getnewargs__
__getstate__
__setstate__
special
special
special
special
.. 989 more ..
special
special
special
special
foo

What's happening here is this: the copy module looks for a __setstate__ attribute, which doesn't exist, so __getattr__ is invoked. It tries to access self.special, but that doesn't exist either, because this is a newly created object which hasn't had __init__ invoked to create self.special. Because the attribute doesn't exist, __getattr__ is invoked, and the infinite recursion begins.

The Python interpreter limits the recursion to 1000 (or so) levels, but why don't we see the exception? Because the attribute access is inside the copy module's hasattr(o, "__setstate__"), and hasattr takes any exception to mean, "No, this attribute doesn't exist," returning False. So hasattr swallows the exception, and we never hear about it.

To fix the problem, we have to prevent the recursion due to looking up self.special:

def __getattr__(self, name):
    if name == "special":
        raise AttributeError()
    if name in self.special:
        return "yes"
    raise AttributeError()

Now there's no error due to reaching the recursion limit, and everything works the way it should. The moral of the story is that if you access an attribute in __getattr__, you have to defend against recursion, even if there's "no way" it could be missing from the object.

« | » Main « | »