I had need for a parser written in Python, so I investigated the available Python parser generators. I found more than I thought I would (sixteen!), and my head was swimming with all the choices and differences. So I started taking organized notes. It seemed like something others would find useful, so I present it to you: Python Parsing Tools.

It's incomplete and disorganized, but I would like to keep it up to date and/or improve it. Let me know if you have changes to the information there.

Two productivity memes are swirling through the blogosphere. I like them both, but because I am a Windows user, I'm mostly an envious bystander.

• 43 Folders is a blog about productivity, especially on computers. It has tips and tricks, software recommendations, and so on. Too bad it is so Mac-centric.
• Quicksilver is a Mac OS X utility which is unusally slick and stylish even by Mac standards. It's like a keyboard wizard which does what you mean. The best quote about it I've seen is "after using Quicksilver, Macs without it seem broken".

Pete Lyons discusses the last work to do on his Pete's editor project, and begins looking around for another side project:

I will need to ponder this and come up with something new and fun to work on.

I know just where he is: Side projects (work outside of for-pay work) have always been really important to me. They provide an arena for experimentation, independence, and even frivolity that is often inappropriate in a day job. I almost always have at least one side project. Heck, this whole blog is a side project!

The best of both worlds is having a project at work that feels like a side project. My Cog code generator was like that: It was motivated by a need at work, but was designed and implemented almost as a side project. NotesPeek was the same way.

At the other extreme, Damien Katz's foray into distributed file systems is an attempt to work full-time on a side project. I'm jealous.

I think side projects are a sign of a true software craftsperson. Working on side projects requires passion for the art of writing software, either for its results or for its process. My side projects often let me learn about new technologies or techniques. My changes in career direction have been presaged by side projects that let me dip a toe in waters my day job wouldn't let me explore.

I've started a significant side project recently, but I'll wait a little longer to talk about it specifically. It's why my postings here have tapered off a bit.

A German team has built an electro-mechanical clone of Pong. The entire machine is built using 1930's-era technology, down to the telephone relays that comprise the computer. It was inspired by Conrad Zuse's original computers built with relays. They've managed to exactly reproduce the user experience of Atari Pong, but with technology that can be watched inside the case. The video on the site is very informative about exactly how they built the thing. It's impressive.

Tony Ramos runs what he calls Tony's PowerPoint Weblog, or "The Internet's first business weblog dedicated to PowerPoint, presentations and related topics." To me, Tony seems a bit clueless about the web. By formatting the entire blog as one huge bullet list, he has unintentionally mocked the entire PowerPoint culture. His tagline is overblown: are there weblogs anywhere else but on the Internet? And the disclaimer at the top of his page: "(Due to the fluid nature of the web, I can't guarantee these links will work past their original posting date.)" Yes, that's the way the web works.

Paolo Trasforini sent me a link to his Origami di Trasfo page. He made a bunch of stuff out of business card cubes, including a number of Menger sponges. I can't read the Italian, but the pictures sure are nice.

Ben Poole starts his recent blog posting The DXL buzz with "Got to love those freaky coincidences", and I know what he means. Suddenly I'm hearing more about these two creations of mine, DXL and NotesPeek.

For those not from the Notes/Domino world, DXL is an XML representation of a Notes database, and NotesPeek is an interactive viewer for Notes data. At a certain level, they are very similar: walk over a Notes database, and express everything found in an unambiguous authoritative form. For DXL, the form is XML. For NotesPeek, the form is on-screen display. Of course, DXL also allows you to perform the inverse: read XML and create Notes data (though I didn't write that half of it, Dave Schlesinger did the original code).

So here's some stuff that happened recently about DXL and NotesPeek:

• Bruce Elgort recently asked Do you DXL?
• Damien's thoughts about Domino as the perfect blogging tool ended up at building tools with DXL.
• Mac Guidera pointed me at his new tool DXLPeek, which roughly speaking, is NotesPeek reimplemented on top of DXL. It sounds very cool, and I will have to get around to trying it this week.

But the thing that happened that struck me most forcefully was the most unexpected. Recently Rob McDonagh did me a great kindness just because I wrote NotesPeek and DXL. I haven't worked on these projects in over four years. And suddenly out of the blue, a stranger reached out to me to thank me for that work. It's a great feeling to have your work acknowledged in that way. Thanks, Rob.

Here's a cool feature I just discovered: colorable console text. Download TweakUI, the venerable expose-all-the-secret-features tool from Microsoft. On the Command Prompt page is the setting Enable Selection Coloring. When enabled, you can select a chunk of text in a command window, then hit Alt+0 through Alt+9 to change the text color, or Ctrl+0 through Ctrl+9 to change the background color. The changes stick, so you can mark up command output, or any other console window. Very handy.

The National Alliance for Autism Research raises money through fundraising walks all over the country throughout the year. Nat and I will be walking this year on October 3rd. I would love it if you would sponsor me in this walk by pledging money on my fundraising page. Any amount helps. Thanks.

Here's a blurb about the organization and the event:

Walk FAR for NAAR is the signature fundraising and autism awareness event of the National Alliance for Autism Research (NAAR) and is the oldest and largest autism research walk program in the United States. Walk FAR for NAAR events have attracted over 100,000 participants to date, raising over $15 million for autism research. To date, NAAR has committed $14.9 million to fund 169 autism research projects and fellowships worldwide, more than any other non-governmental organization. NAAR has been able to make these investments in large part through the success of WALK FAR for NAAR and the volunteers who bring the event to life.

One more story from yesterday's marathon debugging session. At one point, we were using RegFind to search the registry for badly registered COM interfaces. I was working on a branch, and we thought that old executables from my trunk build might still be registered. I used a command like this:

$ regfind -y ned\trunk\bin\


to find if any stuff from my trunk build was registered in the registry. The -y flag means search case-insensitively. When nothing turned up, our Windows expert said to also search with forward slashes since .NET uses them in paths when registering. So I did this:

$ regfind -y ned/trunk/bin/


and we found some registry entries.

Wouldn't it have been cool if the -y flag also meant that "\" and "/" are equivalent? In VMS, the shell was case-insensitive, and as I recall, it considered "-" and "_" as equivalent. Case insensitivity usually only means that letters should be paired for equivalence, but why?

I haven't mentioned Sysinternals in a while, but they should be promoted forcefully and often.

Yesterday I wasted five hours trying to figure out why a C# module couldn't call a COM interface. The method was obviously properly defined, and called properly, but it always raised a No Such Method exception. I used oleview and ildasm and regasm until my eyes were crossed. I cleaned and rebuilt. I rebooted. Hours passed and I still could not figure out why it didn't work. A handful of workers were similarly baffled after wasting more time.

Finally we had the bright idea of using Filemon to monitor exactly what files were being used by the process in question. Two minutes later, it was clear that my local download assembly cache (WTF?) had a stale copy of the assembly. I executed "gacutil /cdl" to clear the cache, and the problem was gone.

Thank you, thank you, thank you Sysinternals!

After much fiddling, my RSS feeds all officially validate, and Firefox seems happy. While I was in there, I added my secret Atom feed to the mix, so now I have three feeds to choose from:

• RSS 1.0 excerpts
• RSS 1.0 full entries
• Atom 0.3 full entries

I still think these formats are all a pain. To make my RSS excerpts feed validate, I had to make it 30% larger by adding entirely redundant information to it. Yuk.

Mozilla Firefox has released its Preview Release. This includes a new feature called Live Bookmarks, which are bookmarked RSS feeds. They appear as folders full of bookmarks to the items in the feed. A cool idea, but it doesn't work for this site.

Firefox displays the orange RSS icon in the status bar on this page, and correctly finds my two RSS feeds. But using the bookmarks just displays "Live Bookmark feed failed to load." As far as I know, other RSS readers have no problems with my feeds, but there could be problems I don't hear about. I'll have to dig into it, I guess.

Update: Feed Validator says my feed is invalid. I fixed a few things, but it still wants me to add more redundant information to make the feed 100% valid. Can I just say that I hate RDF, I hate RSS, and Atom seems overly complicated as well? None of them are simple. Blah.

I was talking to a very interesting software architect the other day, and we were discussing a layer he had built on top of .NET. I asked if the .NET framework didn't already do whatever it was he was trying to do. He said,

An architect's job is to build the framework on top of the framework.

On one level, this sounds like re-inventing the wheel. But I really agree with it. In fact, it neatly encapsulates something I've been trying to understand for a long time. I've worked with people who were nominally architects, but didn't feel to me like they were "doing architecture". I couldn't put my finger on what was missing.

But now I think I see it: those non-architects weren't thinking in terms of building frameworks. I don't mean "framework" here as "a packaged API that will be offered to people you haven't met". Most software is not shipped as a framework. But many private layers that no-one will ever see should be designed as if they were public frameworks. The framework mind set forces you to clearly define semantics, isolate modules from each other, and prepare for change.

The non-architect architects I've worked with had many good skills. They were masters of many technologies, they learned quickly, they could work at many levels of the system, they could write a lot of code, they could mentor others, they cared about the quality of the product, and so on. But they were lacking the essential framework mentality, and it showed.

Yesterday, a one-hour meeting to finalize some details of another developer's feature turned into a multi-meeting all-day extravaganza that re-examined a number of central architectural issues, including modularity, data model, separation of powers, capabilities of APIs, and the nature of God.

OK, we didn't get to the nature of God. But it was exhausting, and at the end of it, a handful of architectural decisions that I really liked were in question. Now I have to put on my thinking cap and make some proposals. It means getting past the disappointment of my direction not being taken, and getting down to the key issue: what is the best design for the current set of requirements. No wonder they call it "work".

Here's one of those recurring misconceptions about XML. I've run into it a number of times, most recently from a co-worker. There's no difference between character data in XML (you know, the text that isn't part of any tag), and CDATA (the stuff inside the wacky <![CDATA[ .. ]]> tag).

» read more of: CDATA isn't special... (13 paragraphs)

The Daily WTF is a snarky collection of those pearls of bad programming we all encounter from time to time. They've got one per day, covering a broad range of real-life abominations against the coding commandments. My favorite so far? There are so many tasty delights, it's hard to know where to start. But this approach to exception handling hits home:

try {
    //...
}
catch (Exception e) {
    e = e;
}


Apparently an empty catch was warning about the unused e variable, so...!

If you want to generate a parser, there are lots of choices of software to help you. Most follow the classic yacc model: you write a grammar in a dedicated grammar language, run it through a parser generator, and end up with executable code for your parser. Newer tools run the same way: Bison, ANTLR, and Lemon.

So I was interested to find the Spirit Parser Framework, which employs a completely different technology. Grammars are written in highly stylized C++, and executed directly. The grammar syntax is unconventional, because it's all based on C++ operator overloading. Here's part of a SQL parser:

program =  +(query) ;

query
= longest_d[ select_from_clause
| select_from_where_clause ] >> SEMI;

select_from_clause
=  select_clause >> from_clause ;

select_from_where_clause
=  select_from_clause >> where_clause ;

select_clause
= SELECT >> !(DISTINCT) >> ( STAR | var_list_clause ) ;

var_list_clause
= list_p( varname >> !alias_clause, COMMA ) ;

from_clause
=  FROM >> list_p( table_name >> !alias_clause, COMMA ) ;


Significant template wizardry is at work behind the scenes, I don't claim to understand it.

We've got an aggravating problem with a C# project, and I'm hoping someone out there can point me in the right direction.

» read more of: Help with C# calling changing COM interface?... (4 paragraphs)

George Candea and Armando Fox have an interesting paper entitled Crash-Only Software. They noticed that it is faster to crash a system and reboot it than it is to shut it down cleanly and reboot it (because of the time saved not doing an orderly shutdown). And since robust systems have to be prepared to recover properly from crashes anyway, why not skip the orderly shutdown and just crash? I haven't gotten my head around all of the implications, but their logic is both counter-intuitive and compelling at the same time.