Chandler and Python: don’t blame the tools

Thursday 24 January 2008This is nearly 17 years old. Be careful.

Chandler, the uber-PIM project from the Open Source Applications Foundation, has had a major life event. Many people are calling it the end of the project, and it may well be. People are still working on it, but the funding is cut, and the future is very very unclear.

This is too bad, it was an ambitious and idealistic project, perhaps too much so. I don’t really know what was going on over there. I haven’t read Dreaming in Code, I don’t read the mailing lists, I haven’t even looked at their software in years.

Remarkably, Carlos Perez blames it on Python. His piece is really something, because he starts by admitting that he suspected from the beginning that Python wasn’t up to the task, and now he’s pretty sure his suspicions have been confirmed. Except that he doesn’t do any analysis of what actually went wrong at Chandler. He repeats his biases against dynamic languages, and concludes that they are to blame. He comes to this conclusion even as he points out how strange it is that Python is only used in the client, and the server is Java!

The consensus seems to be, and it seems right, that the problems with Chandler were:

  • Organization: too many heavy-hitters heading in their own directions. See the previous point about Python at the client and Java at the server.
  • Lack of focused target: replacing people’s email infrastructure, while at the same time re-thinking how those activities are accomplished is a huge challenge.

Carlos’ viewpoint will probably persevere. Chandler was in some ways seen as a poster child for Python, and its demise I’m sure will cause at least a minor meme along the lines of, “We can’t use Python, look what happened to Chandler.” This is too bad. I think the choice of language matters to a project, but not nearly as much as we techies often think. So many other things matter a great deal more.

Carlos points to Eclipse as an example of what can be accomplished with Java. Eclipse is an amazing achievement, but it isn’t Java that did it. There’s a huge amount of effort that goes into the focus and support that makes Eclipse what it is. As an example, have you ever read their release notes? These are amazing! No project in the history of the world has had such thorough, helpful, and well-crafted release notes. That’s got nothing to do with Java, and everything to do with the care and work that the Eclipse organization puts into producing and supporting their software.

So here’s to Chandler, a (nearly) departed friend. I hope that it continues in some meaningful form. Here’s to good craftsmen (and their kibbitzers) not blaming their tools. Here’s to using your language of choice as just one tool in your tool set. And here’s to seeing your projects for what they are: a collection of people trying hard to achieve difficult goals the best they can.

Comments

[gravatar]
No, Chandler's failure wasn't because of python. I suspect it had something todo with getting a bunch of java developers to write in python, without figuring out how to structure the project in a way that made sense with python. This post from 2004 talks about how Chandler used python badly (it is about Chandler, though the author was too tactful to explicitly say so).

I spent a good while looking at Chandler code a couple of years ago, and it was some of the most long-winded, convoluted python I've ever seen. They were trying to write java in python - deeply nested structures, thousands of getters and setters, etc. Given their background, it would probably have made more sense to write the thing in Java.

Still, I agree with your general point: the problem with Chandler was the organization, not the language.
[gravatar]
I think the main part of the failure of Chandler isn't the tools, but that it's harder to write a productivity suite than a programming editor / IDE. Look at the number of productivity suites out there for Linux, and look at the number of editors / IDEs. Writing an IDE is far easier than a productivity suite like Chandler. Look at Evolution and Sunbird for proof of how difficult it is. Calendaring, Task Lists, and Addressbooks seem like simple concepts, but there's a lot of hidden integration that goes on behind the scenes to make it happen.
[gravatar]
Ah ... i am coming from ruby, but I lately hear a lot of Java-trolls that jump on a bandwagon by flaming ruby-on-rails - and as a result suddenly, they attack ruby as language. They purposely do not give any other language as alternative - not even python, so I suspect that these are Java folks. (May be wrong, but anyone that wrote for a long time in ruby will have a hard time to understand why anyone is passionate about Java at all. Python, albeit with a different philosophy to ruby, is much more a brother to ruby than java will ever be.)

I will continue to watch out for all these posts on the blogosphere, and I hope that someone from the Java side will be able to reason these guys to stop their attacks without any base. Eclipse was successful because people put in a lot of effort. But the Java folks will probably never admit that what they can do, can be done with less verbosity just as well.
[gravatar]
Yeah - I saw that article too. I couldn't agree with you more. Here's another piece of roughage:
http://blogs.tedneward.com/2008/01/24/Can+Dynamic+Languages+Scale.aspx
[gravatar]
I agree 100% that "the choice of language [doesn't matter] nearly as much as we techies often think" -- very good point.

That said, we dynamic language weenies are reaping a bit of well-deserved blow-back over this. Java (and friends) have long been the scapegoats we've used to promote our languages and tools; DHH, for example, deliberately promotes Rails as a major improvement over Java ("no more XML sit-ups!"), and I've certainly made jabs at Java as a way of promoting Django/Python.

We can't have it both ways, unfortunately, and by blaming Java for so many failed projects we've set Python up to be blamed for this particular failure.
[gravatar]
Jacob: you are right that Pythonistas can be to blame here too. I was at a talk about Pylons, and a question came up about why it is configured with .ini files rather than .xml. One answer given was, "Because Java uses XML files". I'm not totally sure whether the speaker was serious, but even in jest, it can set up an unnecessarily oppositional environment.
[gravatar]
@Jacob: That's a very good point - I think the take away is that there's no reason to sandbag another language to prop another up: Each is a tool appropriate for a certain person/problem. We (as pythonistas) should compete on technical points objectively, instead of saying "dynamic stuff is cool cause Java has a crappy types implementation" - we should strive to be a little more expressive instead of dismissive.
[gravatar]
"Chandler was in some ways seen as a poster child for Python"

Trust me, that was *many* years ago.

"That said, we dynamic language weenies are reaping a bit of well-deserved blow-back over this."

Where? Where I am, Python acceptance and usage has *exploded* lately, with IronPython, Jython 2.2, and Django as the killer applications. One or two blog posts by self-proclaimed Java pundits doesn't an industry make. People are a lot smarter than that.
[gravatar]
FWIW, I was a curious bystander when some of the very first Chandler design decisions were made (frankly I don't even know if it was called Chandler then, it might not have had a name yet). One of those decisions was to use ZODB as a storage "backend". *Someone* made this decision, but I don't know whom, because the people in charge of implementing the Chandler database API were utterly baffled by it. ZODB is an object database, but it doesn't have SQL, doesn't have any constraints, etc (these all must be implemented in application code). But the folks doing the implementation kept trying to push these responsibilities down into ZODB itself (after all, it says its a database, and all databases do that, don't they?). They just didn't get it.

It was pretty much a train wreck, very painful to watch, and they started over basically from scratch after months of prototyping using ZODB, going IIRC to BerkeleyDB (which is itself ironic, because it has even fewer of the types of features that they were requesting than does ZODB; I don't know what they use now). This was *not* ZODB's fault, it was just a case of extremely maladjusted expectations. I'm sure this mistake wasted an immense amount of time. If the way that situation was handled was any indication of how other design decisions were being made, I'm 100% unsurprised that the project got itself into trouble.
[gravatar]
While a lot of people were pumped when Chandler was announced and it was using Python, but after that I don't think any of us tried to make it a post child. And of course Chandler is hardly the first Python project that got bogged down and didn't do well. It was all a lot more public with Chandler than other projects, but of course you could also say that Zope 2 was a post child for Python, and... well, no one really defends Zope 2 anymore. If anything I think these kinds of failures are helpful. Not that helpful for the Chandler project and its mission of course, but they are helpful for the people who went through them and those of us who watch from outside.
[gravatar]
I made it through the first half of Dreaming in Code before I got bored but if the second half was anything like the first Chandler would have foundered in almost exactly the same way with any other language. Basically it boiled down to bad project management, *very* lofty goals, too many changes in direction and infrastructure, and getting caught in the rdf semantic web quagmire. The made practically all of the classic software development mistakes.
[gravatar]
This is not the fault of the language, or any technology choice they made. The problem is almost certainly an inability to establish priorities. The clearest evidence of this is that after 6 years, they are only on v0.7.3.

I will be the first to admit that making a judgement call based on those two numbers is presumptious - especially given how little information I have about the project. But when developing a product, the toughest and most important decisions all come as a result of asking the question, "When are we going to ship v1.0? ... and what it's gonna take to get there?"

Any team that's not asking themselves this question on a more-or-less weekly basis after 6 months, let alone 6 years, is going to be in real trouble.

@Craig: An IDE more complex than a productivity suite? It's hard to say, but I think you're over-generalizing too much to make a meaningful argument. Both are "suites" in their own way, and both can be made as complex or as simple as needed based on the resources available. I've been involved in the development of several productivity suites (one Java based, one web application based) and one IDE (Objective-C/Java, a long time ago). Both were substantial efforts of roughly the same scale. if I had to point at one, I'd say the IDE was the harder of the two because of how much more sophisticated the users are and the proportionally greater expectations they have.

@Fredrik: The "blowback" is well-deserved. The software industry as a whole has been toward smaller applications that leverage a larger body of existing (mostly opensource) code. A natural byproduct of this is the a proportionate increase in dynamic languages where rapid development is naturally much easier. But if you have to write 1M or 10M lines of code that integrates with dozens of libraries written by 3rd parties, or that lots of people are going to depend on ... well, the argument for strongly typed/structured languages can be pretty compelling.

This is not to say that Java is better than Ruby or Python or JavaScript. Simply that these communities would do well to be humble about their successes and think about some of the problems that will inevitably emerge as they mature and the collective body of code grows more complex.
[gravatar]
Blaming the failure of the project on language choice is frankly absurd. I did read the book, and I installed Chandler, and it seems to me that the failure has more to do with two things, neither of which have to do with programming language -- organizational challenges, yes, but also the lack of product uptake. (That is, do you, or does anyone you know, actually use Chandler?)

Users aren't overlooking Chandler because it's written in Python, or because it "doesn't scale." They're overlooking it because it's competing with a lot of other really good products, and isn't providing a compelling-enough alternative to them. That's it. I tried using Chandler, myself, after reading Dreaming in Code (excellent book, by the way -- highly recommended), and while I was really excited about it, and wanted very much to use it, it just didn't click with me -- it didn't make sense. If it did, I'd use it. But even as a developer myself, I couldn't care less what language it's written in, as long as it works, and it does what I need. Unless OSAF was employing a bunch of Python noobs (and it wasn't), all this can really only be attributed to human factors -- organizational challenges, product-design shortcomings, and the like. The vision may have been great -- we could argue about that -- but it's tough to get around the fact that the execution was, at best, so-so. And it seems to me the end result of that execution would've looked pretty much the same in Java.
[gravatar]
Of course it's absurd to blame Python for the supposed failure of Chandler to fulfill expectations in a timely fashion; one need only take a look at the strategic directions of the project and the lack of regard for prior work for some kind of explanation of the matter. As I've mentioned previously, it seemed to me that various people involved were totally unaware of projects which already did most of what they had set out to do: existing PIM applications like Kontact, Evolution and Outlook can do peer-to-peer calendaring (hint: it's in the iCalendar standards); Kolab and other open source products already support things like calendar sharing, perhaps not in a sophisticated way, but then it's all about smart clients (just as the Chandler project was meant to be).

When people insist that they're doing something new when they aren't (although I'll accept that this happens all the time in the Python community), a dose of skepticism is required all round.

P.S. "Nøff, nøff!" if the avatar picture doesn't change when I submit this. ;-)
P.P.S. Non-ASCII characters don't seem to like the round trip through the preview...
[gravatar]
I'm glad to see so many like-minded people! This whole "language war" craziness just distracts from the real work of figuring out how to build great software against impossible odds.

And: Philip Eby (a Chandler team member) has written his take on the Chandler re-org: Rumors of Chandler's Death Are Greatly Exaggerated.
[gravatar]
@Paul: sorry about the non-ASCII mess-up. It's fixed now (I hope!)
[gravatar]
I used Chandler, and kept quiet about it because I didn't want to be seen as sniping at a worthy project (which I believe it was, and perhaps even still is). But for a project of its age it delivers remarkably little functionality and, at least on Windows, isn't particularly intuitive nor particularly usable.

Considering there were agile development proponents on the team I have to wonder what happened to "deliver early, deliver often". User feedback might have helped, and would have been easier to give with more frequent releases. I agree that it isn't helpful to characterize the problems as due to choice of language.

There does seem to have been a somewhat isolationist approach, even though many of the individuals involved are extremely communicative about other projects they are involved in. It also suffered from unreasonable expectations from the outside. Even though the team tried to squash the myth of the "Outlook-killer", that association remained, and wasn't helpful.
[gravatar]
Sounds like it was run by the government.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.