Coalescing Python parser info?

Monday 14 November 2005This is over 17 years old. Be careful.

Last year, I wanted a parser generator for a Python program I was working on, and I went in search of a tool. As is typical in the Python world, I found many, of varying degrees of maturity, activity, and depth. To make sense of it all, I compiled a list of Python parsing tools and put it on my site.

Since then, many people have found it. About ten people a day come to this site with search queries like “python parsing”, “bison for python”, “lexical analyzer python”, and the like.

Recently, though, Robert Kieffer wrote to me, asking me to do the Right Thing:

Hi Ned,

I just ran across your python parser page. Great information!

However, I stumbled across your page only after also finding One of the commenters on your page, “Henning”, also refers to this url.

If I may be so bold, it seems like the Right Thing (tm) to do here is to transition the information on your site to the page. Since is a Wiki, it will be easier to maintain and contribute to. Also, by virtue of being on the site it has at least the appearance of being a little more legitimate.

I’d be happy to update the page, assuming you’re interested.


— rwk

I responded:

I’ve thought a lot about this very issue, and I’m torn.

On the one hand, I know that the spirit of sharing is very strong in the Python community, and it is that spirit which has made it so great. I want to contribute, and give back to the community.

On the other hand, I want (selfishly, to be honest) to be strongly associated with the work I put into my site. And I want to have more control over the content than the wiki page would afford me.

Balancing these forces is tricky. I think I will leave my parser info where it is. After all, it is freely available to the community where it is, so I am giving back.

Of course, if you want to use my page to update the wiki page, that’s fine too. I can understand why you wouldn’t want to if my page will continue on where it is.

What’s your take on these issues: do you think I am doing The Wrong Thing by not merging the pages?


Robert replied:

Hey Ned,

Thanks for the response. I have to confess that I sympathize with your dillemma. It’s tough to give freely what you’ve worked so hard on. That said, I also know that as one ponders these things the idea tends to grow on you.

I actually began a response talking about how I believe the roll of a programmer is changing, but it became so philosophical (and long-winded) that I decided to turn it into a blog entry. The essence of it is that we come from a generation of developers that find this open-sharing of code and information a little unnatural, and that it is something we will consciously have to embrace. The risk of not doing this is obsolesence.

So, are you doing “The Wrong Thing”? Well... yes.

I really, really, want to be nice and say, “no, of course not!” but I just can’t. So, yes it’s wrong, but not because it stems from selfish motivations. a) They aren’t that selfish (you are publicizing this stuff after all) and b) recognition for our effort is what motivates all of us — it’s a good thing. Rather, it’s wrong because keeping this information in two places derails the inspiration that originally led you to create that page — that is, the goal of having a single, comprehensive source for information about Python language parsers.

The fact you put this information on your website speaks to your desire to help people. You just need to go one step further and put that information in the obvious place for it. As for recognition, I think it would be fine for you to put a note at the top of the Wiki page to the effect that you compiled the information and would like to be involved (read, “coordinate”) efforts to keep it current.

— rwk

So, now I ask my readers: what do you think? I still want to leave the page where it is. People are obviously able to find it, and I can put a link on the wiki page for people who don’t. Does the spirit of sharing demand centralization? Is my desire for control at odds with the greater good? Am I an old timer who doesn’t fully get it?


His points are quite correct — your intent is to provide a useful resource for the community, and that is best done through a central Wiki location.

On the other hand, part of its value is that it is edited, not community-generated, and part of the motivation to edit it is a sense of ownership. You only maintain this by, well, maintaining it!

I would suggest that linking from the Wiki is the best option for now — you get to ‘keep’ your work, and the community gets to see it — but when you tire of maintaining it, contribute it to the Wiki. That way things don't rot.
Right thing shmight thing. If you are okay with it being copied onto the wiki, surely everybody wins.

Perhaps he was concerned about the particular style of creative-commons license you use (sharealike + restrictions).
Technically this probably means you can't cut and paste stuff onto a wiki with more liberal terms. Although by the sound of your reply you are okay with that being done. Explicit is better than implicit I guess.

Now an ethical dilemma for you: should responses to blog entries go in the comments or on the blog of the responder? :-)
Robert's full of crap. Link to your page from the wiki.

The only part that gives me pause is this line from your page:

> If you have updates to the information here, let me know.

.. which is, after all, the point of a Wiki..
As long as the information is freely available on the web, I don't think you have any obligation at all towards projects like Wikipedia. The web is not some sort of coercitive Soviet where you MUST relinquish your rights to "superior beings". Besides, your material is even published under a fairly relaxed license, so that if they want, they can just include the content taking care of giving proper attribution. That's much more than some "hackers' favourites" like Paul Graham or Joel Spolsky do, for example.

This said, what you might do is just add a link on your page pointing to Wikipedia.
It's up to you whether you make your information a page on the PythonInfo Wiki. One of my resources was just copied to the Wiki, and then I was notified after the fact by the person who did it. Now, given that I didn't want to keep adding entries to that resource, and that I had been dealing with a lot of requests to add various people's favourite/pet projects, I was quite relieved to offload it and went with the idea. To an extent, I can't really blame anyone for what happened because I didn't put explicit copyright and licensing terms on the Web page, but I still consider the act of just copying the content over without asking first somewhat impolite.

As for the merits of moving information to a Wiki, whilst the original copier actually did a good job at editing the content, more recent edits have more or less butchered the content as people's bizarre perspectives and agendas start to have an effect. Unless you start to get lots of comments or suggestions for amendments, my advice is to keep the information where it is now.
Thanks for reminding me that I had a copyright on the page. I've changed it to be in the public domain. And there already is a link to the wiki page in question, under the See Also section at the end.
Great responses, all. And, Ned, thanks for opening up the discussion - I can't wait to see what else folks have to say. Just to add grist to the mill, here are a few random observations and questions ...

Adding a link to the Python Wiki helps people discover this page (and vice versa), but doesn't answer questions such as, "Which page is more current?" and "What's the best way to request changes?" Nor does it help readers that would like to contrast the information split across the pages. E.g. Ned doesn't list the Module language on his page but does. So how do I read Ned's nice descriptions for just the C-based modules?

Commenters seem to believe a Wiki is a quagmire of random contributions, but is that really the case? My admittedly limited experience with Wikis is that people are pretty good at self-moderating, especially when a page contains instructions about how to contribute. Am I just being naive?

An unmoderated Wiki is likely to encourage casual contributions of useful information more than a moderated web page. (Right?)

What about the issue of maintaining access to this information. There will come a day when Ned decides he's fed up with us ungrateful parasites and pulls the plug on his site - probably sooner than later, thanks to me. What happens then? Is it worth worrying about? The Python Wiki doesn't have this issue.

The "ownership" point is a critical one. If Ned is the only person willing to maintain this information, and he's only willing to do that by virtue of the ownership he feels in having the page on his site (something I completely understand) than in real-world terms, that is the "Right Thing".
Do you want me to kick his ass?
Given that already has a copy of Ned's page it won't be lost forever even if Ned's site goes the way of the dodo.

Frankly I think that you should keep you page, if that is what you want to do. There is no right place for the information.

The worry that the site will disapear is just as much of a concern with the wiki. Just because it is hosted on doesn't mean it cannot disapear
Jeff: yes, I've requested updates. And that is something a wiki page is good at. But it needn't be the exclusive domain of a wiki. For example, I've had people write to me asking to add their tool to the list, and I looked at it and decided it wasn't a parser generator, it was just a language that you could write a parser in. So I'd like to get feedback, but I'm the editor of the page.

As to my long-term commitment, there's no guarantee the page will be there in a year, or in ten. Frankly, there's no guarantee the PythonInfo wiki will be there in that time, though I grant you the chances are must greater for the wiki.

But that's the nature of the web. In my page, I linked to the original author, and when his page disappeared, I grabbed a copy from and put it up on this site. All is transient, it's just a matter of the time frame.

Andrew: no thanks. Everyone else: Andrew is just kidding. You have to get used to his sense of humor...
Michael Chermside 1:06 PM on 15 Nov 2005
In my opinion (take it for what it's worth and no more), the best place for it is the wiki. Keeping it two places defeats DRY ( Keeping it on your site and referencing it from the wiki is just one more mild annoyance (an extra click) for users.

However, I don't think you're under any "obligation" to send it to the wiki. In fact, you've gone FAR beyond the call of duty by saying it's OK for the wiki to copy your list. But frankly, I think DRY applies, and the wiki should either (1) just reference your list on your site, or (2) copy your list, give credit to you, but then explicitly state that future editing should be done in the wiki and that the copy on your site is out of date. I really believe in DRY!

Frankly, I think the best option for wiki maintaners (ie, anyone who feels inspired to do this) is to just link to your page from the wiki, until the day when your page gets out of date, then "take it over". I don't think you ought to feel in the least bit guilty for trying to keep the cannonical version on your site. I just don't think it's the "best' overall solution.
The great thing about open source is open contribution, and being able to fix things when they break, and improve things if you feel you're able to.

The same thing applies for open content. If I didn't see dparser (my favourite of the parsing tools) on that page, I would scroll up and down looking for the edit link so that I could put it in.

If I weren't able to append it to the list wiki style, I'd send you an email, asking you to add it - thereby increasing the cost (in time) of maintaining your website.

Copying the whole lot to the wiki is probably a good idea, and redirecting your page to the wiki page, because when I release PyParserPlusPlusOpenEnterpriseEditition I can add it to the list, without you even having to know that I'm keeping it up to date.

Or you may want to know about all changes to the information, and don't mind spending the time adding it in. That's perfectly up to you, and completely within the spirit of the open content/source ethos.

At the end of the day, I don't care who maintains the information, provided it stays maintained. :)
In the spirit of this discussion, most of what I have to say is written in a post to my own blog.

Ned, I don't see anything wrong in keeping it on your site, but a copy on the wiki is definately a good idea. They should both link to each other, of course. (Read my link for my full views)

Giacomo, what is wrong with a socia1ist web? Isn't that really the idea of the web, which has been contorted to bring value to the producers of the content over that of the value of the consumers of said content? Perhaps we need a change to the original values and purpose of the web, now that we have more proper technology to drive it.

(Ned, your spam filter is catching the 'cia1is' in 'socia1ist' so I had to use a 1 for the l. Sorry!)
Stephen: The problem is the threshold between updated information and trusted information... That's kind of the sore point in open wikis, isn't it? I could say something like "At the end of the day, I don't care if the information is updated, provided I trust it's still correct", verify that I trust more my judgement than wiki users, and then decide I want to maintain the info myself. It's all about trade-offs.

Calvin: I've never said anything about "socia1sm", I referred to a Soviet-like interpretation where you are forced to relinquish your knowledge to an "higher being". I like to think that there's nothing forceful about the web... Sorry for nitpicking, but socia1ism as a concept is not the same as some mid-20th century russian regime :)

And I do miss good ice cream in the UK!
Calvin, "keeping a copy on both sites" simply isn't a viable option. There is too much administrative headache involved in keeping the two pages in synch. (That is the one aspect of this whole discussion that I am certain of!)

Also, I've been asking some of my coworkers about their impressions of Wikis and the reactions have been pretty consistent:
a) they're a good thing
b) they don't have any reservations about contributing, nor do they view them as being owned by any one person or group (i.e. I'm not sure if my suggeston about Ned coordinating the contributions to the Wiki would actually work)
c) They've never had a problem with a Wiki page getting too "polluted". Or, if it has, that's generally perceived as a good thing - it means good content is getting added, it just needs to be organized.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
Comment text is Markdown.