« | » Main « | »

Structure Synth and Sunflow

Wednesday 31 December 2008

Structure Synth is a simple language for writing recursive scripts that can generate complex structures. Think of it like Processing, but in 3D, although it's technically more of an L-system. Sunflow is a rendering engine that excels at realistic lighting.

Put the two together and you get landmarks from the intersection of Escher and the Matrix:

Escher's Collapse

Box Pyramid

Menger Crates

It all puts me in the mindset of tinkering with the stellated dodecahedron logo on this site...

PyCon double-talk

Tuesday 30 December 2008

It seems a number of people are giving two talks at PyCon 2009 (André Roberge, Jesse Noller, Tarek Ziadé, and Mike Fletcher at least), and I can add my name to that list as well: both of my proposals were accepted.

The first is a straightforward talk about code coverage testing, and coverage.py in particular. I'm hoping to have some major new features in place in time for the talk. Maybe the conference as a deadline will help get it done.

The second talk is more ambitious: how to write a C extension, in 45 minutes. Here, I've got a few options how to proceed. One possibility is to go slowly through however much of the topic as will fit in 45 minutes, making sure it is understandable and people come away with a solid footing in the beginnings of the topic. The other way to go is to blaze through as much as I can, firehosing information on the theory that people will be able to go back and read about it later, and it will be familiar to them for having heard it once from me.

Personally I prefer the second style of talk. I learn the general lay of the land, get a quick 10,000-foot overview of the landscape, hear about places to go back to learn more in depth, and so on. But others could feel battered by the rapid-fire barrage of facts with little to grab onto.

Either way, there's no way to cover even a fraction of the complex topic of writing C extensions. My main goal is to de-mystify extensions so that people coming into the talk feeling like C extensions are deep magic leave the talk feeling like extensions are within the realm of possibility.

I don't know if previous PyCons had a similar number of double speakers. If not, what does it say about this PyCon?

HP hibernation

Sunday 28 December 2008

We're about half-way through Hewlett-Packard's two-week holiday shutdown. I like the time off: it's an extended period to get away from the usual stresses of the work day and commute. The boys are home too, so everyone can relax their schedules and find new things to do with each other.

But HP's policy is that the time is either vacation or unpaid leave. My group isn't too strict on tracking vacation time. But if it were, I would be royally pissed at the company deciding I was taking my vacation in the winter instead of the summer. At previous jobs, I've always found the quiet time between Christmas and New Year's to be very productive, I don't doubt that it would have been here as well.

Especially irksome about the whole process is the email from the head of HR announcing the closure. After a few paragraphs of details and legalese, it concluded:

Shutting down during a period when many employees traditionally take vacation helps HP achieve significant operational savings and allows employees to enjoy more time with their families.

I find this happy-spin condescending (and repetitive: if you read the Globe story linked above, it contains the same sentence, uttered by a company spokesmodel). Whoever wrote this is probably a nice person. But if they had to deliver the closure news to a small group face-to-face, I doubt very much that they would have done it this way. They would have had to acknowledge that from the employee's point of view, the closure might not be a good thing. Once the message gets put into a company-wide email from a senior vice-president, though, it turns into this sort of corporate zombie-speak. Why?

I understand a two-week closure is better than laying off more employees. I understand the company needs to take unusual measures in unusual times. But don't treat me like an idiot. Why not say,

We understand this might inconvenience employees. We realize you may have wanted to spend your vacation time in the summer. We apologize. These are hard times, and we've had to make some hard choices.

Wrapped up in all this is the corporate model of earning vacation time that then becomes a liability on the company books until it is paid out. My understanding it that IBM doesn't track vacation time at all, releasing it from the fretting over how much is being accrued and how to get employees to use it or lose it.

Referrers in iframes

Tuesday 23 December 2008

Yet another in a recent spate of browser discoveries, this one has to do with displaying pages in iframes, and how referrers work with them.

Here's the quiz: suppose page A displays page B in an iframe. What is the referrer for page B? The answer is simple: page A. Now suppose that B contains a link to page C. Click the link, and the iframe now shows page C. What referrer is used for C?

The answer isn't so simple. Turns out Firefox 3 and Safari 3 both report that C's referrer is B, while IE 6 reports that it is A! I don't have IE 7 or 8 at the moment to try.

BTW: all these niggling browser behavior posts are the result of trying to hire a web front-end developer, and me having to help out while we look...

Newgrange solstice

Monday 22 December 2008

Yesterday was the winter solstice, the shortest day and longest night of the year. I did nothing to mark the event (other than spend some of the long night outside shoveling).

Newgrange, the Irish neolithic passage tomb, is constructed so that the rising sun of the solstice penetrates 60 feet down the internal passage to illuminate the tomb chamber. Heritage Ireland has an hour-long video of the sun-rise. Much of it is filler, and the 2008 weather was too cloudy for the sun to shine inside the monument, but there's shots of last years' solstice showing the sun from inside.

Newgrange is both more elaborate than Stonehenge, and 1000 years older than it. It's also older than the pyramids at Giza. I'm surprised that I've never heard of it before. It's a remarkable structure.

More information about Newgrange:

Snow rules

Monday 22 December 2008

We've had quite a start to the snow season this winter, and I'm reminded of snow rules, sometimes the hard way:

  • Lift with your knees.
  • When shoveling sidewalks, don't forget to shovel cuts at the corners so people can get onto the sidewalk from the street.
  • Find the drain nearest your driveway and also shovel it clear so melting snow has somewhere to go.
  • When you take off your snowy outerwear, brush off the snow so it doesn't soak through as it melts.
  • When you take off your boots, put the laces inside the boots so they aren't wet when you need to go out again in four hours to shovel some more.
  • When you clear off your car, don't forget your headlights. The backlit snow dome effect is charming, but not very safe!
  • Also, get the foot of snow off your car roof. It seems stuck there pretty good now, but soon you'll hit the brakes and it will all end up on your windshield.

Accidental HTML entities in URLs

Friday 19 December 2008

Here's yet another browser mystery which seems to affect Internet Explorer, though after the last one (about conditional definition of JavaScript functions) turned out to be maybe the correct behavior, I'm reluctant to lay the blame at IE's feet.

We had a problem where certain links in IE were producing mysterious stack traces about not being able to convert characters somewhere deep in the HTTP handling code. Turns out we had a URL like this:


and by the time it got to the servers, it looked like this:


Here's the odd thing: the URL wasn't just an HTML attribute. If the HTML had looked like this:

<a href='http://nedbatchelder.com?foo=1&copy_from=quux'>go</a>

then I could understand the problem: HTML needs to have ampersands escaped. &copy; is an HTML entity for the copyright character, so the URL string really does have a copyright symbol: In this case, HTML doesn't care that the trailing semicolon is omitted, and as an extra twist, the underscore doesn't count as a word character in HTML, so &copy_from becomes ©_from.

To use this URL in HTML, you'd have to escape the ampersand to avoid the entity conversion, like this:

<a href='http://nedbatchelder.com?foo=1&amp;copy_from=quux'>go</a>

But in my case, the URL wasn't in HTML, it was in a JavaScript string, like this:

document.location = "http://nedbatchelder.com?foo=1&copy_from=quux";

In this case, the ampersand ends up in the string as a plain ampersand. But when the time comes to set the document location, IE applies another entity replacement, while other browsers don't. In IE 6, this code results in a visit to a URL with a copyright symbol in it.

If you'd like to try for yourself, here's a small HTML fragment to try:

function try_1() {
    // A simple URL string with &copy in it:
    var url = 'http://nedbatchelder.com?foo=1&copy_from=2';
    document.location = url;
function try_2() {
    // Try to avoid entity replacement by breaking up the string:
    var url = 'http://nedbatchelder.com?foo=1&' + 'copy_from=2';
    document.location = url;
<p><a href='http://nedbatchelder.com?foo=1&bar=2'>bar</a></p>
<p><a href='http://nedbatchelder.com?foo=1&copy_from=2'>copy</a></p>
<p><a href='http://nedbatchelder.com?foo=1&amp;copy_from=2'>copy</a></p>
<p><a href='javascript:try_1()'>js try 1</a></p>
<p><a href='javascript:try_2()'>js try 2</a></p>

When you click either of the JavaScript links, an alert will appear to show the URL you're about to visit, so you can see that the string really is "&copy_from=2". In Firefox or Safari, you end up at the URL shown in the alert. In IE, you see "&copy_from=2" in the alert box, but the URL you visit is "©_from=2".

One of the tricky aspects of any kind of programming, but web programming in particular is understand all the different phases of interpretation your code and data is subjected to. Where exactly is &copy; turned into ©? When the browsers don't agree, and subtle manipulations are being applied where you don't expect, the debugging gets that much trickier.

Once this problem was identified, it was easy to fix the problem: change the parameter name from &copy_from to &copyfrom, so that it doesn't match an HTML entity. Unfortunately, this means we can never use a parameter name that conflicts with an HTML entity name, and there are 252 HTML entities, including ones with potentially useful names like &lang (〈) and &times (×).

I couldn't find any discussion of this issue in a quick search, other than this blog post about HTML escaping.

Genetic image evolution

Saturday 13 December 2008

Roger Alsing has made a fascinating image hack: reproduce an image with randomly-placed translucent polygons: Genetic Programming: Evolution of Mona Lisa. At each step, a random polygon is placed on the image, and a simple image comparison function determines whether a new proposed polygon makes it look more or less like the goal image. It's not fast, but the results are kind of startling. A photo of me and a friend:

Matt and Ned

And a polygonal reproduction with 108 polygons:

Matt and Ned evolved with polygons

This reminds me a little of the CSS Homer, recreating an image by amassing unlikely components in just the right way. BTW: the comments on Roger's post contain a number of AI nerds debating whether this is truly genetic programming, and if so what kind, or if it is simply a hill-climbing algorithm. In any case, it's very cool.

Weather forecasting

Friday 12 December 2008

Living in New England, and working far from home, predicting winter weather is very important. It really sucks to have an hour commute home turn into a three-hour drive because of a snow storm. At Tabblo, we used to have a weather fanatic who would give us detailed forecasts and help us avoid the bad days. Without him, we have to do our own predictions, and we're still learning the ropes. We almost got into a freezing rain situation yesterday which luckily didn't materialize.

If you think weather.com has detailed reports, you need to take the red pill and see how far the rabbit hole goes. The National Weather Service has incredibly detailed hour-by-hour charts for your zip code. Navigating is a bit rough though: enter your zip code in the box on the left, you get to a page about the area. Near the bottom right is an Hourly Weather Graph link. You should get to a page that looks like this one for Marlborough MA.

It was this graph that convinced me yesterday that although it was going to be close, the rain wouldn't actually freeze, and it didn't, it was an uneventful drive home. Thanks federal tax dollars!

Internet explorer mystery #1376

Tuesday 2 December 2008

Digging into a mysterious failure today at work, I discovered an odd factoid about the way Internet Explorer interprets Javascript.

Here's a small sample HTML file with a chunk of Javascript:

/* When are functions defined? */
function really() { alert("Original"); }
if (0) {
    function really() { alert("Yes, really"); }

What alerts will display when you load this page? In Firefox, you'll see "Original". In Internet Explorer 6 (and I think 7), you'll see "Yes, really". In neither browser will you see "No".

In IE, the redefinition of really() is interpreted and used even though it is inside a block of code that is not executed. Wonder of wonders...

You can fix it by changing the redefinition clause to:

if (0) {
    var really = function() { alert("Yes, really!"); }

Oh, Internet Explorer, how can you be right when you feel so wrong?

« | » Main « | »