« | » Main « | »

PNG optimization

Monday 31 August 2009

A simple step I often forget when building image assets for web pages is to optimize the graphics files. There are a whole host of utilities that reduce the size of images. I tried a few and the best for PNG seems to be PNGout.exe, from Ken Silverman's retro-ugly utility page.

PNGout fiddles with a bunch of degrees of freedom in the PNG spec to find the smallest sequence of bytes that express the same pixel values. This morning it reduced one of my images to 61% of its size. Nice.

If you want to understand what's going on, OptiPNG has a page explaining the details of what parameters exist in PNG that can be twiddled to acheive better compression.

If you are willing to consider art-level changes to get the absolute smallest sizes, Sergey Chikuyonok's Clever PNG Optimization Techniques gets into the nitty-gritty of how to re-think your image to take advantage of PNG's characteristics. He also has a companion piece on Clever JPEG Optimization Techniques which gets even trickier.

Weird URL data encoding

Sunday 30 August 2009

As a builder of web applications, I'm interested to see how others do it. This morning I received a promotional email from Snapfish about printing Facebook photos, and noticed the "view this email as a web page" link at the top. I figured the URL had to be unique enough to identify the campaign and the recipient, and it had to be obscure enough to prevent hacking so that I couldn't peek in on others' emails.

The typical way to do this is to include some keys in the URL, and then to also include a hash of those keys that includes a secret only the server has. The URL can be checked for authenticity, and the keys can be used to retrieve the data to display. The URL can't be hacked, because if I fiddle with the keys, the hash won't compute properly.

This is the actual URL (broken to fit):

http://email.snapfish.com/servlet/cc6?
kgHiMpkoQSUYSQSVgLKxgLKIHlJoLtKLjQJhuVaVSVupjjhjiHnLmjtVolli://
LuHptQkgHiMpkoQJhu/kLjNtLl/OLIkplL/yLjkhgHtpFLKzghju? tnapfithGze228X27X42XQWRaceboojXPSNX3zeGQTAMBILXBQ
TLQTLPQTAFFGnedzgnedbauchekdes9colGthoyGn

The odd thing here are the almost-words that appear in it: "tnapfith", "Racebooj", and "nedzgnedbauchekdes9col". That's almost "Snapfish", "Facebook", and my email address. And there's the tell-tale "://" sequence with "olli" before it, which looks like ROT-13 "http" but is not.

I understand why the URL is so long: if you can store all of the data about the message in the URL itself, then you don't need to store it on your server and then retrieve it by key when the link is clicked. But what's with the grade-school encryption going on here?

Explaining Python at DevDays Boston

Saturday 29 August 2009

I will be speaking about Python at DevDays Boston, a stackoverflow event.

Stackoverflow is a Microsoft-heavy community, so Joel Spolsky wanted someone to introduce them to Python. His idea was to do a talk explaining Peter Norvig's Python spelling corrector. It's an interesting approach: rather than start with the basics of the language, dive into a meaty piece of code, and explain as you go.

I like it, but the spelling corrector isn't the best showcase for what makes Python special. It makes good use of data structures and comprehensions to concisely complete an interesting task, but it doesn't show much more of the language.

I wanted to include another piece of code, perhaps even instead of the spelling corrector. A recent piece of code that pleased me was templite.py, a bare-bones templating class along the lines of James Tauber's Python Templates exposition. Templite adds loops into the mix, which is too much complexity, but going through some version of James' evolution I think would be a good way to show off duck typing, classes, and the power of special methods.

But I'm sure there are other ideas out there. If you had an hour to explain Pythonic awesomeness to skilled developers, how would you do it?

Plus signs in email addresses

Saturday 22 August 2009

One common complaint about "bad" email validators is that they don't allow plus signs in email addresses. Sophisticated users use plus signs to have "different" email addresses delivered to the same inbox: user+cat@gmail.com and user+dog@gmail.com both go to user@gmail.com.

It occurred to me though, that the prohibition against plus signs might not be an oversight, but an intentional attempt to prevent one user being able to register with multiple email addresses on a single site. In other words, the very characteristic of plus signs that make them appealing to geeks also make them a useful tool for spammers. As CAPTCHAs show, spammers must be thwarted even if it inconveniences some honest users.

Humane email validation

Saturday 22 August 2009

My recent work on a consumer-facing product brought up the old problem: how to validate an email address before using it? There's a classic tension here between those developers that want to prevent typos from floundering around in the system, giving users feedback as soon as possible that it seems like they've made a mistake; and those developers that want to be sure that any valid email address can be used.

The usual advice on this matter is to not bother with validation, because it's a fool's errand, instead simply send an email to the address with a confirmation link. If the user clicks the link, then the address must have been valid.

I don't like this advice because the vast, vast majority of email addresses do validate with a simple regex, and the vast, vast majority of failures against the regex represent real mistakes, not obscure but valid email addresses. Catching user mistakes early is a good thing. Having the user wait for an email that will never come, then go back to enter their email address again is a pain.

This is the regex I used:

/^[^@ ]+@[^@ ]+\.[^@ ]+$/

In other words, an email address has to have stuff, at-sign, stuff, dot, stuff. The stuff can have dots in it, but can't have at-signs or spaces. And by the way, before matching against the regex, trim whitespace from the ends of the address.

As a gesture of reconciliation with the purists, I propose this: check the user-entered email address against this regex. If it matches, it's valid. If it doesn't match, show the user an "invalid email address" error box that has two buttons: "Fix mistake" which lets the user re-enter an email address, and "Use it anyway" which takes the email address as-is even though it failed the match.

This is the best of both worlds, since the common case of a catchable typo in an email address will force the user to double-check their entry, but any address can be used if the user knows what they are doing. Most users will never see the error box, since they'll enter their address correctly.

I've never seen a work flow like this, but it seems like a really simple solution to the problem. Is there something I'm over-looking? Is it too geeky?

Eunice Kennedy Shriver, 1921–2009

Tuesday 11 August 2009

Eunice Kennedy Shriver died this morning. She was known for two things: being JFK's sister, and founding the Special Olympics. The first was an accident of birth, the second a triumph of will.

These days, Special Olympics has become such a commonplace part of American (and global) culture, that it's hard to imagine the time before it. But that was a very different time, when institutionalization was the norm, when disabled children were not seen in public activities, much less exalted for their achievements. While Special Olympics seems obvious now, it was unthinkable 50 years ago.

Kennedy understood something that no one else did. Her revolutionary insight was that disabled children can accomplish things, that they could be acclaimed for the things they could do rather than hidden for the things they couldn't. She saw that no less than any other child, these children could be given a forum in which to compete and excel.

I've written many times about Special Olympics, about the incredibly supportive crowds, about the unusual feeling of belonging. None of this existed before Eunice Kennedy, and likely would not exist still were it not for her. She had an incredible drive to create a place in the world for disabled people, and did something not many others could have.

She started Special Olympics as a camp in her backyard, personally coaching and exhorting the first athletes. Her one-on-one style has trickled down through the organization to the coaches and volunteers who work with my son and all the athletes like him. Her spirit has infused the organization so thoroughly, it is hard to imagine Special Olympics without her.

I met Eunice Kennedy once, at a dinner in honor of her birthday, and that is a special occasion I will always remember. But I felt like I already knew her. The very concept of Special Olympics is so strong an expression of her passions that actually meeting her seemed almost redundant. I knew her through her work, and more importantly, through the hundreds of coaches, volunteers, parents and athletes I've met in Special Olympics who were set in motion by her.

Kennedy will be missed, but she has created something that will outlast her, because her insight, though revolutionary, is right.

Super awesome cheap ball

Saturday 8 August 2009

On a road trip a few months back, we stopped at a gas station on the Mass Pike (the Massachusetts Turnpike for you out-of-staters), and were just idly looking over the cheap junk they had for sale there. A interestingly geometric ball caught my eye. It was green, and had a strange structure that seemed like it would move in some way. I tried pulling gently at its panels but nothing happened.

Then I tossed it lightly and caught it: now it was orange! Look:

Even after you've tried it a few times, it's still a bit startling to see a sphere turn inside out and still be a sphere.

This ball entertained us for the whole weekend, and since it had only cost me $4 at a gas station, we dubbed it Super Awesome Cheap Ball, in homage to Super Happy Fun Ball.

A later stop at the gas station unearthed the name of the ball, and its creator: Switch Pitch, by Chuck Hoberman, the creator of the familiar Hoberman Sphere.

Mysterium 2009

Wednesday 5 August 2009

Over the weekend, Max and I traveled to Spokane for Mysterium 2009, the annual gathering of the die-hard fans of Myst and its sequels. I've played a number of the games, but was really just along as chaperone (I joked that I was there because "I know how to work the credit card"). In fact, I played Myst when Max was just a baby, but he has now at 17 greatly outstripped me in both interest and expertise.

The convention itself consisted of about 50 nerds from across the country, spending three days immersed in all things Myst. Max was in heaven, and it was great to see him in his element. The attendees were remarkably diverse, with a close to 50/50 split male/female, a number of families, and a woman who had 4 great-grandchildren.

Also in the hotel was a regular meeting of the local bridge club, and it was interesting to see the two contingents, each obsessed in their own way with their own game. I was also surprised how many of the civilians, on hearing the con was about Myst said, "I remember Myst, I played that game."

The highlight of the weekend was Saturday afternoon when we visited Cyan Worlds, the company that wrote all the Myst games. Rand Miller is the founder, and still leads the company (he also played roles in the games). He was incredibly good-natured, excitedly greeting the fans, some as if they were old friends. He gave us a tour of the building, which had been built with Myst money, and in its spirit. For example, the elevator for the two-story building has many extra buttons labeled with D'ni numerals.

Rand brought out some props from the games, which the fans of course treated as holy relics. One of the linking books, on closer inspection, turned out to be a Webster's dictionary.

One unfortunate note: the place was mostly empty. It was very clear that the company is practically shut down. A number of ex-employees had come back for the afternoon. In fact, a number of them had brought their children. I was touched both by the emptiness, and the loyalty of the fans and workers. I felt like I was at a family re-union.

The weekend as a whole was way more Myst than I wanted. I spent a lot of time by myself hacking on various projects. But I was really glad to be able to get Max there, and to see Cyan Worlds in person.

Of course, Max has his own summary, including links to photos and video.

« | » Main « | »