New idioms for old

Saturday 18 March 2006

I was spelunking through the Django source, and noticed two odd idioms (comment mine):

# Apply a function to every item in a list:
tables = [quote_only_if_word(t) for t in tables]

# Make a list of n None's:
l = [None for _ in range(n)]

Both of these use list comprehensions to accomplish jobs that could have been done with older Python facilities:

# Apply a function to every item in a list:
tables = map(quote_only_if_word, tables)

# Make a list of n None's:
l = [None] * n

I’m not saying they should have been done the older way. It’s interesting to see how powerful list comprehensions are that they can displace other language features. I think in this case the new way is clearer for the first statement, and the old way is clearer for the second.

But for a new Python coder, list comprehensions are one tool to understand, compared to the finer-edged but more narrowly-applicable map. So I expect to see more use of list comprehensions like this, and less use of the classic ways.


Fredrik 12:25 PM on 18 Mar 2006

Using [func() for-in] instead of map(func) is pretty much a question of personal preferences (but for simple cases like this, map can often be slightly faster).

Using [None for-in] instead of [None]*n is a really lousy idea, though; it doesn't only use more byte code instructions, you'll also end up doing 2n+2 object allocations and n list reallocations, instead of 3 object allocations. Even for n=1, it's twice as slow. For n=100, it's over 10 times slower under 2.4, and nearly 20 times slower under 2.3.

Jay P 1:25 PM on 18 Mar 2006

I rarely use map anymore for anything. There's just too much talk about it being deprecated in the future, that I've been scared away from it.

I'm surprised not only that they created the list of Nones in that slow way, but also that they used range(n) instead of xrange(n). No reason at all to use range() in this case.

Nathan 3:50 PM on 18 Mar 2006

I thought xrange was slated to be removed too.

Adrian Holovaty 4:55 PM on 19 Mar 2006

Ned, where'd you find that second example in the Django source? I'll go ahead and change it, because using simple [None]*n is much better.

Adrian Holovaty 6:46 PM on 19 Mar 2006

I've checked in those changes to Django's SVN. Thanks for the pointer!

L Clarke 7:58 AM on 28 Mar 2006

List comprehensions are one of the best Python obfuscation devices ever invented.

Here's one of my worst abuses, to print the first five squares.
[sys.stdout.write(str((lambda y: y*y)(x))+", ") for x in range(1,6)]

Should I have used xrange?

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.