I wrote a low-tech terminal-based version of the classic 2048 game and had some interesting difficulties with iterators along the way.

2048 has a 4×4 grid with sliding tiles. Because the tiles can slide left or right and up or down, sometimes we want to loop over the rows and columns from 0 to 3, and sometimes from 3 to 0. My first attempt looked like this:

N = 4
if sliding_right:
    cols = range(N-1, -1, -1)   # 3 2 1 0
else:
    cols = range(N)             # 0 1 2 3

if sliding_down:
    rows = range(N-1, -1, -1)   # 3 2 1 0
else:
    rows = range(N)             # 0 1 2 3

for row in rows:
    for col in cols:
        ...

This worked, but those counting-down ranges are ugly. Let’s make it nicer:

cols = range(N)                 # 0 1 2 3
if sliding_right:
    cols = reversed(cols)       # 3 2 1 0

rows = range(N)                 # 0 1 2 3
if sliding_down:
    rows = reversed(rows)       # 3 2 1 0

for row in rows:
    for col in cols:
        ...

Looks cleaner, but it doesn’t work! Can you see why? It took me a bit of debugging to see the light.

range() produces an iterable: something that can be iterated over. Similar but different is that reversed() produces an iterator: something that is already iterating. Some iterables (like ranges) can be used more than once, creating a new iterator each time. But once an iterator like reversed() has been consumed, it is done. Iterating it again will produce no values.

If “iterable” vs “iterator” is already confusing here’s a quick definition: an iterable is something that can be iterated, that can produce values in a particular order. An iterator tracks the state of an iteration in progress. An analogy: the pages of a book are iterable; a bookmark is an iterator. The English hints at it: an iter-able is able to be iterated at some point, an iterator is actively iterating.

The outer loop of my double loop was iterating only once over the rows, so the row iteration was fine whether it was going forward or backward. But the columns were being iterated again for each row. If the columns were going forward, they were a range, a reusable iterable, and everything worked fine.

But if the columns were meant to go backward, they were a one-use-only iterator made by reversed(). The first row would get all the columns, but the other rows would try to iterate using a fully consumed iterator and get nothing.

The simple fix was to use list() to turn my iterator into a reusable iterable:

cols = list(reversed(cols))

The code was slightly less nice, but it worked. An even better fix was to change my doubly nested loop into a single loop:

for row, col in itertools.product(rows, cols):

That also takes care of the original iterator/iterable problem, so I can get rid of that first fix:

cols = range(N)
if sliding_right:
    cols = reversed(cols)

rows = range(N)
if sliding_down:
    rows = reversed(rows)

for row, col in itertools.product(rows, cols):
    ...

Once I had this working, I wondered why product() solved the iterator/iterable problem. The docs have a sample Python implementation that shows why: internally, product() is doing just what my list() call did: it makes an explicit iterable from each of the iterables it was passed, then picks values from them to make the pairs. This lets product() accept iterators (like my reversed range) rather than forcing the caller to always pass iterables.

If your head is spinning from all this iterable / iterator / iteration talk, I don’t blame you. Just now I said, “it makes an explicit iterable from each of the iterables it was passed.” How does that make sense? Well, an iterator is an iterable. So product() can take either a reusable iterable (like a range or a list) or it can take a use-once iterator (like a reversed range). Either way, it populates its own reusable iterables internally.

Python’s iteration features are powerful but sometimes require careful thinking to get right. Don’t overlook the tools in itertools, and mind your iterators and iterables!

•    •    •

Some more notes:

1: Another way to reverse a range: you can slice them!

>>> range(4)
range(0, 4)
>>> range(4)[::-1]
range(3, -1, -1)
>>> reversed(range(4))
<range_iterator object at 0x10307cba0>

It didn’t occur to me to reverse-slice the range, since reversed is right there, but the slice gives you a new reusable range object while reversing the range gives you a use-once iterator.

2: Why did product() explicitly store the values it would need but reversed did not? Two reasons: first, reversed() depends on the __reversed__ dunder method, so it’s up to the original object to decide how to implement it. Ranges know how to produce their values in backward order, so they don’t need to store them all. Second, product() is going to need to use the values from each iterable many times and can’t depend on the iterables being reusable.