For/else

Tuesday 18 October 2011 — This is 13 years old. Be careful.

Python has an unusual appendix to its looping structures: a “for” or “while” loop can also have an “else” clause. It took me a while to get it, but now it makes complete sense to me. Let me explain.

The classic (trivial) description of for/else is this: the “else” clause is executed if the loop completes normally, but not if it was interrupted with a “break”. This explanation is technically correct, but is also confusing. It seems like “else” should instead be “and also”, since the else clause isn’t an alternative to the loop, it’s added on, but only if the entire loop completed. So it seems like the wrong word. One of the great things about Python is Guido’s insistence on usability of the language as a guiding principle in its design. There should be a user-centric explanation that makes sense.

Consider a common structure of a “for” loop:

for thing in container:
    if something_about(thing):
        # Found it!
        do_something(thing)
        break
else:
    # Didn't find it..
    no_such_thing()

Here you can see something interesting: there’s an “if”, and there’s an “else”. Those two words often go together, but not the way they are used here. But the pairing is important, and is the whole reason for the “else” keyword. When this loop is run, the “if” will be evaluated a number of times, and if it is ever true, the loop will break. A “break” means the “else” isn’t executed. Only if the “break” isn’t taken, that is, if the “if” is always false, will the “else” clause be run.

This is where the “else” keyword makes total sense. If you focus on the “if” statement, it is evaluated many times, and only if it is always false, will the “else” clause of the loop run. As the loop executes, the “if” becomes like a giant if/elif/elif ladder: each time around the loop adds another test of the condition, but only if all previous tests were false. The “else” clause caps it all off, just as it would in an explicit if/elif/else structure. And it works just like the explicit structure: if all of the conditions are false, then the “else” clause is run.

I think of the code above like this (informally):

if any(something_about(thing) for thing in container):
    do_something(that_thing)
else:
    no_such_thing()

This is the explanation that makes sense to me, because the “else” in my conceptual code is just like the “else” in the actual code. Maybe this isn’t a trivial explanation, but it’s one that makes sense to me, and I hope, to others.

Comments

Nick Coghlan 10:18 PM on 18 Oct 2011

I actually explain the while loop case first rather than trying to start directly with for loops.

Consider:

  while condition:
    # loop body

Let's use 'break' to separate the loop structure from the condition test:

  while 1:
    if condition:
      # loop body
    else:
      break # Terminate the loop if the condition is false

Now let's do something in that second branch before we terminate the loop:

  while 1:
    if condition:
      # loop body
    else:
      # Run this code if the condition is false
      break # Terminate the loop

And then merge the loop and the condition check back together while keeping the contents of the else clause:

  while condition:
    # loop body
  else:
    # Run this code if the condition is false

The else clause is the "other branch" of the implicit if statement in a while loop. The implicit if statement in a for loop is significantly less obvious, but it's still there, and so that construct inherits the else clause as well.

The reason break, return and exceptions bypass a loop's else clause is because they terminate the loop from a point that's already on the other branch of the implicit "do I go around again?" check.

Julian 11:34 PM on 18 Oct 2011

I do understand the for/else construct now, but I did find it non-intuitive.

Let me explain my initial confusion, as I think it will help your explanations.

The first time I saw it, I assumed it was a short-cut for this:

# Reminder: This is the WRONG interpretation.
if condition:
   while condition:
      # loop body
else:
   # deal with the situation that the loop never ran.

This remains to me the most intuitive way of interpreting an unusual construct, and I still sometimes need to mentally correct myself when I use it.

The reason I bring this up is this: Consider the comment and identifier in the else clause in your first for-loop example:

    # Didn't find it..
    no_such_thing()

These are a little ambiguous. They don't distinguish between not finding it because (a) the list to search was empty, or (b) it wasn't a member the (possibly empty) list. The second interpretation is the correct one, but the first one could still be taken by someone who hasn't yet got it.

I suggest you choose examples that couldn't be misinterpreted this way to help people like the old me.

Eli Bendersky 3:30 AM on 19 Oct 2011

I don't think this construct is particularly hard to understand, but it's just a tad over the "non-intuitive" line, which makes me avoid it completely. I just found that:

1. Whenever I see it, I usually require an extra mental cycle to remember what it does
2. Then newbie programmers see it, they usually don't understand it at all and have to read the tutorial or other explanations

As such IMHO this construct violates the Principle of Least Astonishment. Not something that Python tends to do much...

Ian Phillips 3:53 AM on 19 Oct 2011

I didn't know about this feature, and you're right, the classic/trivial description is not at all intuitive, but I think that your first code snippet explains it wonderfully.

In fact, taking the text from "The classic (trivial) description" to "So it seems like the wrong word." and then showing the code snippet seems complete, and it has that "huh? - aha!" combo that makes it seem like a rewarding discovery to the reader/student.

Ed Davies 5:25 AM on 19 Oct 2011

For the record, I've never used the "else" form because I couldn't remember which of the two interpretations Julian mentions is correct. Alex Martelli's Nutshell book is perfectly clear on the subject yet still the "else clause executed if zero iterations" version seemed natural enough to me that I'd have had to go look if I'd wanted to be sure.

Michael Kohne 6:09 AM on 19 Oct 2011

From a python newb: Your post and example seem like reaching for a reasonable explanation for a bad choice of keyword. It's not a hard feature to understand, but the else keyword really mucks up the issue.

Some advice from the C/C++ trenches, where bad use of keywords and operator overloads that don't make any bloody sense reign supreme: Don't spend too much time trying to rationalize a mis-choice of keyword. If you can't fix it, avoid it. If you can't avoid it, comment copiously.

Ned Batchelder 6:30 AM on 19 Oct 2011

The "else means the loop was empty" is the other "intuitive" meaning, and is the one most people would probably guess if given no guidance. This is bolstered by the fact that in the Django template language, the {% for %} tag has an {% else %} clause that means exactly that!

André Roberge 6:48 AM on 19 Oct 2011

This is actually on my list of five small pet peeves with Python, where I wish a different choice would have been made - in this case, the introduction of a new keyword: nobreak. (Among the others would be using "function" instead of "lambda" as a keyword, and using something like ##[ multiline comment ]## in addition to triple quoted strings, so that editor could identify multiline comments...)

Steve Holden 7:39 AM on 19 Oct 2011

You're right about my feeling that "else" was a stretch for the purpose, and I still do. Given the amount of Python code in existence, however, I am not proposing a change ;-). I doubt it's ging to make anyone stay away from Python (though it remains unfortunate that a poor choice of keyword keeps some people from using a useful construct).

Steve Ferg 8:05 AM on 19 Oct 2011

I think the key to understanding for..else is to understand the use case that it is intended to address (rather than just understanding the way it works). Your post did a great job of that. So I would only like to add a little icing to the cake.

In other languages, I've often written code like this.

#-------------------------start----------------------------------
found = False
for item in container:
    if isGood(item):
        handleGoodItem(item)
        found = True
        break

if not found:
    raise SomeKindOfError("I could not find a good item.")
#--------------------------end-----------------------------------

the "else" on a for loop allows this to be coded more compactly as:

#-------------------------start----------------------------------
for item in container:
    if isGood(item):
        handleGoodItem(item)
        break
else:
    raise SomeKindOfError("I could not find a good item.")
#--------------------------end-----------------------------------

So the else clause on the loop allows you to avoid defining and then testing a "found" flag.

Nate 8:19 AM on 19 Oct 2011

This is one of those bad language design choices that I think it's best to avoid. It doesn't gain you anything you couldn't do with normal code, and the potential confusion is not worth saving a few lines.

for foo in bar:
    if pass_test(foo):
        break
else:
    do_no_pass()

Is the above really so much better than the below? You save two lines, big deal. The code below is much more clear about its meaning.

found = false
for foo in bar:
    if pass_test(foo):
        found = true
        break
if !found:
    do_no_pass()

I see no reason I'd ever use for/else. Never. Clarity is too important.

Even if it were not so unclear, the "found" flag is sometimes useful further down in the code, so I might end up using that anyway.

Nate 8:20 AM on 19 Oct 2011

Oh, also, god forbid someone comes back to your code and thinks you screwed up the indent on your do_no_pass code and "fixes it":

for foo in bar:
    if pass_test(foo):
        break
    else:
        do_no_pass()

yeesh.

Steve Ferg 9:07 AM on 19 Oct 2011

A reasonable question might be: why did Guido use the word "else" for this particular language construct?

I think it must have been on analogy with try/except/else, where the "else" clause is the alternative to the "except" clause.

try:
    ...
except: # executes if an exeption was raised
    ...
else:   # executes if no exeption was raised
    ...

A completely analogous construct for loops (the "for" or "while" loop) would have looked something like this, with a "break" statement being treated as if it raised a kind of exception.

for...
    ...
broken:  # executes if a  break was raised
    ...
else:    # executes if no break was raised
    ...

In this construct, the "else" clause is the alternative to the "broken" clause.

Actually, I sort of like the idea of a "broken" keyword. It would make looping constructs symmetrical with try/except/else, and it would make it obvious what the "else" in looping constructs is an alternative to.

Maybe we need a PEP to propose the addition of a "broken" keyword. :-)

Kevin Edwards 11:34 AM on 19 Oct 2011

As everyone has said, it is bad design because the "else" does not apply to the for -- it applies to the breaks.

for/while's "else" should have been called "if_no_break" and that's probably how it should be explained.

jeremy avnet .:. brainsik 12:22 PM on 19 Oct 2011

To find out how confusing this construct is, a few of us took a survey at PyCon 2011. The results tell me to never use it in code I want others to read:

https://twitpic.com/4a52sh

Pekka Klärck 1:03 PM on 19 Oct 2011

As Steve Ferg explained, the main use case for for/else is removing some boilerplate code when you loop over items, try to find something, and want to have a special action when no match was found. Personally I don't think that's such a good use case, though, as you can save even more boilerplate by extracting the logic to a helper function and using return:

    for item in container:
        if isGood(item):
            handleGoodItem(item)
            return
    raise SomeKindOfError("I could not find a good item.")

When I saw for/else for the first time I also though the else branch would be executed when the container is empty. That felt both logical and very useful. Such a shame the construct cannot be changed at this point without too much work.

Steve Ferg 2:52 PM on 19 Oct 2011

I agree that in a "for..else" loop, it would be more intuitive if the "else" was executed when the container is empty.

Remember that Python provides not only a "for..else" looping construct, but also a "while..else" construct. Execution of the body of a "while" loop is determined by a loop condition, not by the contents of a container. And when we use "for" to iterate over the results returned by a generator, what we're doing is as much like testing a loop condition (while generator_function returns a result...) as it is like processing the items in a container.

So I don't think that we can specify our intuitive behavior of "else" in terms of the emptiness or non-emptiness of containers. We probably need to specify it this way: in a looping construct, an "else" clause executes if and only if the loop body is never entered. Or: iff the loop condition never evaluates to True.

In the case of a "while" loop, for example, that would mean:

    while someCondition:
        ...
    else:   
        # executes if someCondition is not initially True, 
        # and the loop body is never entered
        ...

Actually, now that I think about it, that kind of behavior would be pretty nifty!

Jeff 8:28 PM on 19 Oct 2011

I'm not sure what the 'empty container' comments (@Steve/@Pekka) are about. For me, it works as expected, iterating an empty container will trigger the else because you never visited a break inside the loop

Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> j = 6
>>> for i in ():
...     if (i==j): break
... else:
...     print j,"not found in empty list"
...
6 not found in empty list

Julian 9:07 PM on 19 Oct 2011

@Jeff, your example demonstrates precisely the confusion to be avoided. It suggests that the else part is only run if the list is empty, which is not the case.

Consider what happens if the list is not empty:

>>> j = 6
>>> for i in ("this", "is", "NOT", "empty"):
...     if (i==j): break
... else:
...     print j,"not found in empty list"
...     # The above message is NOT appropriate.
...
6 not found in empty list

Adam 9:26 PM on 19 Oct 2011

Usually Python does the right thing but this construct needs some work. I would not use this in it's current form.

Using else as the keyword I agree with others that I would expect the else to be triggered based on an empty list.

If this construct is to be made intuitive "else" should be renamed to "last" perhaps...

Ned Batchelder 6:17 AM on 20 Oct 2011

To the question of whether code is more readable without the for/else construct: to some extent it depends what you are used to. I was explaining "enumerate" once, and said that instead of writing this:

i = 0
for x in thing:
    do_something(i, x)
    i += 1

you could just write

for i, x in enumerate(thing):
    do_something(i, x)

I asserted that the latter was more readable. My student claimed actually the first was more readable, because he understood it, while "enumerate" was foreign to him. He had to put in some effort to understand the new thing before he could consider it readable.

I hope we all agree that 1) enumerate is more readable, and 2) it only is because we understand what enumerate does.

Isn't the same true for "for/else"? Yes, you can replace "for/else" with an explicit "found" variable, but you're adding to the bulk of the code, increasing opportunities for errors, and not using one of the tools Python provides for concise code.

Simon Hibbs 7:31 AM on 20 Oct 2011

while ... else # good
for   ... else # bad

Ned Batchelder 7:54 AM on 20 Oct 2011

@Simon: Since "while/else" and "for/else" have the same semantics for the else clause, why do you like one, but not the other?

Christopher Allen-Poole 1:52 AM on 22 Oct 2011

It seems, upon inspection, that this is hearkening back to Knuth and Structured Programming with GOTO Statement. In following that type of logic it is very sound. The problem is that this particular case it is non intuitive for a variety of reasons.

There is indenting inconsistency:

for x in y:
    # one level of indentation
    if test(x):
       # second level
       break
else:
   # this generally has to do with code indented at least 
   # two levels deep. That is inconsistent.
   do_something()

There is the fact that you already have a condition being evaluated as an implicit part of the for: looping constructs boil down to the logical steps:

:test
if condition:
   loop_eval()
   goto test

It is implied, in most circumstances, that an "else" in that circumstance will relate to the implied if and not something contained within the body of the if.

It also contradicts the way that while...else works. A language which wishes to be consistent, should have for loops simply be semantic sugar for a complicated while loop.

Ned Batchelder 5:44 AM on 22 Oct 2011

@Christopher: you are now the second person to claim that "for/else" and "while/else" are somehow different. They aren't. Where is this confusion coming from?

AlexR 7:04 PM on 23 Oct 2011

for ... :
# do something
else :
# the coder wants job security - to monopolize maintenance of the code

Jeff 9:53 PM on 23 Oct 2011

@Julian, you missed my point. @Steve/@Pekka suggested that the else does not get called if the list is empty. I was only demonstrating that "yes it does".

I was not trying to imply that the else only gets called for empty lists, and in fact explicitly said "iterating an empty container will trigger the else".

Having said that, I'm definitely leaning towards banning the construct in my development team because of the confusion it makes possible.

Julian 9:58 PM on 23 Oct 2011

@Jeff,

Wait, so there was confusion over the confusion over the confusion?

I think I'd better apologise for my piece in this, and step carefully away from this conversation!

Wil Tan 7:05 AM on 24 Oct 2011

I've definitely been confused by this construct at least once in my Python journey, and this post got me paranoid so I wrote a script to find instances of it in my code, after verifying that pylint doesn't catch even the simplest mistake like a for loop with an else clause but without any break statement.

Hope it's useful for someone: https://gist.github.com/1308847

klo 3:09 AM on 27 Oct 2011

x = [1, 2, 3]
y = {}
y[2] = 4

--------------------
for z in x:
  try:
    if y[z]: print y[z]
  except: pass
--------------------
# returns 4

--------------------
if any(y[z] for z in x): print y[z]
--------------------
# can't try

Richard 8:00 PM on 8 May 2018

Okay ... So, I understand that it CAN make sense ... but it seems to ONLY make sense with an if statement, and a break at the end. However, I am running into situations that make understanding it confusing with examples that I type into the interactive without any if statement, and with/and without break. It seems inconsistent to me still ....

JCZD 9:47 AM on 10 Nov 2022

Following on the comment by Steve Ferg, I would have suggested the following behaviour:

while/for ... :
    if some_condition:
        break
broken:
    print("loop was broken at some point")
else:
    print("loop did not run even once")
then:
    print("loop ran at least once and was never broken")

however, we would have a confusion with the current meaning of “else”, so using another keyword would (such as ‘empty’) be better suited.

the current syntax and the way ‘else’ - although elegant - works feels a bit useless because a similar result can be achieved with a try/except statement:

try:
    while/for ... :
        if condition:
            raise LoopWasBroken
except LoopWasBroken:
    print("loop was broken")
else:
    print("loop was not broken, or didn't run at all")

IMHO this makes it quite clear why the current meaning of “else” in a loop construct causes so much confusion and - may I say - disgust.

Currently so far, there is no syntaxically clean way to do something if a loop never ran ; the only way is to use an additional variable that gets set over and over on every iteration:

loop_ran_once = False
while/for ... :
    loop_ran_once = True
    if condition:
        break
if not loop_ran_once:
    print("loop did not run")
# and to avoid garbage although it may not be considered good practice:
del loop_ran_once

For/else

Comments

Add a comment: