« | » Main « | »

Multiple inheritance is hard

Sunday 28 October 2012

Multiple inheritance is hard, we all know it, but even after warning people about it myself, I found myself tripped up by it yesterday. All I wanted was a mixin for my unittest.TestCase class.

Unittest's TestCases use setUp and tearDown to affect the test state, and I wanted a way to share setUp and tearDown implementations between two different test classes. A mixin seemed like a good solution. I already have a BaseTestCase of my own that inherits from unittest.TestCase, so my mixin looked like this:

class BaseTestCase(unittest.TestCase):
    def setUp(self):
        super(BaseTestCase, self).setUp()
        #.. set up that everyone needs ..

    def tearDown(self):
        #.. tear down that everyone needs ..
        super(BaseTestCase, self).tearDown()

class MyMixin(object):
    def setUp(self):
        super(MyMixin, self).setUp()
        #.. do something here ..

    def tearDown(self):
        #.. do something here ..
        super(MyMixin, self).tearDown()

class RealTestCase(BaseTestCase, MyMixin):
    # I didn't need setUp and tearDown here..

    def test_foo(self):
        #.. etc ..

The theory is that mixins avoid some complications of multiple inheritance: they provide methods, but no attributes, and don't inherit in a way that produces diamonds in the inheritance hierarchy. And I did the right things here, using super to keep the MRO (method resolution order) intact.

But this code doesn't work, MyMixin.setUp is never invoked. Why? It's because TestCase in unittest doesn't invoke super():

# unittest.py (some details omitted :)
class TestCase(object):
    def setUp(self):

The method resolution order of RealTestCase is:

  • RealTestCase
  • BaseTestCase
  • unittest.TestCase
  • MyMixin
  • object

Since TestCase.setUp doesn't use super, the sequence stops there, and MyMixin is never consulted. At first I thought, "TestCase.setup should use super!" But, if it did, it would fail in simpler hierarchies that don't use a mixin, because it would try to invoke object.setUp(), which doesn't exist.

I suppose TestCase could be re-written like this:

# unittest.py (hypothetical)
class TestCase(object):
    def setUp(self):
        parent = super(Base, self)
        if hasattr(parent, "setUp"):

This works, and now MyMixin.setUp() is invoked, but then it fails for the same reason: it tries to invoke object.setUp, which doesn't exist, so MyMixin also needs to have the defensive check of its parent. Yuk.

The simple solution is to swap the order of the mixin and the base class:

class RealTestCase(MyMixin, BaseTestCase):

With this class declaration, the MRO is: RealTestCase, MyMixin, BaseTestCase, TestCase, object. All the setUp's are invoked, and it ends cleanly with no complicated parentage check.

But I can't help feeling this isn't great. For one thing, conceptually, BaseTestCase is the real parent, and the mixin is just tossed in, so I would rather be able to write my base classes in the other order. But more worrying, this solution also means that I have to be very careful to know the lineage of all my classes all the way up to object.

Maybe this is the fundamental truth about Multiple Inheritance, and why it is so difficult: details about base classes that you thought were abstracted away from you can suddenly be critical to understand. Like all action at a distance, this is mysterious and confusing.

My take-aways from this:

  • Mixins come first even though it looks odd.
  • Multiple inheritance is hard.
  • There are yet more Python details to master.

The question remaining in my mind: would class hierarchies be better if the top-most classes (derived from object) used the defensive super style? Or is that overkill that defers rather than removes the pain? Would something else bite me later?


Wednesday 10 October 2012

I've been organizing the Boston Python user group for a few years now, and I like it a lot. Except for ordering pizza. But finally I've brought some technology to bear on the problem!

First, I've taken a poll of people RSVP'ing for tonight's project night, so I now have an empirical basis for deciding what fraction of the pizzas should be meat, vegetable, cheese, or vegan. Suprise (to me): vegetable wins.

Second, I've written what may be the world's most useful Python program: pizza.py!

"""How many pizzas do we need?"""

import math
import sys

if len(sys.argv) > 1:
    people = int(sys.argv[1])
    people = int(raw_input("How many RSVPs? "))

# The MUC (Meetup Universal Constant)
attending = people * .65

print "%d people will show up (guess)" % attending

# Appetite estimation
slices = attending * 2.5

# Basic pizza geometry
pies = slices / 8

print "%.1f pizzas (or so)" % pies

# From answers to the 10/2012 project night:
#   81 answers
#   26 meat 32%
#   37 veg 45%
#   16 cheese 20%
#   2 vegan 3%

vegan = int(.03 * pies) or 1
meat = int(.33 * pies) or 1
veg = int(.45 * pies) or 1
cheese = pies - vegan - meat - veg
if cheese < 1:
    cheese = 1
cheese = int(math.ceil(cheese))

print "%2d cheese" % cheese
print "%2d meat" % meat
print "%2d veggie" % veg
print "%2d vegan" % vegan
print "%2d total" % (cheese + meat + veg + vegan)

The hard truth here is the Meetup Universal Constant. The MUC has been empirically determined, and says that no matter how much you wheedle people to show up if they say they will, and vice-versa, about one-third of the RSVPs will not attend. This number has proven remarkably stable over the 25 or so events that we've measured.

As an example, for tonight's event, we have 127 RSVPs:

How many RSVPs? 127

82 people will show up (guess)
25.8 pizzas (or so)

 6 cheese
 8 meat
11 veggie
 1 vegan

26 total

Your numbers may vary. Perhaps Boston is a vegetarian hotbed compared to where you are. Maybe your city has more-predictable weather and fewer people abandon their intention to attend. Tweak pizza.py as you see fit!

« | » Main « | »