As I said in Interfaces, the most important concept in designing software systems is the interface. Yet Python has no support for them. Here's a story about my quest for Pythonic interfaces.

One way to do interfaces in Python

I like interfaces. They let me create a concrete representation of the behavior of a class, separate from the implementation of that behavior. I also like Python, which has no language support for interfaces as such. I've been building a system in Python, and wanted to use interfaces as an organizing principle. I figured it would be simple to simulate interfaces in Python. They are just completely abstract base classes:

class Executable:
    """ A thing which can be executed.
    """
    def execute(self, environment):
        """ Executes itself in the given environment.
        """
        pass

Now a class that implements the Executable interface simply inherits from Executable:

class Statement(Executable):
    """ A programming language statement.
    """
    def execute(self, environment):
        """ A concrete implementation of execute for statements.
        """
        # .. Do whatever it is you need to do ..

This works well, and serves some of the goals of interfaces:

  • The interface is expressed as a first-class entity in the system. It has a name, and documentation just like any other piece of code.
  • The implementation class has a concrete indication that it is related to the interface: it derives from it.
  • The interface can be used in code. For example, Statement is an Executable, and instanceof will report it as such.

Slicker Python interfaces

While the simple Executable interface above will work, it has a problem: if an implementing class makes the mistake of not implementing a method, the interface's method will be used instead, and the interface's method silently does nothing. Leaving empty code paths is an invitation to mysterious bugs. This is a particular annoyance of mine: see Erroneously empty code paths for more on this.

To close that hole, I created a function to use as the body of interfaces' methods. If called, it will raise an exception:

import sys

def _functionId(obj, nFramesUp):
    """ Create a string naming the function n frames up on the stack. """
    fr = sys._getframe(nFramesUp+1)
    co = fr.f_code
    return "%s.%s" % (obj.__class__, co.co_name)

def abstractMethod(obj=None):
    """ Use this instead of 'pass' for the body of abstract methods. """
    raise Exception("Unimplemented abstract method: %s" % _functionId(obj, 1))

class Executable:
    def execute(self, environment):
        """ Executes itself in the given environment.
        """
        abstractMethod(self)

We've used some helper methods here to get helpful reporting. Now if a class doesn't implement a method, an exception is raised with the name of the missing method:

Exception: Unimplemented abstract method: __main__.Statement.execute

Complications

But all this Python hacking to emulate Java interfaces may be missing the point: As I mentioned in Interfaces, Python doesn't have interfaces partly because interfaces are an artifact of staticly typed languages. Python's dynamic nature makes interfaces far less important. And interfaces don't have as much flexibility as Python offers in the first place.

As my system grew, I wanted to have an interface with many required methods, but then two methods, only one of which would ever be implemented by a base class:

class Executable:
    # .. (many methods) ..

    def executeOneWay(self, environment):
        """ Executes itself in the given environment.
        """
        abstractMethod(self)

    def executeTheOtherWay(self, globals, locals, services):
        """ Executes itself with globals, locals, and services.
        """
        abstractMethod(self)

Once we've done this, though, how do we tell which method to call? Because of the interface as a base class, there is no way to distinguish between a OneWay Executable and a TheOtherWay Executable. Both instances have both methods.

The statically typed way would be to extend Executable into the two interfaces we really want to discuss:

class Executable:
    # .. (many methods) ..

class ExecutableOneWay(Executable):
    def executeOneWay(self, environment):
        """ Executes itself in the given environment.
        """
        abstractMethod(self)

class ExecutableTheOtherWay(Executable):
    def executeTheOtherWay(self, globals, locals, services):
        """ Executes itself with globals, locals, and services.
        """
        abstractMethod(self)

But now we're getting complicated, and if another method with two possibilities arises, we have to fracture our interfaces yet again. The extreme end of this scenario is many interfaces with very few methods each. This is the way some of the Java library is beginning to look.

Does-a

The thing about dynamic languages like Python is that you don't have to limit yourself to declaring groups of methods via interfaces. Traditional object-oriented language give programmers two building blocks to choose from: inheritance (is-a) and delegation (has-a). Interfaces are a tool to use is-a for figuring out what an object can do. With Python, there's a better way to figure out what an object can do: you ask it. Introspection lets you ask about the availability of a method (lets call this does-a).

The Pythonic way to handle the dual-natured Executable is to ask the object if it can do executeOneWay, and if it can't, to do executeTheOtherWay:

if hasattr(ex, 'executeOneWay'):
    ex.executeOneWay(...)
else:
    ex.executeTheOtherWay(...)

But consider the interfaces I've implemented above. My fancy all-in-one interface has truly let me down: because my object derives from Executable, and Executable implements both executeOneWay and executeTheOtherWay, there's nothing a derived class can do to control its destiny. All Executable objects implement both methods, just not in a useful way. The stub exception-raising methods in the interface have neutralized the power of the Python "does-a" mechanism.

I could add methods to the interface to answer the question about the methods:

class Executable:
    # .. (many methods) ..
    def doesTheOtherWay(self):
        """ Returns True if executeTheOtherWay should be used.
        """
        abstractMethod(self)

but now we're really duplicating the Pythonic hasattr approach, and building a whole parallel system for talking about methods.

The double ExecutableOneWay and ExecutableTheOtherWay interfaces could handle this split, but then I'm just moseying down the path to having an interface per method.

What to do?

So I'm stuck: I don't know how I want to move forward. I like interfaces when they work well: a shorthand name for a whole set of behaviors. But once the behaviors become more complex, the interface shorthand breaks down.

Python makes it simple to ask about particular methods, but then fails me when I really do want to talk about a whole set of behaviors. There's a line here, with methods on one side, and interfaces on the other. Some problems are solved more naturally on one side, others on the other, but it doesn't feel line a smooth transition between the two.

I'll have to keep trying out possibilities, I guess.

Ways out

Of course this topic has a long and distinguished history:

See also

  • Interfaces, about the theory and practice of interfaces.
  • My blog, where other similar topics are discussed.

Comments

[gravatar]
Duncan McGreggor 5:14 AM on 20 Oct 2004

Are you familiar with PyProtocols? Both PyProtocols and ZopeX3 have interface implementations that are very robust. One of PyProtocols's advantages is that it contains Zope interfaces as a subset.

http://peak.telecommunity.com/PyProtocols.html

[gravatar]
Michael Haubenwallner 5:57 AM on 20 Oct 2004

There is a separate distribution of the zope.interface package used in Zope 3, along with the packages it depends on:

- ML announcement
http://mail.zope.org/pipermail/interface-dev/2004-June/000036.html

- Product download
http://zope.org/Products/ZopeInterface/

[gravatar]
Jay P 7:34 AM on 20 Oct 2004

I also recommend PyProtocols. It can be a bit much to get into though. The documentation is very detailed, but you almost have to understand the whole library before the documentation is helpful.

Take a look at http://www-106.ibm.com/developerworks/linux/library/l-cppeak.html for a pretty good tutorial on how to use PyProtocols. It should at least be enough to get you started, and get your mind churning.

[gravatar]
Jonas 7:41 AM on 20 Oct 2004

Please excuse my ignorance if I completely misunderstand, but perhaps there are some design-by-contract-ish functions that can help? I've been trying to add Class::Contract to my Perl typing and so far I like it. You explicitly declare both parameters and methods for objects in one place only. Like a poor man's type checker in a dynamic language... not a bad fit.

[gravatar]
Gheorghe Milas 10:03 AM on 20 Oct 2004

Would something like this be usefull?

def makeAbstract(f):
f.abstract=True
return f
class NotImplemented(Exception): pass
class Executable:
@makeAbstract
def oneWay(self):
raise NotImplemented
@makeAbstract
def secondWay(self):
raise NotImplemented
or for py2.3
oneWay = makeAbstract(oneWay)
secondWay = makeAbstract(secondWay)

class Statement(Executable):
def oneWay(self):
print "one way executed"

s = Statement()

if not hasattr(getattr(s, 'oneWay'), "abstract"):
s.oneWay()
else:
s.secondWay()

[gravatar]
Gheorghe Milas 11:49 AM on 20 Oct 2004

I noticed the code I posted is not indented. Yose my blog to see the code indented.
The idea is to declare a method as being somehow: method.isAbstract=True and then inspect for that attribute of tyhe method object.

[gravatar]
Andrew W 9:42 PM on 20 Oct 2004

Why wouldn't you just use:

try:
s.oneWay()
except AbstractMethodError:
s.otherWay()

This seems quite Pythonic - I must be missing something....

[gravatar]
Brian Mahoney 12:05 PM on 21 Oct 2004

I looked at PyProtocols this spring and couldn't see that it was taking advantage of the new-style classes. If Python now allows me to subtype the built-in types I want to avoid any out-of-date coding tricks to do similar subtyping. Is PyProtocols relying on the new-style classes of Python?

[gravatar]
Jay P 8:55 AM on 22 Oct 2004

PyProtocols does rely on new-style classes, as it uses meta-classes for a lot of the fun and magic it does

[gravatar]
Robert Schmitt 6:08 AM on 28 Mar 2007

Bruce Eckel made an interesting presentation on "Python and design" for PyCon (cf http://us.pycon.org/zope/talks/2005/wed/track1/44/talkDetails)
.

One interesting aspect: He says that python inheritance is about inheriting code, not about inheriting structure -
duck typing takes this to the extreme.

Then again, many Python users use test-driven development, so the meaning
of a class or method "emerges" from
the use of it. The only purpose of
inheritance, then, is to communicate
how to use a class in a proper way.

The hasattr-Method you suggest may be
useful, but OO folks would probably say
that it violates the Liskov Substitution
Principle, i.e. the behavior of a type
changes with a subtype, which makes the
code harder to understand.

Regarding the exceptions of non-implemented methods: The "pythonic
way" of doing this is "raise NotImplemented", if I'm not mistaken.

[gravatar]
Alice Bevan-McGregor 5:51 AM on 27 Sep 2009

All of my use cases have been simple so far, but I'm using a combination of two things, these days:

A class like IMyInterface(object) (prefixed with an I for clarity) which raises 'NotImplementedError's, with exception handling for optionally implemented methods… and callables.

For very simple APIs that define single callables (e.g. a render method for a templating engine), the caller of the API doesn't care if the thing handed to it to call is a base function or a class that implements __call__, allowing developers that use the API to choose the level of complexity as needed. Don't keep state? Function. Keep state? Class instance.

I'll probably use an implementation of your _functionId concept, as it makes true error conditions far more legible.

Add a comment:

name
email
Ignore this:
not displayed and no spam.
Leave this empty:
www
not searched.
 
Name and either email or www are required.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.