A quest for pythonic interfaces
Created 18 October 2004
As I said in Interfaces, the most important concept in designing software systems is the interface. Yet Python has no support for them. Here’s a story about my quest for Pythonic interfaces.
One way to do interfaces in Python
I like interfaces. They let me create a concrete representation of the behavior of a class, separate from the implementation of that behavior. I also like Python, which has no language support for interfaces as such. I’ve been building a system in Python, and wanted to use interfaces as an organizing principle. I figured it would be simple to simulate interfaces in Python. They are just completely abstract base classes:
class Executable:
""" A thing which can be executed.
"""
def execute(self, environment):
""" Executes itself in the given environment.
"""
pass
Now a class that implements the Executable interface simply inherits from Executable:
class Statement(Executable):
""" A programming language statement.
"""
def execute(self, environment):
""" A concrete implementation of execute for statements.
"""
# .. Do whatever it is you need to do ..
This works well, and serves some of the goals of interfaces:
- The interface is expressed as a first-class entity in the system. It has a name, and documentation just like any other piece of code.
- The implementation class has a concrete indication that it is related to the interface: it derives from it.
- The interface can be used in code. For example, Statement is an Executable, and instanceof will report it as such.
Slicker Python interfaces
While the simple Executable interface above will work, it has a problem: if an implementing class makes the mistake of not implementing a method, the interface’s method will be used instead, and the interface’s method silently does nothing. Leaving empty code paths is an invitation to mysterious bugs. This is a particular annoyance of mine: see Erroneously empty code paths for more on this.
To close that hole, I created a function to use as the body of interfaces’ methods. If called, it will raise an exception:
import sys
def _functionId(obj, nFramesUp):
""" Create a string naming the function n frames up on the stack. """
fr = sys._getframe(nFramesUp+1)
co = fr.f_code
return "%s.%s" % (obj.__class__, co.co_name)
def abstractMethod(obj=None):
""" Use this instead of 'pass' for the body of abstract methods. """
raise Exception("Unimplemented abstract method: %s" % _functionId(obj, 1))
class Executable:
def execute(self, environment):
""" Executes itself in the given environment.
"""
abstractMethod(self)
We’ve used some helper methods here to get helpful reporting. Now if a class doesn’t implement a method, an exception is raised with the name of the missing method:
Exception: Unimplemented abstract method: __main__.Statement.execute
Complications
But all this Python hacking to emulate Java interfaces may be missing the point: As I mentioned in Interfaces, Python doesn’t have interfaces partly because interfaces are an artifact of statically typed languages. Python’s dynamic nature makes interfaces far less important. And interfaces don’t have as much flexibility as Python offers in the first place.
As my system grew, I wanted to have an interface with many required methods, but then two methods, only one of which would ever be implemented by a base class:
class Executable:
# .. (many methods) ..
def executeOneWay(self, environment):
""" Executes itself in the given environment.
"""
abstractMethod(self)
def executeTheOtherWay(self, globals, locals, services):
""" Executes itself with globals, locals, and services.
"""
abstractMethod(self)
Once we’ve done this, though, how do we tell which method to call? Because of the interface as a base class, there is no way to distinguish between a OneWay Executable and a TheOtherWay Executable. Both instances have both methods.
The statically typed way would be to extend Executable into the two interfaces we really want to discuss:
class Executable:
# .. (many methods) ..
class ExecutableOneWay(Executable):
def executeOneWay(self, environment):
""" Executes itself in the given environment.
"""
abstractMethod(self)
class ExecutableTheOtherWay(Executable):
def executeTheOtherWay(self, globals, locals, services):
""" Executes itself with globals, locals, and services.
"""
abstractMethod(self)
But now we’re getting complicated, and if another method with two possibilities arises, we have to fracture our interfaces yet again. The extreme end of this scenario is many interfaces with very few methods each. This is the way some of the Java library is beginning to look.
Does-a
The thing about dynamic languages like Python is that you don’t have to limit yourself to declaring groups of methods via interfaces. Traditional object-oriented language give programmers two building blocks to choose from: inheritance (is-a) and delegation (has-a). Interfaces are a tool to use is-a for figuring out what an object can do. With Python, there’s a better way to figure out what an object can do: you ask it. Introspection lets you ask about the availability of a method (lets call this does-a).
The Pythonic way to handle the dual-natured Executable is to ask the object if it can do executeOneWay, and if it can’t, to do executeTheOtherWay:
if hasattr(ex, 'executeOneWay'):
ex.executeOneWay(...)
else:
ex.executeTheOtherWay(...)
But consider the interfaces I’ve implemented above. My fancy all-in-one interface has truly let me down: because my object derives from Executable, and Executable implements both executeOneWay and executeTheOtherWay, there’s nothing a derived class can do to control its destiny. All Executable objects implement both methods, just not in a useful way. The stub exception-raising methods in the interface have neutralized the power of the Python “does-a” mechanism.
I could add methods to the interface to answer the question about the methods:
class Executable:
# .. (many methods) ..
def doesTheOtherWay(self):
""" Returns True if executeTheOtherWay should be used.
"""
abstractMethod(self)
but now we’re really duplicating the Pythonic hasattr approach, and building a whole parallel system for talking about methods.
The double ExecutableOneWay and ExecutableTheOtherWay interfaces could handle this split, but then I’m just moseying down the path to having an interface per method.
What to do?
So I’m stuck: I don’t know how I want to move forward. I like interfaces when they work well: a shorthand name for a whole set of behaviors. But once the behaviors become more complex, the interface shorthand breaks down.
Python makes it simple to ask about particular methods, but then fails me when I really do want to talk about a whole set of behaviors. There’s a line here, with methods on one side, and interfaces on the other. Some problems are solved more naturally on one side, others on the other, but it doesn’t feel line a smooth transition between the two.
I’ll have to keep trying out possibilities, I guess.
Ways out
Of course this topic has a long and distinguished history:
- There is a Python Extension Proposal for interfaces: PEP 245 — Python Interface Syntax, but I don’t know what its implementation status is.
- I don’t yet understand meta-classes, but they of course are used to solve these problems as well.
See also
- Interfaces, about the theory and practice of interfaces.
- My blog, where other similar topics are discussed.
Comments
http://peak.telecommunity.com/PyProtocols.html
- ML announcement
http://mail.zope.org/pipermail/interface-dev/2004-June/000036.html
- Product download
http://zope.org/Products/ZopeInterface/
Take a look at http://www-106.ibm.com/developerworks/linux/library/l-cppeak.html for a pretty good tutorial on how to use PyProtocols. It should at least be enough to get you started, and get your mind churning.
def makeAbstract(f):
f.abstract=True
return f
class NotImplemented(Exception): pass
class Executable:
@makeAbstract
def oneWay(self):
raise NotImplemented
@makeAbstract
def secondWay(self):
raise NotImplemented
or for py2.3
oneWay = makeAbstract(oneWay)
secondWay = makeAbstract(secondWay)
class Statement(Executable):
def oneWay(self):
print "one way executed"
s = Statement()
if not hasattr(getattr(s, 'oneWay'), "abstract"):
s.oneWay()
else:
s.secondWay()
The idea is to declare a method as being somehow: method.isAbstract=True and then inspect for that attribute of tyhe method object.
try:
s.oneWay()
except AbstractMethodError:
s.otherWay()
This seems quite Pythonic - I must be missing something....
.
One interesting aspect: He says that python inheritance is about inheriting code, not about inheriting structure -
duck typing takes this to the extreme.
Then again, many Python users use test-driven development, so the meaning
of a class or method "emerges" from
the use of it. The only purpose of
inheritance, then, is to communicate
how to use a class in a proper way.
The hasattr-Method you suggest may be
useful, but OO folks would probably say
that it violates the Liskov Substitution
Principle, i.e. the behavior of a type
changes with a subtype, which makes the
code harder to understand.
Regarding the exceptions of non-implemented methods: The "pythonic
way" of doing this is "raise NotImplemented", if I'm not mistaken.
A class like IMyInterface(object) (prefixed with an I for clarity) which raises 'NotImplementedError's, with exception handling for optionally implemented methods… and callables.
For very simple APIs that define single callables (e.g. a render method for a templating engine), the caller of the API doesn't care if the thing handed to it to call is a base function or a class that implements __call__, allowing developers that use the API to choose the level of complexity as needed. Don't keep state? Function. Keep state? Class instance.
I'll probably use an implementation of your _functionId concept, as it makes true error conditions far more legible.
Add a comment: