|Ned Batchelder : Blog | Code | Text | Site|
Skipping C extensions
» Home : Blog : December 2012
Coverage.py has a C extension: the trace function is implemented in C for speed, since it's invoked for every line of your program. This works great, and makes things much faster, but not every user has a C compiler properly installed on their machine. There's also a Python implementation of the trace function, so we can make do without a compiler, but this make the installation a little tricky.
It turns out that distutils is great at attempting what you instruct it to, but not that great at letting you know what happened. In particular, building an extension can fail in a variety of ways. Separating actual build failures from other pathologies is not simple.
I had a way I was doing it, but still got bug reports from people who could not install the kit without getting tangled. Two suggestions were to do it the way SQLAlchemy did it, or the way simplejson did it. It turns out they do it the same way!
For example, compare SQLAlchemy's setup.py:
ext_errors = (CCompilerError, DistutilsExecError, DistutilsPlatformError)
with simplejson's setup.py:
if sys.platform == 'win32' and sys.version_info > (2, 6):
The code is clearly copied from the same source, even the name ve_build_ext is the same, as is its comment. SQLAlchemy has a new twist based on Python bug 7511, but it's clear we're all copying and pasting the same solution.
In fact, a Google search for that comment reveals a dozen or so packages using the same technique, with varying degrees of automatic fallback.
Does anyone know where this code first came from? What does the "ve" stand for in "ve_build_ext"? Is any of the work on better packaging tackling this issue? And how do people test this code? One suggestion was mocking out distutils, but the central issue here is properly catching the diverse exceptions that might occur if the compile fails. Help!
tagged: python» 10 reactions