« | » Main « | »

Coverage.py v3.5.1 beta 1

Wednesday 31 August 2011

Coverage.py v3.5.1 beta 1 is ready. There's not much new: the one new feature is for people measuring coverage on a number of different machines, then combining the data together for reporting. Now you can specify how differently-named source directories on those machines get combined together.

Branch coverage works better for "for-else" structures, and for the "with" statement, so if you've been experimenting with branch coverage, but have been frustrated by false warnings, give this release a spin.

Max is off to NYU

Saturday 27 August 2011

My middle son Max is starting his freshman year at NYU in a few days. Move-in day was supposed to be tomorrow, but Hurricane Irene threw a wrench in the works, and now he moves in Monday. Getting there from Boston will still be a challenge due to the weather, so we're all on tenterhooks watching the forecasts.

It's been a joy having him home for his gap year, and now we get an extra hurricane day with him, but then he is off.

Of course we made a cake, themed around New York City and NYU, with a skyline of buildings, a yellow cab, the Washington Square arch, and a giant torch-like thing in honor of NYU's symbol:

New York City cake for Max

It's hard sending Max off, but it's also exciting to know that he's on a new adventure, one we can enjoy vicariously.

Stack ninjas

Wednesday 24 August 2011

Running a Django test suite today, we had a frustrating problem: database objects were leaking from one test to the next. To track down how this was happening, I used some aggressive debugging tricks.

First, a little background: Django provides two test case classes for creating tests: TestCase, which should usually be used, and TransactionTestCase, which should be used for testing code that does explicit transaction management.

TestCase uses transactions to restore the state of the database at the end of each test. This helps guarantee test isolation, an important property of tests that ensures that each tests runs without being tainted by the side effects of previous tests.

We didn't have any TransactionTestCase classes in our code, but we were definitely seeing bleed-through from one test to the next. One particular test failed unless it deleted all User objects at the very beginning. But there shouldn't have been any Users in the database at the start of the test in the first place. Clearly, some database state wasn't being restored after some test: there was one User object left in the database when it should have been empty.

I knew what test was tripped up by the User object, but which test left it behind? There are hundreds of tests, and many of them create User objects. Reading the code wasn't revealing anything. A more powerful technique would be needed.

It's easy in the failing test to examine the errant User object. What if the User object could itself point to where it had been created? If the creator of User objects annotated the object with a stack trace, then we could examine that stack in the failing test, and we'd identify the culprit.

Keep in mind here: my plan wasn't to add features to Django, or to our product code, or even our tests. I was going to hack whatever was needed to find the test that polluted the database, and then I was going to get rid of it all. So all sorts of dirty tricks were allowable.

Getting a useful stack trace at a point in the code isn't difficult: inspect.stack gives us the stack itself, and from there we can pull useful information to put into a string:

import inspect

def get_stack():
    """Get a string describing the current stack."""
    parts = []
    for f in inspect.stack()[1:]:
        frame = f[0]
        code = frame.f_code
        if code.co_varnames and code.co_varnames[0] == "self":
            data = frame.f_locals['self']
        else:
            data = ""
        parts.append("%s:%d:%s:%r" % (f[1], f[2], f[3], data))
    return "\n".join(parts)

Calling this function returns a string that looks like this:

/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/contrib/auth/models.py:151:create_user:<django.contrib.auth.models.UserManager object at 0xa7bc56c>
/ned/ibis/nest/tests/test_nest.py:49:create_user:''
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/util.py:478:try_run:''
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py:267:setUp:ibis.nest.tests.test_nest.test_create_user
/opt/python2.6/lib/python2.6/unittest.py:270:run:ibis.nest.tests.test_nest.test_create_user
/opt/python2.6/lib/python2.6/unittest.py:300:__call__:ibis.nest.tests.test_nest.test_create_user
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py:151:runTest:Test(ibis.nest.tests.test_nest.test_create_user)
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py:133:run:Test(ibis.nest.tests.test_nest.test_create_user)
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py:45:__call__:Test(ibis.nest.tests.test_nest.test_create_user)
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis.nest.tests.test_nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis.nest.tests.test_nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis.nest.tests>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis.nest.tests>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis.nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis.nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis.nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis.nest>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=ibis>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=ibis>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:223:run:<nose.suite.ContextSuite context=None>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/suite.py:176:__call__:<nose.suite.ContextSuite context=None>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/core.py:61:run:<nose.core.TextTestRunner object at 0xb2a810c>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/core.py:197:runTests:<nose.core.TestProgram object at 0xab4c34c>
/opt/python2.6/lib/python2.6/unittest.py:817:__init__:<nose.core.TestProgram object at 0xab4c34c>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/core.py:118:__init__:<nose.core.TestProgram object at 0xab4c34c>
/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/core.py:284:run:''
/ned/ibis/nose_test_runner.py:105:run_tests:''
/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/commands/test.py:34:handle:<django_nose.management.commands.test.Command object at 0xa02cf8c>
/ned/ve/lib/python2.6/site-packages/South-0.7.3-py2.6.egg/south/management/commands/test.py:8:handle:<django_nose.management.commands.test.Command object at 0xa02cf8c>
/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/base.py:220:execute:<django_nose.management.commands.test.Command object at 0xa02cf8c>
/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/base.py:191:run_from_argv:<django_nose.management.commands.test.Command object at 0xa02cf8c>
/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/__init__.py:379:execute:<django.core.management.ManagementUtility object at 0x9e3de8c>
/ned/ve/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/__init__.py:438:execute_manager:''
./manage.py:49:<module>:''
    

Every frame in the stack is a single line, with the file name, line number, function name, and if present, the value of self. Getting the stack string is simple enough: User objects are all created with a helper function called create_user in django/contrib/auth/models.py. Adding a call to get_stack there will get us the stack trace we want.

But where to store it in User? Ideally there'd be a description column or something that we could stuff this into, but there isn't. I tried storing it as the password, but that failed miserably, because the password is hashed before storing, and then dozens of tests failed when the passwords didn't match. If these had been ordinary Python objects, I could have just added new attributes to store the stack trace, but these were written to the database, then later read back out, so only information that got stored in the database was useful to me.

So I was a bit stuck: User had no usable text column in which to store the stack trace, so I couldn't annotate the User object itself. Instead, maybe I could store the stack trace somewhere else, and associate it with the User object. My first thought was to use the primary key of the User as an identifier, but of course, as the database is rolled back and Users created, they re-use the same keys, so that was no good.

Then I noticed that Users are created with the current time as their last_login value. Those datetimes have enough resolution that each User gets a unique value, and of course, time progresses independently of database rollbacks, so there's no danger of values getting reused.

Now I had a plan: modify django/contrib/auth/models.py to keep a global dictionary mapping User creation times to the stack trace that created them. Then, in my failing test, I could examine the User object, and use its last_login time to look up the stack trace.

I modified the creation code like this (simplified):

USER_TIMES = {}     # <-- new

def create_user(self, username, email, password=None):
    """
    Creates and saves a User with the given username, e-mail and password.
    """
    now = datetime.datetime.now()

    user = self.model(username=username, email=email, is_staff=False,
                    is_active=True, is_superuser=False, last_login=now,
                    date_joined=now)

    USER_TIMES[now] = get_stack()       # <-- new
    user.save()
    return user

Then, in the failing test, I added a check at the very beginning, where we expected to have no Users:

for user in User.objects.all():
    from django.contrib.auth.models import USER_TIMES
    print "\n\n*** Extra user created at\n%s" % USER_TIMES[user.last_login]

With these modifications in place, running the tests gave me just what I wanted: a stack trace that pinpointed where the User was created. The actual trace is the one I've included at the top of this post. The most useful part of the stack was this line:

/ned/ve/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py:267:setUp:ibis.nest.tests.test_nest.test_create_user
    

which shows that the User was created in the setUp method of the test_create_user test. Looking at that test showed what the problem was:

def create_user():
    try:
        User.objects.get(username=USERNAME)
    except User.DoesNotExist:
        User.objects.create_user(username=USERNAME, email="%s@example.com" % USERNAME, password=PASSWORD)

@with_setup(create_user)
def test_create_user():
    '''Ensure that user creation is working'''
    assert User.objects.get(username=USERNAME)

As I mentioned at the very beginning, Django provides two test case classes to derive from. But here's a test that isn't part of a class at all! Nose allows the use of test functions in addition to the classic unittest-style test classes. Many people like the cleaner Pythonic feel of functions. But Django relies on the setup and teardown that the test case classes provide. Using pure test functions in a Django project is a good way to miss out on the machinery that cleans the databases between tests, which is precisely what we'd been experiencing.

Removing this test (it also happened to be redundant!) solved the problem.

Fog-free goggles

Monday 15 August 2011

We engineers tend to apply creative problem-solving to all aspects of our life. No difficulty is too small to try to overcome. When swimming laps, goggles are important, unfortunately, they constantly fog up, which is frustrating. But I recently developed a technique that works great to keep them clear.

The answer is simple: wash the inside of the goggles with baby shampoo, but don't rinse them. Use just enough water to get a good film of shampoo on the inside. If there are any bubbles, blow them away. The goggles will stay perfectly fog-free, and the baby shampoo won't bother your eyes.

How we do it in Boston

Sunday 7 August 2011

I'm an organizer of the Boston Python Meetup, and in the last six months Jessica McKellar and Asheesh Laroia came on board, and we've ramped up the activity in the group. We're also starting to talk more with other Python user groups elsewhere, so I thought it would be useful to describe how we ccurrently run ours. Of course, if you are in the Boston area, I heartily encourage you to join us!

We have three types of events we've been doing since the spring: a monthly presentation series, a monthly project night, and a weekend workshop series.

Presentation series

The monthly presentation series is what we've been doing for years now, though we've added to it: the third Wednesday of the month (usually), we have a speaker or two for a traditional "sit down and listen" presentation. The speakers are from the local community, and need not be huge experts. Anyone with enough interest in a topic to talk about it is welcome.

You can look over our previous meetups to see some past topics. Sometimes a speaker will volunteer, but more often I have to cajole people into presenting. A few of the topics have simply been things that I wanted to know more about, or that I thought other members would want to know more about, and I ask around to find someone willing to learn enough about it to present it. Other times, people are doing something interesting, and I ask them to talk about their work.

We've tried to keep a good mix of beginner and advanced topics, though I think we average around the intermediate-to-advanced level.

A great source for speakers is PyCon. Often you have knowledgable speakers with presentations ready to go right in your neighborhood, but you have to root them out because they don't know your user group exists. For example, every year, in January and February, we run meetup nights with the upcoming Boston-area PyCon presenters rehearsing their talks. Last year we had nine presenters to schedule!

Following Yannick's advice, we've recently added two new components to the presentation nights: lightning talks and beers afterwards. Lightning talks are a good way to get more people talking, though we've only just started, and I haven't figured out how to keep a steady stream of them going. People don't think they have anything to say, but as I put it recently,

Everyone has two things in them: a novel and a lightning talk. We don't have time for your novel, but we'd love to hear your lightning talk.

We don't follow strict PyCon lightning talk rules, for example, they need not fit strictly in five minutes. Anything up to 10 or 15 minutes is fine. I'd like to do a night of only lightning talks soon.

Beers are easy: find a bar near your meetup location, and decide to go there. At the beginning of the meetup, announce that you will go there, then afterward, go there. Even better is if you can get a sponsor for the beers. These days, the Python hiring market is very hot, so it has not been difficult for us to get sponsors to pay for a round of drinks. Adding the word "free" to "beer" really seems to pique people's interest!

Attendance at presentation nights varies for all sorts of reasons, but we typically have about 60-90 attendees, and a sponsored beer afterward will get 20-30 people.

Our next presentation night is Alternate Pythons: PyPy, Jython, IronPython, plus lightning talks and beers on August 17th.

Project night

Project nights are informal gatherings for people to just hang out and work on whatever they want. Often there is more chatting than working, which is also great. There is virutally no structure to a project night, just a lot of Pythonistas getting together in one place to see what happens. We have a brief period at the beginning when people can stand up and make an announcement to clump together by interest. For example, "I'm working through the tutorial, if any other beginners want to work with me," or, "I'm trying to debug a C extension that leaks, if anyone knows anything about that," or, "I want to contribute to a project but need some ideas," and so on. Depending on how many people you have, this only takes 5-10 minutes. The rest of the evening is just people working and talking.

Some people come to the project night expecting that we'll be sprinting on Python, or will have chosen a project to work on as a group. We've never tried either of these, though other groups have had success with them.

Lately, we've been running the Project Nights regularly every month, and have been getting about 50 people each night.

Our next project night is August 11th.

Weekend workshops

We've run three weekend workshops to introduce beginners to Python. All of them so far have been specifically targetted at women, but the intent is to open it up to other demographics eventually. Jessica has written an extensive set of notes about the workshops, so I'll let those speak for themselves.

One of the reasons we've been running the Project Nights so regularly is to give the workshop attendees a supported place to continue their learning.

Other ideas?

Those are the events we've been running in Boston. Judging from the growth in the group, they work well to build community. It's been really great meeting new members, and seeing their interest in the group.

We have some other ideas for events, and they may be realized in the fall. Other people do things differently, I'd love to hear what works for you.

« | » Main « | »