Mocking is a great way to isolate your code from distracting dependencies while testing. But, it can be an arcane art unto itself. Today I wrote a test for code that uses the current date. While testing, it can't use the actual current date, because the test will produce different results on different days.

A solution is to mock out the datetime.datetime.today() function to return a known fixed date. But a few factors complicate matters. First, datetime.datetime is written in C, so Mock can't replace attributes on the class, so you can't simply mock out just the today() function.

Second, I want to mock just one function in the module, not the whole module. There are a few suggestions of how to do this out there: Michael Foord wrote about Partial mocking in the Mock docs, William John Bert showed another way, and of course, a Stack Overflow question about it.

None of these worked for me, perhaps because of subtle differences between my code under test and theirs. When mocking, it's critical to mock at the appropriate place. If your module has "import datetime", then you need to mock "mymodule.datetime" as a module. If instead you have "from datetime import datetime", then you need to mock "mymodule.datetime" as a class. Datetime's eponymous class structure only adds to the confusion.

I ended up with help from tos9 (Julian Berman) on the #python-testing IRC channel, and used this code in my test class:

def setUp(self):
    datetime_patcher = mock.patch.object(
        my_module.datetime, 'datetime', 
        mock.Mock(wraps=datetime.datetime)
    )
    mocked_datetime = datetime_patcher.start()
    mocked_datetime.today.return_value = datetime.datetime(2012, 6, 16)
    self.addCleanup(datetime_patcher.stop)

Here, mock.patch.object is being used to patch the datetime attribute (the class) of the datetime import in my module. It replaces it with a mock, one that wraps the real datetime class. Here, "wraps" means that anything not explicitly changed on the mock is proxied to the real datetime class, so most of our functionality is in place. We change the return value of today() to be a specific date, accomplishing our goal.

If you haven't seen it before, addCleanup() is a new feature of unittest in 2.7. Instead of writing a tearDown method in which you clean up all the stuff you did in setUp, you can register callables with addCleanup, and they will be called to clean up at the end of tests. Because you can register as many as you like, it's easier to modularize your setup and teardown code.

BTW, Julian also has a clever decorator to auto-register the cleanup functions for patches, and has packaged it into a mixin: ivoire/tests/util.py. Check it out.

tagged: , » 11 reactions

Comments

[gravatar]
Wolfgang Schnerring 11:56 PM on 13 Sep 2012

Speaking of collecting and resetting mock's patches, I've taken to writing this down a little differently (see http://pypi.python.org/pypi/gocept.testing#mock-patch-context), and while there has been talk of including something like that upstream (http://code.google.com/p/mock/issues/detail?id=30), it hasn't happened yet, unfortunately, because I think it's really handy.

[gravatar]
Jonathan Hartley 10:43 AM on 14 Sep 2012

When I've had difficulty mocking out a single method on datetime.datetime, I've changed the code-under-test to call a local method "get_now", which simply returns datetime.datetime.utcnow(). Then my tests can easily mock out 'get_now' without affecting anything else on datetime.datetime.

If you are able to change the code-under-test, this seems a simple solution.

[gravatar]
Ned Batchelder 10:55 AM on 14 Sep 2012

@Jonathan, that's another good solution. Another idea was to use a DI-like argument to the class which is the date to use, defaulting to today(). I'm glad to have found the mocking solution, but maybe one of these is better in the long run.

[gravatar]
Adam Skutt 6:51 AM on 19 Sep 2012

It's a cute technique, but a lousy idea. As bad ideas go, this is at least an 8/10. By doing this, you're masking bugs in both your test cases and your actual code, where they cannot properly handle the fact that the value returned by datetime.datetime.today() can change from call to call. One way to solve this is to pass the date as an argument, another is to just be clever in constructing the test.

Generally, though, it's a bad idea to mock out side-effects that your code must handle correctly in order to be defect free. It should only be done when no reasonable alternatives exist.

[gravatar]
Jonathan Hartley 8:14 AM on 19 Sep 2012

@Adam: Hey, I don't yet understand. Can you help get me up to speed?

It seems to me that we constantly cause tests to inject known, constant values into code-under-test, where usually, at run-time, a value would be obtained from some source that produced non-constant values. I agree that mocking should be used judiciously, but are you saying that mocking out the source of a datetime is any worse that mocking out, say, a database call?

[gravatar]
Adam Skutt 8:57 AM on 19 Sep 2012

Yes, since it will mask bugs more easily: simply calling datetime.today() twice as part of the same operation is quite likely a bug. Few applications run the same query twice as part of the same operation. Moreover, relying on the query results to be the same may not be a bug, if running at the proper transaction isolation level and performing the proper set of operations.

That being said, I wouldn't generally mock a database either, as properly emulating all of its semantics is rather difficult. If all of your transactions are simple, then it's not entirely unreasonable, but I'm not sure if it really buys you anything either.

Certainly, most of the relational database schemas I've worked with have triggers and complicated constraint tests and the like, so merely testing the application made the right DB API calls isn't enough to ensure the operation completed successfully.

[gravatar]
Ned Batchelder 9:05 AM on 19 Sep 2012

@Adam: you seem to be assuming that this is the only test I make of the code in question. Often complex code requires a number of tests to address different aspects of its behavior.

But this is an interesting conversation. Adam, what would you mock?

[gravatar]
Adam Skutt 10:10 AM on 19 Sep 2012

It doesn't matter how many other tests you're doing, any test that involves holding the current date constant is suspect. It's indictive of a design and/or logic issue in the code under test.

As for what should be mocked, generally there's a few valid reasons:
* When access to the real thing is not possible, such as a unique or shared piece of hardware or a web service without a test mode / partition.
* When use of the real thing is too costly, such a web service that bills even for testing, or a piece of hardware that's rather expensive to operate or would be damaged in the test.
* When coaxing the desired behavior out of the real thing is too difficult; this is common when testing responses to error conditions. Generally though, this must be done with great care because outputs that are hard to generate are also hard to emulate correctly.
* When the operation under test normally requires privilege.
* When political issues make it impossible to create a proper testing environment, then mocks are certainly better than nothing.

There might be a few other corner cases, but those certainly cover the bulk of things.

[gravatar]
Jonathan Hartley 12:09 PM on 19 Sep 2012

@Adam. Thanks for expanding on your thoughts.

It does make sense, now that you say it, that a bug where datetime.now() was called multiple times might by masked by mocking to return a fixed value. Thanks for enlightening me.

It sounds like I thoroughly agree with your mocking philosophy when I'm writing acceptance tests (i.e. top-level black box testing.) But when I'm writing unit tests, I do use mocking more liberally.

I might mock out complex called code to produce known return values to the code-under-test with minimal test setup. This makes it easy to quickly write many simple tests. I'd also mock to prevent the code-under-test from hitting the filesytem, database or network, purely for performance reasons. Then my 'many unit tests' still run in seconds or less, so can be used not just for pre-commit checks, but also for real-time feedback while editing.

I recognise there are dangers in tests which mock out too much, and would prefer a code design which didn't require it, but if that isn't forthcoming, I think the advantages of judicious mocking-for-convenience are, for me, too great to rule it out altogether.

Can I ask, for my education, how would you test, for example, a function that has different behaviour on different days of the week? You could design the code-under-test to accept the current date as a parameter, or a flag to select behavior, or make the behaviour polymorphic, but haven't you then just moved the call to datetime.now() up into the caller or object factory? Don't the tests for the caller (factory) now face the same problem of wanting to mock datetime.now()? Or am I being dumb?

Thanks for chatting me through this.

[gravatar]
Jonathan Hartley 12:31 PM on 19 Sep 2012

The more I think about this, the more I think the distinction between acceptance versus unit tests is of importance here. Adam said "it doesn't matter how many other tests you write" - but I think that isn't true if the other tests are of a different kind, i.e. if they include acceptance tests. If a small number of acceptance tests verify that functionality "really works" without mocking, then a large number of unit tests can then relatively safely mock things out with greater abandon, to test edge cases and permutations of inputs. It might not be watertight, but for my way of working it's a nice compromise between strict correctness and convenience.

[gravatar]
Roberto Guerra 6:41 PM on 28 Sep 2012

I agree with Jonathan. The test you want to do is a system test(or whatever people call it these days). Unit tests are focused and small. I would create a date() function that generates whatever date I want, then I can create dates for my test without having to mock out datetime. I can stub out my_date() to return the date I need and this way I can control what I test (without needing to fire a missile everytime). If its hard to test, then it the design is wrong and it should force us to change your design so we can test it.

Add a comment:

name
email
Ignore this:
not displayed and no spam.
Leave this empty:
www
not searched.
 
Name and either email or www are required.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.