Thursday 13 September 2012 — This is close to 11 years old. Be careful.
Mocking is a great way to isolate your code from distracting dependencies while testing. But, it can be an arcane art unto itself. Today I wrote a test for code that uses the current date. While testing, it can’t use the actual current date, because the test will produce different results on different days.
A solution is to mock out the datetime.datetime.today() function to return a known fixed date. But a few factors complicate matters. First, datetime.datetime is written in C, so Mock can’t replace attributes on the class, so you can’t simply mock out just the today() function.
Second, I want to mock just one function in the module, not the whole module. There are a few suggestions of how to do this out there: Michael Foord wrote about Partial mocking in the Mock docs, William John Bert showed another way, and of course, a Stack Overflow question about it.
None of these worked for me, perhaps because of subtle differences between my code under test and theirs. When mocking, it’s critical to mock at the appropriate place. If your module has “import datetime”, then you need to mock “mymodule.datetime” as a module. If instead you have “from datetime import datetime”, then you need to mock “mymodule.datetime” as a class. Datetime’s eponymous class structure only adds to the confusion.
I ended up with help from tos9 (Julian Berman) on the #python-testing IRC channel, and used this code in my test class:
datetime_patcher = mock.patch.object(
mocked_datetime = datetime_patcher.start()
mocked_datetime.today.return_value = datetime.datetime(2012, 6, 16)
Here, mock.patch.object is being used to patch the datetime attribute (the class) of the datetime import in my module. It replaces it with a mock, one that wraps the real datetime class. Here, “wraps” means that anything not explicitly changed on the mock is proxied to the real datetime class, so most of our functionality is in place. We change the return value of today() to be a specific date, accomplishing our goal.
If you haven’t seen it before, addCleanup() is a new feature of unittest in 2.7. Instead of writing a tearDown method in which you clean up all the stuff you did in setUp, you can register callables with addCleanup, and they will be called to clean up at the end of tests. Because you can register as many as you like, it’s easier to modularize your setup and teardown code.
BTW, Julian also has a clever decorator to auto-register the cleanup functions for patches, and has packaged it into a mixin: ivoire/tests/util.py. Check it out.
If you are able to change the code-under-test, this seems a simple solution.
Generally, though, it's a bad idea to mock out side-effects that your code must handle correctly in order to be defect free. It should only be done when no reasonable alternatives exist.
It seems to me that we constantly cause tests to inject known, constant values into code-under-test, where usually, at run-time, a value would be obtained from some source that produced non-constant values. I agree that mocking should be used judiciously, but are you saying that mocking out the source of a datetime is any worse that mocking out, say, a database call?
That being said, I wouldn't generally mock a database either, as properly emulating all of its semantics is rather difficult. If all of your transactions are simple, then it's not entirely unreasonable, but I'm not sure if it really buys you anything either.
Certainly, most of the relational database schemas I've worked with have triggers and complicated constraint tests and the like, so merely testing the application made the right DB API calls isn't enough to ensure the operation completed successfully.
But this is an interesting conversation. Adam, what would you mock?
As for what should be mocked, generally there's a few valid reasons:
* When access to the real thing is not possible, such as a unique or shared piece of hardware or a web service without a test mode / partition.
* When use of the real thing is too costly, such a web service that bills even for testing, or a piece of hardware that's rather expensive to operate or would be damaged in the test.
* When coaxing the desired behavior out of the real thing is too difficult; this is common when testing responses to error conditions. Generally though, this must be done with great care because outputs that are hard to generate are also hard to emulate correctly.
* When the operation under test normally requires privilege.
* When political issues make it impossible to create a proper testing environment, then mocks are certainly better than nothing.
There might be a few other corner cases, but those certainly cover the bulk of things.
It does make sense, now that you say it, that a bug where datetime.now() was called multiple times might by masked by mocking to return a fixed value. Thanks for enlightening me.
It sounds like I thoroughly agree with your mocking philosophy when I'm writing acceptance tests (i.e. top-level black box testing.) But when I'm writing unit tests, I do use mocking more liberally.
I might mock out complex called code to produce known return values to the code-under-test with minimal test setup. This makes it easy to quickly write many simple tests. I'd also mock to prevent the code-under-test from hitting the filesytem, database or network, purely for performance reasons. Then my 'many unit tests' still run in seconds or less, so can be used not just for pre-commit checks, but also for real-time feedback while editing.
I recognise there are dangers in tests which mock out too much, and would prefer a code design which didn't require it, but if that isn't forthcoming, I think the advantages of judicious mocking-for-convenience are, for me, too great to rule it out altogether.
Can I ask, for my education, how would you test, for example, a function that has different behaviour on different days of the week? You could design the code-under-test to accept the current date as a parameter, or a flag to select behavior, or make the behaviour polymorphic, but haven't you then just moved the call to datetime.now() up into the caller or object factory? Don't the tests for the caller (factory) now face the same problem of wanting to mock datetime.now()? Or am I being dumb?
Thanks for chatting me through this.
Add a comment: