Python’s doctest is very cool technology, letting you write a narrative in documentation that can then be executed directly to test the code. But as cool as it is, I don’t like it very much:
- You can’t run a subset of the tests.
- If a failure happens in the middle of the doctest, the whole thing stops.
- The coding style is stylized, and has to have printable results.
- Your code is executed in a special way, so it’s harder to reason about how it will be executed, harder to add helpers, and harder to program around the tests.
Most importantly, though, doctest seems like an odd way to write tests. Docstrings, and the long sequence of code they encourage, may be good ways to explain what code does, but explaining and testing are two different tasks, and the code you write for each will be different. So why try to serve two masters at once?
Either your expository text will be cluttered with uninformative edge cases, or your tests will merely skim the surface of what your code can do.
I know that doctest can be used independently of the actual docstrings in the code, but then where’s the great advantage? I’d rather use real software engineering tools to write my tests, and the idiomatic way doctest executes code and evaluates results don’t help me.
While I admire the ingenuity that went into doctest, I just don’t find it a good tool for testing real code.