Template coverage measurement

Sunday 8 June 2014

As the maintainer of coverage.py, it’s always been intriguing that web applications have so much code in template files. Coverage.py measures Python execution, so the logic in the template files goes un-measured.

(Of course, in a web application, there’s even more Javascript code, which coverage.py also can’t help with, but there are other tools that measure Javascript coverage.)

Recently I started experimenting with measuring templates as well as pure Python code. Mako templates compile to Python files, which are then executed. Coverage.py can see the execution in the compiled Python files, so once we have a way to back-map the lines from the Mako output back to the Mako template, we have the start of a usable Mako coverage measurement.

This Mako experiment is on the tip of the coverage.py repo, and requires some code on the tip of Mako. The code isn’t right yet, but it shows the idea. Eventually, this should be a plugin to coverage.py provided by Mako, but for now, we’re just trying to prove out the concept.

If you want to try the Mako coverage (please do!), configure Mako to put your compiled .py files someplace convenient (like a mako/ directory in your project), then set this environment variable:

$ export COVERAGE_MAKO_PATH=/mako/

Jinja also compiles templates to Python files, but Django does not. Django is very popular, so I would like to support those templates also. Dmitry Trofimov wrote dtcov to measure Django template coverage. He does a tricky thing: in the trace function, determine if you are inside the Django template engine, and if so, walk the stack and look at the locals to grab line numbers.

As written dtcov looks too compute-intensive to run on full-sized projects, but I think the idea could work. I’m planning to experiment with it this week.


James Mills 3:50 AM on 9 Jun 2014

Hi Ned! This is pretty cool stuff really! Be great to see more of this as sadly a lot of UI functionality in Web Apps are/is driven by templates :/

Randy Syring 1:42 PM on 9 Jun 2014

Good stuff! Glad to see you are working on this. We use Jinja2 mostly and would love to see this functionality become production ready.

Add a comment:

Ignore this:
Leave this empty:
Name is required. Either email or web are required. Email won't be displayed and I won't spam you. Your web site won't be indexed by search engines.
Don't put anything here:
Leave this empty:
URLs auto-link and some tags are allowed: <a><b><i><p><br><pre>.