![[*]](dodeca3_100.gif) | Ned Batchelder : Blog | Code | Text | Site Running code at Python startup » Home : Blog : January 2010 |
Running code at Python startupThursday 14 January 2010 I recently had need to run a piece of Python code before every invocation of the interpreter. In my case, it was to enable coverage testing of processes spawned by a test suite. Certain tests would run Python code in a subprocess, and I wanted to measure coverage within that process. I added a function to coverage.py that is suitable for executing on startup. It examines the environment to determine whether to start measuring coverage. (This code isn't released yet, but if you want to try it, it's on the trunk at bitbucket.) Perl has a command line switch (-M) for this, as does Ruby (-r), but Python doesn't. I came up with two ways to do it for Python, both depending on the site module which is imported automatically on startup. The first way is to create a sitecustomize.py file on the path somewhere. This module will be imported automatically by site.py. Any lines you add to that file are executed when Python starts: import coverage This is simple, but editing an existing sitecustomize.py could be awkward, and you may not even have permissions to do it. The second way is to create a .pth file with an executable line. Usually lines in .pth files are directory names, but if the line starts with "import ", it is executed (what a hack!), so you can create a .pth file like this: import coverage; coverage.process_startup() This has the advantage that you can create a new file without worrying about touching existing files, all .pth files will be examined and interpreted. Unfortunately, you might have to give the file a strange name, since they are executed alphabetically, and a package you need may only be available after its .pth file has been seen. In my case, I named my .pth file zzz_coverage_process_start.pth to be sure it ran after coverage was added to the path. Neither of these feels clean or elegant. I'm wondering if there are better techniques I'm overlooking? Python has a clean way to register functions to be called at process exit (atexit), which has been designed so that multiple authors of exit functions don't need to cooperate to avoid interfering with each other. It'd be nice to have a similar facility for startup functions.
tagged:
python» 11 reactions | |
Comments
Doesn't Python have `python -m module`?
@Tobu, it does, but it merely runs the module. It isn't a way to run the module before running your own program.
Would the PYTHONSTARTUP environment variable meet your need? See http://docs.python.org/using/cmdline.html#environment-variables
PYTHONSTARTUP does precisely what I need, but *only* for interactive prompts. I need the same thing, but for non-interactive invocations of the interpreter also.
But `python -m cProfile myscript.py` runs myscript.py after it loads the profiler. Is that because there is special code in the cProfile module to parse that parameter, load it, and run it? Would that same code work for your scenario?
@Andy: the cProfile module reads the myscript.py argument and runs the Python file. I could write a similar module, but it doesn't really solve my problem. I have a test suite with many invocations of the Python interpreter in subprocesses. I'd have to change the way Python is invoked in those cases to use my new module. I was looking for a way to transparently invoke a module before the real program starts.
Python (at least version 2.6) supports a usercustomization feature. You can put a usercustomize.py module in your USER_SITE directory. For my Linux python install this is ~/.local/lib/python2.6/site-packages.
You can run this to find out where USER_SITE is for your Python installation.
python -c "import site; site._script()"
The usercustomize module is imported late during; after sitecustomize.
About the file names sorting problem ; there is a hint commonly used under Unix : prepend by a number instead of an alphabetic sequence.
01_root_startup.pth
...
50_user_startup.pth
...
99_coverage_process_start.pth
I find this cleaner, but is a matter of taste.
I just ran into this issue myself. The problem with the .pth files solution is that this works differently on different platforms (apparently). On Linux you can put them anywhere on PYTHONPATH, but on Windows you have to put them in certain pre-defined global locations. This last makes it very difficult to do this sort of thing without risking screwing up some other Python process.
My own solution was as follows: use your sitecustomize.py one (using a temporary directory to store it), but add the following code at the end so that the real sitecustomize.py is also loaded:
import os, sys currDir = os.path.dirname(__file__) sys.path.remove(currDir) del sys.modules["sitecustomize"] try: import sitecustomize finally: sys.path.insert(0, currDir)Yes it's a hack, but it only needs doing once for any sitecustomize setup.There is also the -i option, which executes the script given after this option before entering interactive mode. That script can contain any code you like, and that is what I use to customize certain automatic python calls. Also, you can make use of temporary files if the code is not too long, like so:
Oops, haven't used this in a while, and forgot to pipe the actual script to run to the interpreter. So, looking at my scripts, it seems actually more sloppy and hackish than I remembered, but here it is:
and it seems this also print a lot of ">>" while executing script.py (the actual script you want to run). I remember getting rid of that somehow, but I can only find redirection to /dev/null, as I didn't care abotu output.Add a comment: