Wednesday 9 March 2011 — This is almost 14 years old. Be careful.
The Django development server is great: it comes in the box, serves Django, auto-restarts on source code changes, and now even color-codes the log lines based on the status returns.
But it isn’t multi-threaded, which normally wouldn’t be a problem for a development server, unless you’re writing Ajax interactions, and these days, who isn’t?
The Django team has declared that they will not offer a multi-threaded development server, for good or bad, so we are left to our own devices. James Aylett wrote django_concurrent_test_server which offers multi-threading and forking, though I haven’t tried it. David Cramer offers django-devserver which seems to offer a number of interesting new logging options also. Many developers simply use other “real” web servers, like Apache or gunicorn, but those don’t detect code changes, and often don’t provide stdout for debugging with.
I wanted multi-threading on a project but I didn’t want to use a big real web server, and didn’t want to install a new Django app and modify settings.py, so I adapted the patch from the closed Django bug ticket to create threadedmanage.py:
#!/usr/bin/env python
#
# A clone of manage.py, with multi-threadedness monkeypatched in.
import os, sys
from django.core.management import execute_manager
try:
import settings # Assumed to be in the same directory.
except ImportError:
sys.stderr.write(
"Error: Can't find the file 'settings.py' in the directory containing %r. "
"It appears you've customized things.\n"
"You'll have to run django-admin.py, passing it your settings module.\n"
"(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n"
% __file__
)
sys.exit(1)
def monkey_patch_for_multi_threaded():
# This monkey-patches BaseHTTPServer to create a base HTTPServer class that
# supports multithreading
import BaseHTTPServer, SocketServer
OriginalHTTPServer = BaseHTTPServer.HTTPServer
class ThreadedHTTPServer(SocketServer.ThreadingMixIn, OriginalHTTPServer):
def __init__(self, server_address, RequestHandlerClass=None):
OriginalHTTPServer.__init__(self, server_address, RequestHandlerClass)
BaseHTTPServer.HTTPServer = ThreadedHTTPServer
if __name__ == "__main__":
monkey_patch_for_multi_threaded()
execute_manager(settings)
Now I can run “./threadedmanage.py runserver ..” and get the standard development server, but with multiple threads.
The usual caveats apply: This isn’t a real web server, don’t use it in production. Your code likely has threading issues, please fix them. I’m pretty sure there are good reasons not to use this code, but it’s working well for me.
Comments
Prior to mod_wsgi version 3.0, it did indeed by default block writing to stdout for logging. This was because doing so breaks WSGI application portability. Okay, it was only CGI/WSGI bridges that were the problem, but the intent was to still try and get people not to include debugging statements which would effectively preclude use of CGI. This restriction could be disabled in a number of ways, but all the same, overall people seemed not to care or were lazy and instead just complained about the restriction rather than fix their code. As such the restriction was removed in mod_wsgi 3.0 and you can write as much non portable code as you want. See:
http://blog.dscpl.com.au/2009/04/wsgi-and-printing-to-standard-output.html
As to code reloading, this doesn't apply just to Apache/mod_wsgi, but for Apache/mod_wsgi, although it is not the default, it is possible to enable code reloading on all code changes. Thus it can be used during development, with either multithreaded or multiprocess configurations. The only restriction here is that you must use daemon mode on UNIX systems and cant use embedded mode. You also need to hook in a bit of monitoring code to enable this ability. See:
http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html
http://blog.dscpl.com.au/2009/02/source-code-reloading-with-modwsgi-on.html
Doing development inside of Apache/mod_wsgi using this feature not only allows you to make use of multithreaded or multiprocess configurations, it also avoids problems where stuff works in the development server and not in production due to the fact that the development server preloads a lot of stuf where as real WSGI servers, because of how the Django WSGI interface works, lazily loads stuff. This time and time again seems to cause problems for people around order of Python module imports and import cycles. Thus there is quite a bit of sense to the argument of using Apache/mod_wsgi during development and not just in production or a staging/test environment. See:
http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
try: import settings # Assumed to be in the same directory. except ImportError, exc: import sys import traceback sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__) sys.stderr.write("\nFor debugging purposes, the exception was:\n\n") traceback.print_exc() sys.exit(1)We also use devserver (though we're biased), but without any modules enabled by default. Gives a nice summary of perf, but really we only do it because it hijacks the runserver command and enables threading.Add a comment: