Error Reporting in WSGI Applications
Thanks to an google alert on “pygments” I stumbled accross gluon/web2py, some sort of “enterprise ready framework”. It’s definitively not a framework I would use myself for countless reasons but there is one thing on the feature list which I found interesting. Apparently gluon files tickets for tracebacks in the database. While it’s a terrible idea to put that data into the very same database all your application data goes (what happens if the DB is down?) it’s a different approach to Django which sends mails on errors.
Two days ago someone mentioned the paste WSGI middleware which sends emails on tracebacks, similar to the way Django does that. I’m not a fan of error reporting middlewares because I think that should go into the application. On an server error (caused by lost database connections, programmer errors etc.) you should not present the user a black-and-white error page or display an inline traceback like most PHP applications do. We can do better!
First of all there is a module in the standard library everybody seems to forget about. It’s called “logging” and does exactly that — it logs errors. I don’t know why so many people miss it or just don’t use it, but it’s really one of the good things in the python standard library.
Why is it so good? It’s extensible and configurable. The idea is that the application gets itself a logger and logs into that logger (debug messages, information messages, warnings, errors or tracebacks). Independently from the logging there are logger handlers which do something with the logged messages. For example you can tell the logger to log everything except of debug output into a rotated file and mail all errors to some mail addresses.
In the Werkzeug wiki there is a wiki page about error handling in WSGI applications using the logger module. Total amount of code needed for a simple WSGI application with logging is about ten lines or so. And it’s flexible enough to integrate in every WSGI application setup, no need to solve that in a middleware where you don’t have access to your application’s config or whatever.
And the best thing about it is that you can configure the way errors are handled. Like I mentioned before gluon creates entries in the database for logged errors. I wrote a small logger handler that does the same but for an external trac. Whenever an application error occours the logger checks if there is already a ticket for that error, if not it creates one. For every new occurrence of that bug it will create a comment in that ticket.
If you want to try it out yourself, I added the code to the sandbox repository.
Update: I also created a trac hack for it.
> I don’t know why so many people miss it or just don’t use it…
> Why is it so good? It’s extensible and configurable.
That’s also why it’s so bad: it’s so extensible and configurable that’s it’s far too slow for high-performance websites. So please, when you make the latest and greatest web framework, go ahead and add support for the logging module, go ahead and make it the default, but make it easily overridable at the same time.
Comment by Robert Brewer — Friday, January 18th, 2008 @ 11:02 pmThat’s very true. But in a production environment the calls to the actual logger can be reduced to the cases where you actually log an error or exception. For example you can create methods for log / info / warn or whatever and if __debug__ is False you replace them with a no-op lambda.
My point with that post was mostly to not reinvent exception reporting when the stdlib can do that. And that one logger.exception call in the very last except of your WSGI application won’t harm the performance of the application at all.
Comment by Armin Ronacher — Saturday, January 19th, 2008 @ 12:38 amBTW, the documentation for logging has recently been enhanced by a tutorial section, thanks to a GHOP student, see http://docs.python.org/dev/library/logging.
Comment by Georg — Saturday, January 19th, 2008 @ 9:41 amI would like to make two corrections: 1) You say that web2py “files tickets for tracebacks in the database”. No. web2py files tickets for any error that occurs and that is not explicitly catched by the application, including db tracebacks. 2) You also say “it’s a terrible idea to put that data into the very same database all your application data goes (what happens if the DB is down?” I agree and web2py in fact does not do that. web2py tickets are stored in files and they can be accessed even if the db is down. Every app has its own seperate ticketing system provided by web2py.
You also say that you would not use web2py for countless reasons. It would be helpful if you were to list those reasons. Perhaps some of them are consequence of misinformation and I could try convince you that, in fact, you should use web2py. ;-)
One common misconception is to assume limitations because of the GPL license. If you read the web2py license carefully, as long as you do not copy web2py code into your web2py app web2py makes no claims on your app and you can even distribute it in closed form (something that web2py builds for you) and charge for it.
Comment by Massimo — Sunday, January 20th, 2008 @ 6:46 am… another misconception about web2py is that you have to use the web interface for development and for database administration. You can but you are not forced to.
Comment by Massimo — Sunday, January 20th, 2008 @ 7:09 amIn general I’m not a framework lover, even though I’m happy Django user. Mainly because I have a couple of use cases I can solve with Django and because the Django developers do a very good job at keeping the source clean, readable and maintained.
When I looked at the web2py sources I stopped looking at it pretty soon when I found out that it’s bypassing the python import system for controllers and models, that it comes in a nonstandard distribution I can’t deploy on a server that easy and that it’s reinventing the wheel everywhere.
For custom applications I don’t want to miss SQLAlchemy any more or all those template engines available for Python. So no, for things I have to bootstrap fast and quickly I would chose Django because of the community and the number of developers, and for custom applications I go with Werkzeug, especially because I can fix it when it breaks ;-)
Comment by Armin Ronacher — Monday, January 21st, 2008 @ 7:22 pmIf you download the web2py source code instead of the windows/mac executables, you can deploy it as any other python framework. In fact web2py is wsgi compliant, packages the paste httserver (same as pylons) if you want to use mod_proxy, and a wsgi handler like Django if you prefer mod_wsgi. You can install any python module using easy_install and import it from models/view/controllers as you would in Django or Pylons. web2py allows you to bytecode-compile your controllers/views so that 1) no parsing of templates is done in production mode, and 2) you can distribute your apps in semi-closed source; this is why it bypasses the normal import for its own controllers. A small prize to pay for a much faster framework.
Comment by Massimo — Friday, January 25th, 2008 @ 3:25 pm