Pages tagged as ‘wsgi’

Error Reporting in WSGI Applications

January 18th, 2008

Thanks to an google alert on “pygments” I stumbled accross gluon/web2py, some sort of “enterprise ready framework”. It’s definitively not a framework I would use myself for countless reasons but there is one thing on the feature list which I found interesting. Apparently gluon files tickets for tracebacks in the database. While it’s a terrible idea to put that data into the very same database all your application data goes (what happens if the DB is down?) it’s a different approach to Django which sends mails on errors.

Two days ago someone mentioned the paste WSGI middleware which sends emails on tracebacks, similar to the way Django does that. I’m not a fan of error reporting middlewares because I think that should go into the application. On an server error (caused by lost database connections, programmer errors etc.) you should not present the user a black-and-white error page or display an inline traceback like most PHP applications do. We can do better!

First of all there is a module in the standard library everybody seems to forget about. It’s called “logging” and does exactly that — it logs errors. I don’t know why so many people miss it or just don’t use it, but it’s really one of the good things in the python standard library.

Why is it so good? It’s extensible and configurable. The idea is that the application gets itself a logger and logs into that logger (debug messages, information messages, warnings, errors or tracebacks). Independently from the logging there are logger handlers which do something with the logged messages. For example you can tell the logger to log everything except of debug output into a rotated file and mail all errors to some mail addresses.

In the Werkzeug wiki there is a wiki page about error handling in WSGI applications using the logger module. Total amount of code needed for a simple WSGI application with logging is about ten lines or so. And it’s flexible enough to integrate in every WSGI application setup, no need to solve that in a middleware where you don’t have access to your application’s config or whatever.

And the best thing about it is that you can configure the way errors are handled. Like I mentioned before gluon creates entries in the database for logged errors. I wrote a small logger handler that does the same but for an external trac. Whenever an application error occours the logger checks if there is already a ticket for that error, if not it creates one. For every new occurrence of that bug it will create a comment in that ticket.

If you want to try it out yourself, I added the code to the sandbox repository.

Update: I also created a trac hack for it.

Tagged as: wsgi, python | 7 Comments »

New Stuff in Werkzeug (and the WSGI World)

January 16th, 2008

Unfortunately I’m very busy lately so there are few updates on Werkzeug and all the other libraries I personally contributed (and there are even some patches in my Mail queue I have to apply after reviewing) but that doesn’t mean that there is no progress :-)

There is actually quite a lot of new stuff between the 0.1 release and now. Werkzeug tries to implement stupid stuff you reimplement in every second application in a way that you can use it with minor modifications in your application. Because many of those features only come up if you have implemented them often in your own applications they didn’t make it into 0.1. Thanks to all the early adopters we now have cool new stuff that was implemented because there was need for it.

For example Werkzeug now has RFC-compliant Etag and Cache-Control parsing. You can also generate etags automatically for responses and make them conditional for some requests. The utility module was extended with many new stuff that fix limitations in the standard library or implement long missed functions like finding modules in a package (very useful if you want to automatically register controllers), importing modules by a string (useful if your URL map endpoints are (partial) import paths), generating URLs like trac does by calling an Href object, functions to fix URLs similar to the way Firefox fixes them, dumping and parsing HTTP dates, loading and dumping cookies with a simple function call (and support for http only) and probably a lot more before the actual 0.2 release.

Another cool thing is (unrelated to Werkzeug) that more and more cool modules come up that make web development a charm. While babel is available for quite a while I haven’t really used it until last weak and it’s really great. Together with Werkzeug’s routing system and the ability to do multiple inheritance in SQLAlchemy there are so cool ways to do web development of internationalized applications. I know the last sentence doesn’t make sense without the context, so I guess I have to blog about that. sooo freaking cool.

Form handling in Python is still a bit strange if you are not using django’s newforms (I know that there is formencode but somehow it isn’t what I’m looking for) but there is now WTForms which looks promising. With some more small changes it could become a very cool form handling system (for example I’m missing some default validators at the moment and I’m not completely sure how to pass choices to a select box in a per form instance basis). WTForms was derived from an application that is already in production so it solves already many problems nicely. It’s a library I want to watch closely the next weeks.

And I think that approach should become the way Werkzeug is developed in the future. Implement features a release earlier and mark them as “under consideration” if they are not yet used in production applications. If you adopt them early you can give feedback and we can improve it to the next Werkzeug release and streamline the API.

What’s to do until the next Werkzeug release? Georg is currently working on making the sphinx documentation builder independent from the CPython documentation so that other projects can use it too. I then want to semi-automatically build an API documentation for Werkzeug and combine them with hand written rst pages for the Werkzeug 0.2 documentation. I got some feedback for the Werkzeug docs and looks like they are a bit too chaotic and misleading. Especially getting started with Werkzeug is still too complicated so I hope we can address this with a new documentation that combines automatically generated documentation with tutorials.

The documentation tool could probably be useful for other projects too, I guess Georg will drop some lines in his blog once it’s ready.

Updates regarding Jinja will be up shortly, there is currently a branch developed by Lakin Wecker to speed up Jinja template evaluation. And if you already know what GHRML/XAML will gonna be: I will try to get that running this weekend.

That’s it for the moment ;-)

Tagged as: werkzeug, wsgi, jinja, python, update | No Comments »

convert a Request.write() into a WSGI yield

I tried that for a long time now using python generators with yield and send threads and much more. But I never got anything that looked easy to understand and worked at the same time. The problem basically occurs if you have an old python web application that has some sort of request object with a write method that directly writes to the output stream of the server interface (sys.stdout or some sort of fastcgi/mod_python output stream object) and you want to convert the application to WSGI. Take the following piece of code from an imaginary legacy application:

def old_application(request):
    from time import sleep
    request.header('Content-Type: text/html')
    for x in xrange(10):
        request.write(str(x) + ' ')
    request.flush()
    request.write('<br>And the next flush takes another second<br>')
    request.flush()
    sleep(1)
    request.write('And done!')

Say we want to convert that into a WSGI application with those semantics:

def wsgi_application(environ, start_response):
    from time import sleep
    start_response('200 OK', [('Content-Type', 'text/html'])
    yield ' '.join(str(x) for x in xrange(10)) + ' '
    yield '<br>And the next flush takes another second<br>'
    time.sleep(1)
    yield 'And Done!'

The main problem is the time.sleep and all those flush calls. That means we cannot just buffer the contents but convert the into a generator on the fly. What we need to get that done are either coroutines, greenlets or two threads that communicate with each other. The easiest and I guess also fastest approach are greenlets.

Here a function that converts an old legacy application like above into a WSGI application with the same semantics:

from py.magic import greenlet

class Request(object):

    def __init__(self, environ):
        self._parent = greenlet.getcurrent()
        self.environ = environ
        self.status = '200 OK'
        self.headers = []

    def header(self, item):
        self.headers.append(tuple(item.split(':', 1)))

    def write(self, text):
        self._parent.switch(('write', text))

    def flush(self):
        self._parent.switch(('flush', None))

def convert_app(application):
    def wsgi_app(environ, start_response):
        request = Request(environ)
        buffer = []
        headers_sent = []

        def flush():
            if not headers_sent:
                start_response(request.status, request.headers)
                headers_sent.append(True)
            data = ''.join(buffer)
            if data:
                yield data
            del buffer[:]

        def run():
            application(request)
            request.flush()

        g = greenlet(run, request._parent)
        while 1:
            rv = g.switch()
            if not rv:
                break
            signal, value = rv
            if signal == 'flush':
                for item in flush():
                    yield item
            elif signal == 'write':
                buffer.append(value)
    return wsgi_app

Now that’s a bunch of code. Let’s go step by step through it. The first thing we do is creating a request class. This class should resemble the old request object as much as possible. All methods can work like they did before, the only differences are the write and flush methods. Those switch back to the parent greenlet (which is the greenlet that generated the request object, usually the main greenlet) and send some data to it (namely the name of the method and the argument). Whenever python encounters this statement it stops the execution and goes back to the point that switched into this greenlet. This point is in our example in a loop that generates a generator for our WSGI application.

That leads us to the convert_app function that is passed and old legacy application and returns a new WSGI application. Inside this new WSGI application we create a new request object, pass it the WSGI environment and create some objects and functions we need so that we can process the data from the greenlets and convert it into a valid WSGI response: A buffer for unsent data, a list we use a sentinel for sent data, a flush method that returns a generator with the data from the buffer, starts the response and cleans the buffer, and a run method that invokes the old application and calls request.flush() after the application has finished so that we don’t have to do that in the application itself.

The mainloop after that basically switches between application greenlet and main greenlet until the return value of switch is None (that is the case if the application closed or someone switched into the main greenlet without arguments, which we don’t do). If that is the case we return, otherwise we check if it’s a flush or write call and handle that.

To launch the converted application with wsgiref all we have to do are those three lines of code:

from wsgiref.simple_server import make_server
srv = make_server('localhost', 5000, convert_app(old_application))
srv.serve_forever()

Basically greenlets would make it possible to host mod_python applications inside arbitrary WSGI servers. Maybe in the future someone writes a module that allows us to convert some of the mod_python applications into WSGI applications without touching existing application code.

November 18th, 2007 | Tagged as: snippets, wsgi, python | One Comment »

What about a PHP-like Python Application?

June 12th, 2007

Actually there are so many people that want to have a PHP like Python, so why not give it a try? Using Mako and Werkzeug it’s easy to create such an application.

Basically all what I did is processing *.mako as Mako template that knows about the current request and response, and serving everything else as static file. Inside the templates you have access to GET, POST, FILES, COOKIES, REQUEST and RESPONSE and can use all of mako’s power.

Example template:

<%
    data = GET.get('data', 'World')
%>
<title>Hello ${data|h}</title>
<link rel="stylesheet" href="style.css" type="text/css">
<h1>Hello ${data|h}</h1>
<p>
  Welcome on the PHP like Python application.
</p>
<p>
  Say hello <a href="?data=Daddy">daddy</a> or show
  <a href="missing">a missing page</a>. There is also
  <a href="special.mako">a special page</a>.
</p>

The full sourcecode is available here: phplikepy

It needs the SVN version of Werkzeug, Mako and Python2.4 or Python 2.3 with wsgiref.

Tagged as: werkzeug, wsgi, antizen, php, python | No Comments »

Knight of WSGI

June 5th, 2007

I have that title in the German Python Forum for some time already and now I wanted to have an avatar for that. So I tried to draw one, unfortunately I’m not that good at drawing so the result is not the best. Still. Here the full version:
a knight with a WSGI banner
and here the small one for the forum:
same as above just in smaller, so small that you can't read the banner any more

Tagged as: wsgi, art, python, update | One Comment »

cogitations driven by wordpress