the fruity and personal webpage of armin ronacher
Werkzeug was featured in this week’s django podcast. I was joking in the past that after naming a library after an Japanese temple I want to see people pronounce a German name.
And haha, it payed of. Werkzeug was called the “w-tool” after some attempts to pronounce it :-) So now I uploaded a sound sample for Werkzeug.
Today we have to push Werkzeug 0.3.1. 0.3 and below had a possible cryptographic weakness in the secure cookie that would allow attackers to inject additional information into the cookie. 0.3.1 fixes that and can be downloaded from the cheeseshop.
werkzeug.contrib.securecookie is still an undocumented module but used widely. There is a small API change that removed SecureCookie.new_salt. Also keep in mind that this fix automatically invalidates all existing cookies will be reinitialized with new values on the first request a user causes which usually means that the user is logged out automatically.
We’re sorry for the inconveniences caused.
Some open source projects jump from 0.9 to 0.10. So does Trac, so did Pygments. So please, don’t compare versions with string comparison operators. pkg_resources can parse versions for you so that you can compare them:
>>> from pkg_resources import parse_version
>>> parse_version("0.10") > parse_version("0.9")
True
If you don’t have pkg_resources you have problems getting running one of those anyways, but alternatively you can still split at every dot and try to compare the numbers into integers and falling back to strings:
>>> def parse_version(version):
... def _trynumber(x):
... try:
... return int(x)
... except ValueError:
... return x
... return tuple(map(_trynumber, version.split('.')))
...
>>> parse_version("0.10") > parse_version("0.9")
True
However, how this error happened I have no idea. TracMercurial doesn’t do checks on its own but uses setuptools for it, and that knows how to compare versions.
import sys
from threading import local
_cycles = local()
def cycle(*args):
frm = sys._getframe(1)
key = (frm.f_code, frm.f_lineno)
d = _cycles.__dict__
try:
pos = d[key] = (d[key] + 1) % len(args)
except LookupError:
pos = d[key] = 0
return args[pos]
Works like this:
>>> cycle(1, 2) 1 >>> cycle(1, 2) 2
And yes, that’s thread and stack safe but ugly. Was more or less a “yes it can be done” kind of solution, I don’t encourage anyone to use that :-)
Version 0.3 codename EUR325CAT6 of Werkzeug, the WSGI toolkit was released today. There are many changes from 0.2 and I want to cover some of them here.
Headers, MultiDict and others) raise a custom key error now that’s both a key error and a BadRequest exception. If you are not catching the exception and you have an error handler for http exceptions, that will kick in. So it’s save to do request.args['foo'] and not check if foo was actually transmittedshallow=True now. If shallow is set to True, the request object will raise exceptions if an access to the request object causes the input stream to be read. This makes it possible to use the request object in middlewares now without the fear of breaking applications.dump() which either dumps the local variables or the namespace of an object.Get it from the Cheeseshop while it’s hot.
I was talking about Jinja2 in the past already but that was in the middle of developing it and many things changed in the last weeks. And now I’m proud to announce the first release candidate :)
As I said earlier Jinja2 is Jinja1 without the design mistakes. But what exactly is new and why is it so cool?
It’s dynamic. Unlike Jinja1 template inheritance and inclusion is handled at runtime now. That means you can do something like {% extends master_template %} to extend from the template with the name stored in the “master_template” variable. Previously that was not possible which clearly was a design mistake. This also affects includes (and now imports). Besides the obvious advantage that this is much more flexible, it also reduces the memory usage of Jinja. This dynamic inheritance / including directly lead to an import statement to load macros and variables from other templates.
It’s fast. The sandboxed evaluation mode in Jinja2 is now optional and the evaluation rules are simplified which gave it a huge performance improvement. Jinja2 renders a test template in under 0.04 seconds where the latest Django version needs nearly a second for.
Macros are more useful now. Macros in Jinja1 weren’t that great, but they are now! On the one hand because you can import them with a python like import syntax ({% from 'helpers.html' import input_field, textarea %} or {% import 'helpers.html' as helpers %}) on the other because they have their own namespace now. That means they are easier to debug and understand.
Includes and imports separated. Includes just include the template and render it at the current position, imports pull variables and macros from there. This makes it easier to understand templates as it’s clear from where a variable is coming from.
For loops are even cooler now. Jinja1 already had a pretty neat for loop which supported recursion and an optional else block that was rendered if the sequence was empty. It also featured a special loop variable that allowed the template designer to access the number of the current loop iteration, the total length of the loop and much more. Jinja2 extends that by adding support for loop filtering. The following example only lists the users that are visible or all if the current user is an administrator:
{% for user in users if user.is_visible or request.user.is_administrator %}
<li>{{ user.username|e }}</li>
{% else %}
<li><em>no users online right now</em></li>
{% endfor %}
The advantage of filtering directly in the loop head is that the special loop variable will count proplery (eg: excluding invisible users for counting). Additionally the else block will work as expected.
Better undefined behavior. Jinja1 had a very silent undefined behavior. If a variable was undefined you were able to call it without getting errors or access any attribute. Jinja2 ships three undefined types that make it easier to debug templates. The default undefined types allows you to print the undefined variables (which when printed outputs nothing) and loop over it (works like iterating over an empty list). However every other operatation raises an UndefinedError. Additionally there an undefined type with the same behavior but it prints the name of the variable or attribute missing if it’s printed. The third builtin undefined type is the strict type which doesn’t allow any operation except of testing if it’s undefined which is the closest you can get to the default python behavior. The following doctests shows the behavior of those types:
The normal undefined type:
>>> x = Undefined(name="x")
>>> unicode(x)
u''
>>> x.attribute
Traceback (most recent call last):
...
UndefinedError: 'x' is undefined
The strict undefined doesn’t allow anything except of testing if it’s defined from within a template. You can’t even compare it to an arbitrary value without getting an exception:
>>> x = StrictUndefined(name="x")
>>> unicode(x)
Traceback (most recent call last):
...
UndefinedError: 'x' is undefined
>>> x == 42
Traceback (most recent call last):
...
UndefinedError: 'x' is undefined
A better sandbox. The sandboxed environment if enabled is now easier to secure. In the default configuration everything starting with an underscore is considered insecure so it’s not needed any longer to mark those attributes as insecure. It’s also a lot faster now and easier to customize.
Better i18n support. Jinja2 now integrates neatly into Babel. No need to write custom string extraction scripts.
Extension interface. Like Django it’s now possible to write custom tags. I doubt anyone wants to use it as most functionality is already possible out of the box by using the expressions and tags provided, but it may be useful for some people, especially those switching from Django.
Line statements. Jinja2 allows to to specify a line statement prefix that marks a whole line as statement. This concept is inspired from Mako and Cheetah and allows very clean templates in many situations. The following example shows a template with the line statement prefix set to “%”:
<ul>
% for item in seq
<li>{{ item }}</li>
% endfor
</ul>
Easier filters. Filters are regular functions now that get the value as first argument and the parameters provided as extra positional arguments or keyword arguments. In Jinja1 we had functions that return functions which turned out to be unnecessary complex and slow. Additionally you can now use “namespaces” for filters so do stuff like {{ variable|tools.something }}.
Automatic escaping. Jinja2 supports the special __html__ method as specified by pylons which makes it possible to use automatic escaping. (We still don’t recommend it though)
Easier API. The API is a lot easier to use now, custom loaders are nearly one-liners now and the caching is builtin automatically for all loaders. No need to funny mixins that add caching.
Grab it while it’s hot from the temporary Jinja2 website.
Yesterday we upgraded hammet (the server powering the Pocoo projects) to Ubuntu LTS. Unfortunately Ubuntu 8.04 aka hardy doesn’t ship Postgres 8.1 any longer so we had to upgrade to 8.3. If you are ever in the same situation: DON’T UNINSTALL 8.1. I did that because my applications showed an auth error and I guessed that was because of two running postgres servers. Turns out that postgres databases are incompatible between versions and the only way to upgrade a cluster is having both versions running and using pg_upgradecluster to upgrade them. Reinstalling in hardy won’t work because there is no package providing an 8.1 server.
What I did then was installing the 8.1 server again from dapper sources, delete the new cluster with “pg_dropcluster 8.3 main” and migrate with “pg_upgradecluster 8.1 main”. After that getting rid of the 8.1 cluster and server and most of it works again.
I fought that auth error by setting local socket connections to trusted in the config responsible for authentication and had to upgrade trac because implicit string casts no longer work.
Those non portable, version incompatible database files and the fact that I hate psql and those admin tools are by the way the reason why I feel more comfortable with MySQL.