Jinja2 — Making Things Awesome
Some of you might have noticed that pocoo.org redirects to the development center now. No direct reference to the old pocoo project any more. The reason for this is that a part of the pocoo team and the ubuntuusers webteam is working on a new software that combines wiki, forum, planet, pastebin and blog into one big portal software. It’s not yet open sourced but it will become a pocoo project, most likely licensed under the GPL around June, with a semi stable release at the end of 2008. I’ll post some details the next week I think.
But why am I writing about this? Mainly because the work on inyoka (the name of the pocoo successor) showed some weaknesses in Jinja and we did some brainstorming to come up with really cool improvements for Jinja that resulted in a complete rewrite. Because we love backwards compatibility these changes will go into a package called “jinja2″ and both Jinja 1.x and the Jinja 2.x will get updates in the future. Also Jinja 2 is not yet released and you shouldn’t expect a release anytime soon, if you want to play with it you will need the current hg version.
But what are the changes to Jinja 1 and why is 2 better? What we noticed when working on inyoka is that quite a lot of display logic code goes into the templates. Quite a lot of logic is in those templates (not application logic though). This isn’t a bad thing actually but it showed that especially for large templates with many dynamic parts such as loops or even variable tags Jinja doesn’t have the performance we wished it would have. Especially if templates become more complex quite a lot of CPU time is spent there are there was room of improvement in Jinja. Unfortunately a design mistake made it impossible to do any optimization there and I was forced to change the scoping rules to something saner. I would say for 98% of the templates the changed scoping won’t make any difference as many have avoided the side effects of the old scoping anyways. But still, it’s a change that will break things, so Jinja 2.
But right after I decided to break backwards compatibility there was more room for improvement which resulted in some real kickass features. Finally filters are simple python functions again and not factories any longer. They now also support keyword arguments, a feature on the wishlist for nearly as long as Jinja exists. Dynamic inheritance is now something that works without pain too and the template inclusions where simplified. And that also means that templates can be included into namespaces now. So {% include macros = "helpers/macros.html" %} gives you an object with you can print or where you can access all the macros or defined variables etc.
What really makes me happy is that on the surface few things have changed. Most of the implementation details are hidden in the compiler and are not visible on the templates. For example that Jinja 2 knows five different ways to iterate over sequences and that stuff is not visible in templates at all. By changing the scoping rules everything of the user friendliness is still in place.
Other nice changes are that you can now filter for loops while iterating over them like list comprehensions in python and that the lexer knows line statements now. Line statements are lines prefixed with some optional whitespace and one or more marker characters. Everything after that until the end of the line is a single statement. That was shamelessly stolen from Mako and Cheetah and is something you have to enable explicitly, but I kinda like it for some template scenarios:
# for item in seq
* {{ item }}
# endfor
Another very neat idea we implemented in Jinja 2 (wasn’t my idea unfortunately ^^) is that we broke the template scoping into globals and locals. Globals are known at compile time and render time, locals are known at render time only. That way Jinja can automatically evaluate parts of the template at compile time. The idea is that in many templates you have data that doesn’t change that often. A good example is a forum software. The list of forums changes seldom, at ubuntuusers about twice a year but every information that lives longer than fifteen minutes or so can be treated the same way. Anyways. The list of forums is information that doesn’t change, if we can give it Jinja as global variable for the template, Jinja can do quite a lot of compile time optimizations. In the best case the whole list of forums is one giant static block not processed at render time at all.
That keeps the templates designer friendly as the person who designs the template doesn’t have to know anything about that but gives the application developer a simple way to optimize performance. This approach however has some rough edges we have to polish.
And the last important change is that the sandbox is optional now and even disabled by default. Most users were not using it anyways and there is no need to put that into the system by default. It’s however a built in functionality and will get some improvements as well. The new sandbox will give a better control over what’s secure and what not.
How fast is it currently? I really don’t want to throw pointless numbers around but for the test table we’re using currently the speed without sandbox is more than comparable with mako and a lot, lot faster than django’s templates. But please take those numbers with a shovel of salt as we’re talking about an unreleased project here and a more than biased benchmark for one particular use case. Jinja tries to make templating as simple as possible and not as fast as possible.
Making a template engine that’s fast is incredible simple. But making a template engine that doesn’t suck and performs well is a lot harder.
