High Level AST Module for Python
Georg already blogged about the new ast to code compilation feature of Python so I won’t cover that any more. Basically the old compiler package is nor surpassed by python’s real internal AST which is freaking awesome :)
The only thing that is missing is some sort of high level interface to it which makes manipulating and debugging it easier. The motivation behind it is that template engines such as Genshi, Jinja or Mako which all operate on the AST don’t have to write that much boilerplate code just to modify small pieces of the AST.
I don’t know if this module will make it into the standard library but even if not it will surely be available via the cheeseshop. So what does it do? It provides classes and utility functions to manipulate or traverse the AST in a way that is actually useful for real world applications. For example with the transformer it’s not possible to replace and remove nodes. Additionally it’s possible to generate python sourcecode from an AST to simplify debugging.
Here a small example of a transformer that brings Genshi like Loop behavior to a piece of Python sourcecode:
class GenshiSemantics(ast.NodeTransformer):
def context(self, method):
return ast.Expr(ast.Call(
ast.Attribute(ast.Name('data', ast.Load()), method, ast.Load()),
[], [], None, None
))
def visit_For(self, node):
self.generic_visit(node)
return [self.context('push'), node, self.context('pop')]
def visit_Name(self, node):
self.generic_visit(node)
return ast.Subscript(
ast.Name('data', ast.Load()),
ast.Index(ast.Str(node.id)),
node.ctx
)
When applied this piece of code:
seq = [1, 2, 3, 4, 5]
item = 42
for item in seq:
print 'inside', item
print 'outside', item
becomes
data['seq'] = [1, 2, 3, 4, 5]
data['item'] = 42
data.push()
for data['item'] in data['seq']:
print 'inside', data['item']
data.pop()
print 'outside', data['item']
The sourcecode of the module is in the Pocoo sandbox as usual. Unfortunately there is a bug in the initialization function of the ASTs right now so you will need a patch for it. And obviously this requires Python 2.6.
The full sourcecode for the example above can be found here.