Armin Ronacher

For Lincoln Loop I am doing more with the Facebook API currently than is good for me. Now Facebook is not using OpenID but OAuth, but the core issue is the same and it becomes a little bit more obvious when you're doing OAuth based sign in. The problem is that users are lazy, and so are developers. Developers are especially lazy because when they have to sign into their development application multiple times a day for testing the authentication module you will take every measure possible to save time. And in my case it usually involves remembering the password for my OpenID for the session. But even without remembering the actual password of my OpenID account, it still means that my OpenID provider remembers the authentication request for my application and will skip the screen the next time the application requested authentication until the browser closes.

Enter OpenID's (and OAuth's) security problem: the remember me for this computer button.

For as long as I have to do with websites and authentication systems the following three rules were widely followed by application developers:

1. To change your password you have to enter your old password for confirmation.
2. To change your email address for an account you have to enter your password or old email address.
3. To delete your account you have to confirm per email or at least old password, sometimes even a combination of both.

Notice one thing? All these steps require you to enter a password. Now with OpenID / OAuth we don't have a password we could enter on that website, what we could do instead is redirecting to the OpenID provider / OAuth service to let the user confirm the action. Unfortunately these services will remember you. If you are a careful user you will not let the service there remember your password for general authentication, but it will remember you when you come back in the same browser session.

So say you are logging in on your computer and not close the browser. Leave the notebook unattended for a few seconds and someone else comes to the PC to delete your account for example (or do something else of destructive nature). Even if the application does a OpenID auth check to delete the account (which no website I tested does) the OpenID provider will have remembered you (at least the ones I tried all do). Bam, you lost your account.

Deleting here is not really the problem, just think about the OpenID equivalent of password change: linking another OpenID account to your profile. Most websites support more than one OpenID identity. To add another OpenID identity URL to a website you most of the time don't even have to confirm from an old one. It will just happily accept a second OpenID identity URL. And again, even if it would check for the old one, your provider would still have you remembered.

This problem is unique to OpenID because OAuth services usually do not support more than one connection because the OAuth based login mechanism is customized for each website. Signing in with Twitter is a completely different procedure than signing in with Facebook. You usually can't hook a different twitter account to the same user account that was already linked to Facebook or the other way round.

The OpenID Security Best Practices only mention that you have to check the added OpenID identity URLs if they are controlled by the user by passing it trough the authentication handshake, but it does not mention anywhere that you should check at least one of the already authenticated for confirmation. And as mentioned above, if the OpenID provider remembers you that would not solve anything here.

I might be missing something completely obvious, but there does not seem to be a way to force the OpenID provider to force the user to approve the authentication again (same for OAuth). The OpenID PAPE extension does provide some metadata for the application to check if the authentication happened via password or a hardware device but as far as I can see, it also does not provide a way to force the approval screen on the user again. The PAPE extension however does provide a way to specify a maximum lifetime for the approval in advance (like a minute). That would certainly help to fight that problem, but then you would have to do that on the initial sign in and not when you want to force the approval from the user when adding a second OpenID for example.

I might be completely wrong with what I just wrote above, but from my experience existing systems are not providing a way to force this approval screen on users or it's just not obviously documented so that nobody is using it. If there is a way, please let me know. Either way I will be vary careful to not leave my notebook unattended for a second when a browser is still open.

People using my stuff will have noticed a trend that I started using git (and github) for some of my newer projects. Now there are two parts of the story. One is obviously github which is currently unbeatable. Not necessarily due to the size of the community but that it gives you a very neat way to look at what patches are available all over the place (I'm referring to the fork queue). But the other reason is that despite all the hate for git I have, it has grown on me as the tool that does the job better.

This however is a shame because if you remove the github component and some other addons like the hub command, hg is the superior tool. If you would download both tools and compare what they can do out of the box with the stuff they ship, hg has the better user interface and better tools for collaboration. These tools are essentially hg incoming, hg outgoing and the hg serve command. Additionally the zeroconf extension is pretty awesome when you collaborate locally.

Just to give you an example: Georg and me where hacking on Logbook at EuroPython over an ad-hoc Wifi network because the people running the conference were unable to keep the network running. Even though logbook is hosted at github currently, we were using mercurial for hacking locally. You fire up an hg serve in one window and other persons on the network can instantly pull from it. That's patch exchange you don't have to think about, it just works (tm).

On the surface hg and git look very much alike. The commands are now pretty similar and they appear to solve the same problems. However there is a philosophical difference between these two: the handling of branches.

In both hg and git you're exchanging things called changesets. They have an sha1 checksum and are linked with their parents and some other meta information. Whatever information you are changing, the checksum will change as well. So if you would take a changeset and only change the name of the committer, the checksum would change. In hg there is another piece of information that makes a changeset: the branch the changeset was committed to. A branch in hg has a name (one that for a long time was eternal) and in a way a head (the latest commit changeset).

Now to understand my problem you have to keep two things in mind: changesets in hg are linked to the first one that ever happened in the repository (same for git) and that a changeset belongs to a branch, even if merged.

I don't like merging. That might sound strange, but I really don't like merging. I did a lot of merging in hg because it works good enough, but what I started doing with git is picking single changesets and applying their diffs. Why am I doing this? Now first the implications: the checksum changes, because the parents of a changeset will change with that operation. On the other hand it gives you the flexibility to pick a single changeset and apply it to different branches.

So a good example are maintenance branches. My concept of maintenance is to release bugfix releases if enough bugs accumulated. I don't promise a bugfix release upfront, because if the API does not change in a backwards incompatible way and a release is upcoming, I can combine that into one release. Now that might sound like bad release management, and it clearly is not the optimal way, but I am one person doing the releases and I have way too many projects to care about and a limited amount of time.

Now what if at one point there are already two or three bugfixes or a critical bug comes up that absolutely must be released (for example security issues). What I do in git is I check out the tag of the release I want to incorporate the patches and start cherry-picking the patches from the master branch I want to apply. I obviously could not merge with master because then I would pull in all the patches up to the patch I was interested in (remember, they are linked and only exist if the link to the first changeset is fully established).

In git branches are just labels to a changeset. When you cherry pick one changeset from a branch (or a branch of another repo) and apply it to another one, you no longer know where the changeset was coming from. That's a limitation I can totally live with. The cherry picking command in git has a flag that allows me to add a "signed off" flag. That way I can preserve the original author of the changeset and add my own name for the signed off flag. So a changeset shows both authors. Here an example of how this looks like: commit 36a421bb3a235f696131 to Flask. This is actually implemented by just adding a mime-like header as footer to the changeset, nothing fancy with commit metadata etc. However the important thing of this feature is that it's consistently implemented by the little amount of UI that exist in git and other commands. People are aware of this signed off thing.

Doing the same in hg is also possible with the transplant command. The downside is that if you transplant patches from one repo or branch into another one, the incoming and outgoing commands no longer work like you would expect them to do. They compare the checksum of the changeset instead of the checksum of the contents of the patch. And as soon as the UI starts no longer working, user frustration starts happening. But the frustration of a user is not the issue here, the real issue is that this confuses users and will cause them to make (understandable) mistakes. I'm not a revision control system author myself and my knowledge of the tools I'm using is limited to the subset I am using as well. Very often when I get the link to a hg repo from another person I will notice that they have a bugfix hg branch or something similar and screwed up merging with upstream. Sometimes in the process of correcting that mistake I will make another huge mistake myself that I then start correcting in frustration by stripping all the changesets I just made and applying the patch by hand, totally bypassing hg. These mistakes happen because features like remote bookmarks, transplanting changesets and even named branches are not integrated well enough into other commands. Especially that the branch name is imprinted into the changeset is confusing to say the least.

For example the hg repositories on pocoo.org were partially created from a huge subversion repository by an early version of one of the subversion conversion commands. Don't ask me which one, but what it did was naming the default branch everyone works on "trunk". However to the best of my knowledge a hg repository starts with "default" as default branch name and hides that from the user interface. When people now pick up work on one of our converted repos sometimes the confusion kicks in why their hg experiments never had a displayed branch and why certain assumptions are now invalidated. This especially becomes an obvious problem the more git users are out there. A git user has the idea that "git pull" will fetch all the changesets and merge that with the branch they are tracking. Git has a pretty bizarre but working concept to track remote branches that involves entries in the git config I don't understand at all. I'm thankful the git UI provides some support to do that for me. However on the other hand in hg we don't have the concept of tracking remote branches. What we have there instead are just aliases to a URL that points to a hg repository (and lately we can even refer to branches on a remote repo in the URL that is another source of confusion for me I will explain later). hg will pull changes from there and let you do the merge then. Because that is quite repetitive someone came up with the fetch extension for hg that pulls in changes and merges if necessary, otherwise will just update. Now that fetch extension just assumes that if there are two heads it should merge them, otherwise if there is one head it will update, and if it finds anything else it will just error out. You have no idea how many times hg fetch totally destroyed the recent history when I merged in stuff that looked find in hg incoming. Now obviously I should not have used hg fetch in the first place, but a bit of help from the extension would be appreciated. Fetch is also only partially to blame because it predates bookmarks and back then nobody was using named branches.

It all boils down basically to hg having an awesome internal concept that does a lot of things really well, but other things not being properly supported by the user interface I suppose.

In layman terms git's branching concept roughly works like this: all your changesets are linked to at least one parent. A changset has a checksum for the commit + metadata and I think a checksum for the contents of the patch as well. Additionally you can assign names to changesets. There are tags which are aliases to changesets that do not change, and there are branches which are basically names that are moved to a new changeset on commit. Because of how this works you can have changesets that are no longer linked to any place of the history because they point away from one changeset and the name information of their hash is lost. Git has a garbage collector that can remove these things from the repository. There is not much magic in there and obviously it would be very easy for hg to support that, but the core issue here is not the implementation but the user interface. I don't think hg needs git style branches at all!

What hg needs instead is a user interface that is aware of certain operations:

• hg push can push bookmarks to remote repositories nowadays, but this also involves pushing multiple heads to the serverside which the hg command currently disallows by default unless you force it to. This is user unfriendly and even if you are an experienced hg user you will most likely not do the force but ask first if you are doing something wrong here.
• The special label "tip" in hg moves between branches. This is the worst user interface fail because all tools are interested in tip more than what branch you are on. For example if you are using named branches in hg things like bitbucket, trac (and I think even hgweb) will show you the contents of the default file browser view alternating between different branches. If the last commit was on 0.2-maintenance you will see the contents of the 0.2-maintenance branch by default, if the last commit was on 2.0-new-features it will display that branch instead. I think the invention of a tip label was a mistake. It should have been per-branch instead.
• hg incoming / hg outgoing are unaware of things like hg transplant. If you are doing transplanting, hg incoming over the times becomes so filled up with changesets already applied that you better not use it at all. It also forces people you just transplanted patches from delete their local checkout (or strip it to an older revision) and pull again. In git this is not a problem because you just delete your local feature branch and update the separately tracked master branch which will magically include the changes I transplanted. Due to the separate content tracking this will not even transfer larger amounts of data over the wire, because git will already find them locally.
• The hg heads command is intended to help people merge changes locally by visualizing what unmerged heads are there, but when you start using bookmarks and named branches it's intentional that there are multiple heads. The more branches, the more useless the command. The solution here would be git-like branch tracking so that heads can filter out uninteresting pieces of information.

There is that hg over the time started adopting solutions that actually are not solutions. For example you can have an alias not only for the URL of a repository but the URL of a repository + a branch. That sounds like wonderful thing to have, unfortunately due to the fact that the name of the branch is part of the changesets you will be surprised sooner or later when you merge in changes from another persons repo and suddenly see feature branches appear even though you never specified anything like that. This is a problem that becomes more apparent the more people use hg with git knowledge where it's logical and encouraged to create branches for every single feature you are implementing.

I am not quite sure what the named-branch-in-URL is supposed to fix, but for me it just created more confusion when working with repositories that actually have named branches in that differ from the one or two I have.

So why does hg has these "limitations" in the first place? Probably because the history of the project was not the big-ass Linux kernel (though that is not entirely true) but mercurial itself. And for smaller repositories the git model does not appear too appealing at the first glance. It took me a while to start fully appreciating the possibilities of small branches and now I can't live without them any more.

The git UI might suck, and I still have a hard time with it, but the branching works nicely and so is dealing with other people's patches. It's the perfect tool for me currently when it comes to quickly reviewing and applying patches and it grew on me. I still wonder internally why the hell I'm writing git push origin :branchname to delete a branch remotely :) And with all these UI faults, git has the better UI for branchy development :(

But due to the horrible git UI and the incredible hostility of git for people switching from the friendly lands from mercurial I am pretty sure very few core developer of git had a look a serious at "the other tool" to get an idea of what everybody is buzzing about when the topic "hg and git branching" comes up.

If you are a Vim user, you probably have your own set of tricks that make your editing life a more pleasant experience. So here my set of things I love about Vim and might be worth sharing.

Some things first: normal mode = mode you enter with ESC, insert mode = mode you enter with i (or any other insert starting key).

Searching

That's probably a no brainer because everybody does it all the time, but searching in Vim is a pretty cool experience. Standard search is from normal mode by typing /searchword. To go to the next result use n. If you want to search in the reverse direction, use ? instead of /. If you want to continue the search with the last searchword, just use / or ?.

Sometimes you want to search for the word you are just having your cursor over. That's possible by using * in normal mode, which will search for that words forward, # searches for that word backwards.

When you use * you will notice in the status bar that Vim is wrapping the word in \< and \>. This marks beginning of word and end of word. That way if you search for \<foo\>, it will match foo but not foobar.

Replacing

Generally replacing works with the command :%s/search for/replace with/g. For replacing some weird rules exist which are worth knowing. First of all, the / can be replaced with pretty much everything else. If the slash is part of your searchword, you could also use :%s#search for#replace with#g. The second weird rule is that if you want to insert a newline, \n won't do that, \r does. For replacements \n is actually the nullbyte! But don't worry, \r inserts the correct newline sign that is currently in use for this document. The g means: “replace all matches in a line”. Without that, it would only replace the first occurrence of each line. This is a pretty obscure thing that this is not the default, but in some cases you might want to remove the /g. If you replace that way, Vim will do replacements for the searchword on each line in the file without asking. If you add a c to the flags, it will ask for confirmation for each match.

So far, so good. Now how would you search for the last searchword and replace this? Just leave out the last search: :%s//replace with/. This is especially useful when combined with *.

Another helpful thing is to only replace in a selection. That's very easy. Just select the lines you want with visual selection and then leave out the % sign when doing the replacement. The % just tells vim to operate on the whole document, not having anything there would mean just operating on the current line.

Ways to Insert

As programmer in most languages it makes a lot of sense to think in lines. Vim does that. The most common block selection operates on lines and so do searching and replacing. This line-based thinking also comes in handy when doing inserts. o inserts a new line, O creates a new line in front of the current one. In both cases you will end up in insert mode.

But there are cooler ways to insert too by using motion commands!

Motion Commands

Vim has a couple of motion commands. These need a bit of explanation but trust me, they are awesome. First the most common commands that support motion: y{motion} for copying into a register, c{motion} for deleting and stepping into insert mode, d{motion} for just deleting without going to the insert mode and of course v{motion} to select something.

Now what the hell is {motion}? Motion commands are commands that move the cursor. The probably most awesome motion command is iCHAR where CHAR is a special character to search for in both directions. There the extra rule applies that parentheses are properly inverted for you. This works for ", ', ( and others. Vim is clever enough to adhere to the escaping rules of the language you are working with. This for example means that the command ci" looks selects everything between the next two quotes, deletes the contents and goes to insert mode (c = change). If you want to have it selected instead, you can use vi". vi( will select everything between the current set of parentheses etc (v = visual selection).

The following other motion commands exist: b moves the cursor to the beginning of the word, e to the end. {count}j would go {count} lines down (j = jump I suppose). f{char} looks for a character on the right, F{char} to the left. t and T work like f and F but go one character more (before match, after match). Last but not least there is {count}w which moves {count} words.

Awesome File Navigation

There is a little undocumented (or badly documented feature) of Vim called autochdir. Some people warn about it because it might break plugins, but I never had any issues with it and I am using it for a long, long time. To enable it, you just have to add this to the vimrc: set autochdir.

When enabled, Vim will change into the folder of the file of the buffer. This is pretty handy because :e then works relative to the file. So for example if you are in foo/bar.py and want to edit foo/baz.py you only have to do :e baz.py. Thanks to the automatic completion with Tab and ^D this makes navigating in trees fun. If you have a file open already it will be put into a buffer. If you then substitute :e with :b it will only complete to files that were already open. Because this also supports partial matches, very often a :b baz is enough to go to the baz.py file.

Why The Hell is this not a Default?

There are a couple of things to add to the vimrc where I really wonder why this is not the default:

set title. When this is on your vimrc, the terminal title will automatically reflect what buffer you are working in. This also works in GUI mode, but there it just does nothing. This really should be the default.

Steal my Config

If you want my config, you can download it from this hg repository: mitsuhiko-dotvim.

Just wanted to share a few mistakes I made regarding licensing of Flask and some general suggestions of how to handle it. Also I would like to explain a problem I have with open source licenses in general and if you have some suggestions. Disclaimer: IANAL and what I write down there might be completely wrong. If you know better, please let me know. (And I mean it. Granted, I hate licensing, but I also depend on them so I want to have a good understanding of what the heck I'm actually doing by licensing my stuff)

Closed Open

I love Open Source, I really do. All my code is licenses under the very generous BSD license which comes with barely any strings attached: do what you want, keep the copyright around, do not use my name to endorse derived products. And I am totally fine with that. But that is my take on code alone. When it comes to artwork and design I would love to use a less open license. Why? Let's take the Flask documentation as an example. This documentation has a nice and simple stylesheet and is still unique enough that one could recognize Flask or Flask related projects from it. So I would love (but can't) limit that license to something like: use this only for Flask related projects or modify it enough to be visually different. Now of course that is a fishy description, but this could be fixed. The problem however is that this is no longer an open source license (which I would not have any problems with). After all that also forces Linux distributions like Debian to ship a modified version of my project with a different style for the documentation because they have to reject such packages or put it into the non-free section.

Both things I would love to avoid, so I back-paddled on my licensing plans and put the themes under the BSD license with friendly reminder now. But when I was changing the license an interesting thing came up: could it be that the old theme was partially unlicensed? Now one stylesheet was already BSD licensed, that was not a problem, the template files and everything else did not have a license header nor was a LICENSE file in place. The way Flask used that style was with a git submodule. Interestingly this was not present in the source distribution of Flask. So unless Debian made any changes, the docs would not build out of the box on a Debian system anyway which brought back my idea for going the non-free path with docs. But that is now past in any case, so let's move on. But the more interesting question here is what license that thing is under now. I am pretty sure one could argue that the original documentation was not licensed, thus closed source and all Flask extensions I did not write were violating the license (which was never the intention because the documentation strongly recommends the Flask documentation themes).

Licensing Documentation

Licenses from a source POV are already crazy, but when it comes to documentation texts, images and everything else involved in a modern application or even software library the whole thing becomes way more complex. Like many others I so far licensed my code under the BSD license and I did that by adding a quick licensing header to all files and adding a LICENSE file with the license text. The license text was more or less the original three-clause BSD license text with "University of Berkley" replaced by "Copyright Holder" and a line that describes who the copyright holder is/are so that copy/pasting that license gets easier. With Flask I now made a change to that license so that it not only mentions source and binaries, but also the documentation explicitly (read the new license). From a legal point of view it should still be the same as before, but now explicitly mentions the documentation. That documentation should get their own license is something I did not come up with, other people did that before me. Which is why there exists a list of documentation licenses: GNU Free Documentation License, FreeBSD Documentation License, the Creative Commons Licenses and many more.

Now my first plan was to adapt the FreeBSD Documentation License because it is very close to the BSD license and should not cause any troubles switching to without having to ask all the persons who provided patches. Though a licensing change is a licensing change and even if just the wording changed I would not pull this off without sending out mails first. But before I even came to that, I noticed that this would not work out at all. Because if that is assumed to be a different license than the BSD license used previously the final built documentation would be affected by both the BSD and now the FreeBSD documentation license. Why that? Because the sourcecode itself is still under the BSD license and with that the docstrings in there which are pulled directly into the final documentation. While one could argue in many programming languages that the comments are under a different license, this does not work out well in Python where the docstrings are available at runtime. They are clearly part of the program.

My final change was to take the previous BSD license and explicitly add "and documentation" to everything that previously just listed "source" and "binary" form.

I am quite confident that this is overkill but still serves the purpose. To be honest I doubt anyone would ever want to sue anyone based on ambiguous documentation licenses, but for me the whole point of a license is to be something I can defend my project on. And as such I want to have a clear understanding for myself what exactly is under which license. If someone would have asked me a week ago under that license my documentation was, I would have answered with BSD. If then that person would have asked me where this was written down I would have had a harder time explaining that "source" also means restructured text and "binary" also means HTML.

Logos and Design

Now this is where I am totally lost. When I was asking on Twitter what license to pick for a project logo, people answered with the creative commons attribute non-derivative 3.0 license (hereby named CC-ND-3.0). Now I actually did license that logo under that license and even shipped it with the documentation and the Flask tarball, and it also ended up in Debian. The interesting part is that the website clearly mentioned the license as CC-ND-3.0. I just violated my own license and from my understanding of the copyright law, what I released should be considered CC-ND-3.0 for the logo because I would not be legally allowed to license that as BSD. This is pretty weird because as a copyright holder I can release that thing under anything but you could argue that because the LICENSE file was not in the same folder as the logo and the logo was just copied there, it not automatically becomes BSD licensed. That's like if I would copy the Debian logo into my project it does not become BSD licensed, I just made a license violation by not adding the license. Again, in the Debian can I'm not the copyright holder, but I could imagine that if that would have to be defended in court, the same logic would apply.

Either way, the interesting part here is that the CC-ND-3.0 license is not accepted in Debian because it is not a free license. So in theory if my package would be a license violation and I correct this license violation by revoking all current releases, and re-release those packages with a license text for CC-ND-3.0 Debian would have to update all their tarballs to exclude the logo from it. It's fun to see how such small differences can cause such an amount of troubles for all parties involved. So I decided not to be a dick and adapt a very simple license for the logo (read license text) that is formed after the Debian free logo license. It also has a BSD-ish clause regarding endorsements by Flask authors and recommends linking to the Flask project website. From what I understand, old release are now okay in compliance with the new license which also works retroactively so Debian should be in the clear. However the source version of the logo (the SVG file) still has to come with the license text which never was part of the distribution tarballs.

Logo and Design Protection

At that point I'm totally convinced that there is no way, besides registering a trademark, to protect your logos for an open source project. Restrictive licenses are not accepted by Debian and many other Linux distribution causing a lot of work for all parties involved. Sounds pretty bad for small open source projects but in reality the majority of people are reasonable and have no problems with accepting "recommendations" or "kind requests" in licenses.

Of course once your logo is BSD-ish licensed people actually could use the logo for something completely different. I would love to hear about a reasonable recommendation in that regard besides: just ignore it. The point is: as long there is no license, it is closed source and Debian cannot ship it. I am also not totally sure exactly what the Debian logo license (and as such the Flask logo license) exactly allows. May you now use this logo as a bullet point on a website not related to Flask? From my understanding yes, because otherwise it would not be considered open source, but what exactly is the legal interpretation of this sentence be then?

This logo or a modified version may be used by anyone to refer to the Flask project, but does not indicate endorsement by the project.

At that point I am totally lost and just want to ignore that problem, but it would be interesting how an open source project can get a bit of control over the non-source parts. Logos are something I would love to see sort-of protected.

To come back to the copyright problem with the documentation style: I am well aware of the fact that protecting visual appearances on copyright is not possible. One could replicate a 1:1 version of the Flask documentation design for instance by never looking at the original CSS and template files and would be totally fine (clean room reverse engineering). But to be honest, nobody does the work of reproducing the documentation style 1:1, that would be far more complex and time consuming than creating your own. I have no problems whatsoever with people who take the design and add their personal touch to it. In that case why not use my source files?

tl;dr: licensing is hard, let's go shopping.

Howdy everybody. This is the first part of a multi-part blog post about creating web frameworks based on Werkzeug and other existing code. This is obviously based on my Flask microframework. So it probably makes sense to head over to the documentation first to look at some example code. But before we get started, let's discuss first about why you should create your own frameworks.

Why create your own Framework

It is quite unpopular these days to go with building your own framework; everybody quickly shouts "reinventing the wheel" and points you to one of the tons of existing web frameworks out there. But it is actually a really good idea to create a framework for an application and not go with a stock one. Why? Because you are a lot more flexible and your application might require something that does not exist yet. For an application I wrote in the past in the very early Django days the development process looked a lot like this:

Step 1: download django, Step 2: get started and feel happy, Step 3: encounter problems in the framework design and start modifiying the core, Step 4: phase more and more Django code out and end up with a completely new package that everybody hates.

Turns out: Django like every other framework out there is improving quickly, but often not in the areas you might be interested in. Then you start modifying it yourself and when Django improves sideways, you suddenly end up in the situation where it becomes nearly impossible to upgrade to a newer Django version or it's too painful. Obviously Django has greatly improved since then, but a few things continue to work differently than I want them to work. For one I personally don't like the template engine too much and also would love the ORM to ensure that objects with the same primary key are actually the same objects and queries sent less often. These are things that are very unlikely to change in Django and there are really good reasons why this will not change which are totally fine, but certainly not what I want.

Another reason to roll your own framework is that you know everything and you can fix it quickly yourself.

End Result

This is what should work at the end of the day:

from yourflask import YourFlask
app = YourFlask()

@app.route('/')
def index(request):
    return 'Hello World'

if __name__ == '__main__':
    app.run()

Looks a lot like a simplified Flask version, which is exactly what it should be. Not yet as capable, but easier to dive in and to understand the concepts.

In a nutshell: 1) create an application, 2) register functions on that application that listen on a specific path (or URL rule), 3) these functions return response objects or strings. We also pass the request object explicitly to the function for now because that's easier to understand and implement.

The Code

The following code implements the full framework for this blog post. As I said, it's a very simplified Flask but it is capable of producing simple web applications and to run the example from above:

from werkzeug import Request, Response, run_simple
from werkzeug.exceptions import HTTPException
from werkzeug.routing import Map, Rule

class YourFlask(object):

    def __init__(self):
        self.url_map = Map()
        self.views = {}

    def route(self, string, **options):
        def decorator(f):
            options['endpoint'] = f.__name__
            self.views[f.__name__] = f
            self.url_map.add(Rule(string, **options))
            return f
        return decorator

    def run(self, **options):
        return run_simple('localhost', 5000, self, **options)

    def make_response(self, rv):
        if isinstance(rv, basestring):
            return Response(rv, mimetype='text/html')
        return rv

    def __call__(self, environ, start_response):
        request = Request(environ)
        adapter = self.url_map.bind_to_environ(environ)
        try:
            endpoint, values = adapter.match()
            response = self.make_response(self.views[endpoint](request, **values))
        except HTTPException, e:
            response = e
        return response(environ, start_response)

So how exactly does it work and what does it do? The following list is the summary of the above code:

• We create a class called YourFlask that implements a WSGI application and provides methods to register callback functions and binds them to a Werkzeug URL map.
• The route() method can be used as a decorator to register new view functions. It does this by accepting a string with the URL rule as first argument and accepts some more keyword arguments that are forwarded unchanged. The routing system uses an opaque string to identify functions. This is called the endpoint. In this example we will use the function name as endpoint (something Flask does as well for simple setups).
• The run() method just starts the internal development server that comes with Werkzeug. That's just a nice shortcut.
• make_response() is called with the return value from the view function. If it's a string, we create a response object. That's just a nice shortcut.
• In the __call__() method we implement the full WSGI application. First a request object is created from the WSGI environment and then the URL map is used to create an adapter. This adapter is basically bound to the WSGI environment and can be used to match the current URL. If a match is found the endpoint and values are returned (the values are variable parts in the rule as dictionary). In case nothing matched, a NotFound exception is raised which incidentally is also an HTTPException. If all works out we look up the view function and pass it the values and the request object.
• The return value of the function is passed to our make_response() method so that we can ensure it's a response object.
• If an HTTPException is raised we catch it and use it as response object. It's not exactly a response object but close enough to one that we can do the same with it.
• Either way, the response is invoked as WSGI application and the application iterator is returned.

Where WSGI fits in

So what we created is a WSGI application. How exactly does it work and where is the WSGI part? The majority of the pain is handled for us by Werkzeug. WSGI itself looks like this:

1. There is a thing that can be called. It's passed a WSGI environment (which is basically a dict with incoming data) and a function that is used to start the response.
2. What the function returns is an iterable of data send back to the browser, it has to call the response starting function first.

If you look close, we are doing that in our __call__() method. Well, it's not really visible but it happens. When we invoke the response thingy, internally Werkzeug will call the response starting function and all for us. We also use the WSGI environment when we create the request object.

The request object itself gives us access to all the stuff that is incoming from the browser: where the request went, what values were transmitted, what browser is used, the cookies etc. We will focus on that with the next blog post.

Coming up Next

Now that all is working fine we should focus on these things next:

1. explore the concept of thread / context local objects to avoid passing the request object (not saying it's necessarily a good idea but crucial for understanding web frameworks in general. Even if you think Django does not use them, it does. The i18n and database system is powered by thread local objects).
2. add support for a template engine and serving up static files
3. add more helper functions for URL building, rendering templates and aborting requests with errors.

Stay tuned :)