lucumr

I must admit that I was not that much interested in Python 3 pretty much because I maintain a couple of libraries and all that backwards incompatible stuff looks like pain in the ass. Additionally lots of the stuff from the PEPs looks like it completely changes the way the language behaves. But I was wrong.

One of the things I always liked about Python was the ability to use multiple inheritance. Back in the pocoo days we stated using interfaces in a Zope/trac like manner and then noticed that we can do the very same with simple base classes and isinstance calls too. Yes of course you can abuse that and create inheritance trees nobody wants to look at, but seriously, your fault then.

But what always confused me was that nobody used ruby like mixins. Having multiple inheritance is predestined for mixin in functionality but the only classes in the standard library that did something like that was the DictMixin. And that mixin had another problem: it was not a subclass of dict (obviously). Now many libraries do something like isinstance(foo, dict) to switch between modes. A very common situation where this is necessary is if you want to accept iterables of tuples or dicts. This is a situation where duck-typing doesn’t really work out. Of course you can do hasattr(x, 'items') to check if that object implements the dict interface, but that is ugly and can lead to unexpected behavior. For example: what’s a dict? There are ancient dict like implementations missing the __contains__ method and just have has_key for example.

In many languages missing multiple inheritance this is solved by specifying a IMapping interface and implementing that in custom classes. But of course Python can do better and with Python 3 it finally did. And it did that in a incredible cool way that integrates nicely into the language and doesn’t break the zen of python which is freaking awesome.

So what’s the solution Python 3 has? Abstract base classes! So how do they work. As I already said above in the python python developers usually relayed on two things: duck typing (testing if an object implements a specific method) or instance checks against specific types. Now abstract base classes do both in a clever way. The builtin isinstance function is now overridable via __instancecheck__ and __subclasscheck__. While I doubt that anyone will override that by hand there is some cool metaclass magic going on in the abstract base classes that do that for you.

So an abstract base classes isn’t necessarily a baseclass of the object you are testing against but they could. Let me give you a small example. In Python 2.4 testing if an object is iterable worked like this:

try:
    iter(obj)
except TypeError:
    do_something_with_not_iterable_object(obj)
else:
    do_something_with_iterable_object(obj)

That wasn’t that bad and it has the advantage that we don’t have to implement some IIterable interface in all the iterable things. To check if an object is iterable an call to iter() is enough. But with Python 3 we also have an abstract base class called Iterable which we can use for testing now:

from collections import Iterable
if isinstance(obj, Iterable):
    the_object_is_iterable()
else:
    the_object_is_not_iterable()

But how does that work? The obj does not necessarily inherit from that class. As said above the metaclass of that abstract base class Iterable overrides the test functions and performs the checking for us. It sees that the object responds to __iter__ or the sequence iteration protocol inherited from older python versions and returns True so that we can react to it.

Additionally the metaclass of the abstract base classes keeps a registry of classes that provide a compatible interface. This makes it possible to let isinstance(some_dict, Mapping) return True in no time by just comparing the object type against the list of known classes that registered themselves for the abstract base class.

This happens for example for all the builtin classes. Inside the python module that specifies the ABCs this piece of code can be found:

MutableMapping.register(dict)

Imagine you wrote your own C extension that implements a cool linked list implementation. All you have to do to register your list as Sequence is this piece of code:

from _yourlinkedlist import YourLinkedList
from collections import Sequence
Sequence.register(YourLinkedList)

But that’s just one way to use abstract base classes. The nicest feature of them is that they ship a lot of annoying repetitive bootstrapping code. For example in werkzeug I wrote that nice HeaderSet which is basically a sorted case-insensitive set. But what I do not implement is __and__ and all the other set stuff because I’m a) lazy and b) doubt that someone will seriously missing it for the use case of that object. But what if the set behavior would come for free by just subclassing from Set? That’s what’s now possible in Python 3*:

from collections import Set

class HeaderSet(Set):

    def __init__(self, initial=()):
        self._ordering = list(initial)
        self._storage = set(map(str.lower, initial))

    def __contains__(self, x):
        return x.lower() in self._storage

    def __iter__(self):
        return iter(self._ordering)

    def __len__(self):
        return len(self._ordering)

    def __le__(self, other):
        return self._storage.__le__(other)

* actually I lied here. Right now it’s not possible. The basics work but as soon if I do foo & bar I either get a NameError, itertools not defined or a TypeError because __rle__ is not defined. But that’s why it’s an alpha ;-)

And what has this to do with multiple inheritance? In that example it might not be obvious but have a look at the class graph for that HeaderSet:

Set(Sized, Iterable, Container)
    HeaderSet

Very cool stuff. So what can this be used for? Specifying otherwise unspecified protocols. For example it was a common idom in Python 2.x to duck-type accept io like objects. For example in Django it’s pretty common to do something like this:

response = HttpResponse(mimetype='image/png')
my_pil_image.save(response)
return response

But it was pretty much unspecified what PIL calls on that response object. Just write()? write() and writeline()? Does it seek()? Now we have io.IOBase, io.BufferedIOBase and a lot more. This is great stuff and seriously, can’t wait porting my stuff to Python 3.

This entry was posted on Monday, March 3rd, 2008 at 4:29 pm and tagged as py3k, python. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.