Armin Ronacher

Hiho to my first python tutorial. Julian Krause aka thecrypto aka Mr. RhubarbTart mentioned the eigenclass article Plugins in your Ruby Application and noticed that there is no such tutorial for python. So: here it is ;-)

Since version 2.2 python ships a new object model (explained in the documentation) which allows very complex class definitions including metaclasses and other features you probably won't use. But some of them are very useful if you want to create a plugin system.

The hardest part is the loading of these plugins. There are basically three ways for loading plugins:

• using an additional folder `plugins/` which gets added to the PYTHONPATH (sys.path)
• writing an import hook which will handle that (I won't talk about this method here since it's very complex and not necessary in most cases)
• Using setuptools as plugin system

This tutorial will just cover the "old" approach without the use of setuptools. A latter tutorial will introduce setuptools as well, but at first just the plain-old-python way.

Each plugin should inherit from a special base class which keeps track for all of it's childclasses.

class Plugin(object): 
    pass

Now we have a base plugin :-) What is so special about this class? Nothing except that this class inherits from the special base class called object which no metaclass. If you don't inherit from object (if you don't define a parent for your class which is object or a subclass from object) python adds types.ClassType as metaclass to your class.

Every newtype class knows about it's children:

>>> class MyPlugin(Plugin): pass
...
>>> class OtherPlugin(Plugin): pass
...
>>> Plugin.__subclasses__()
[<class '__main__.MyPlugin'>, <class '__main__.OtherPlugin'>]

Very useful for plugins. As long as they where actually loaded. So we need to load all required plugins. Now we have two possibilities:

• use a config file where you enter each plugin
• scan the plugin folder on application launch and import everything that ends with .py or is a folder with a __init__.py in it.

In this tutorial I'm just covering the first case since the setuptools approach automagically imports plugins later.

import sys
import os


class Plugin(object):
    pass


def load_plugins(plugins):
    for plugin in plugins:
        __import__(plugin, None, None, [''])


def init_plugin_system(cfg):
    if not cfg['plugin_path'] in sys.path:
        sys.path.insert(0, cfg['plugin_path'])
    load_plugins(cfg['plugins'])
    

def find_plugins():
    return Plugin.__subclasses__()

Now save it as plugins.py. To enable the plugin system you can now use init_plugin_system:

>>> from plugins import init_plugin_system, find_plugins
>>> init_plugin_system({'plugin_path': 'plugins/', 'plugins': ['testplugin']})
>>> find_plugins()
[<class 'testplugin.SpecialPlugin'>, <class 'testplugin.AnotherPlugin'>]

testplugin.py was just a small test file with two classes inheriting from plugins.Plugin.

The next a plugin needs to do is to tell the programmer of the application it's capabilities. Therefore you could use Interfaces, duck typing, subclassing or just a list with exported capabilities. The latter is very basic and convenient therefore we implement it:

class Plugin(object):

    capabilities = []

    def __repr__(self):
        return '<%s %r>' % (
            self.__class__.__name__,
            self.capabilities
        )

def get_plugins_by_capability(capability):
    result = []
    for plugin in Plugin.__subclasses__():
        if capability in plugin.capabilities:
            result.append(plugin)
    return result

Now an example plugin defines one or more capabilities. The names of the capabillities should be explained somewhere. As well as the methods the plugin should export.

from plugins import Plugin

class ExamplePlugin(Plugin):
    capabilities = ['foo']

    def do_foo(self, name):
        return 'Hello %s!' % name

Saved as testplugin.py you should be able to load it:

>>> from plugins import init_plugin_system, get_plugins_by_capability
>>> init_plugin_system({'plugin_path': '.', 'plugins': ['testplugin']})
>>> for plugin in get_plugins_by_capability('foo'):
...     plg = plugin()
...     print plg.do_foo('Huhu')
...
Hello Huhu!

This example also demonstrates the current problem. You have to create a new instance after each iteration since get_plugins_by_capability yields classes, not instances.

But it's possible to fix this:

_instances = {}

def get_plugins_by_capability(capability):
    result = []
    for plugin in Plugin.__subclasses__():
        if capability in plugin.capabilities:
            if not plugin in _instances:
                _instances[plugin] = plugin()
            result.append(_instances[plugin])
    return result

Now the function returns singleton instances, first created when required:

>>> from plugins import init_plugin_system, get_plugins_by_capability
>>> init_plugin_system({'plugin_path': '.', 'plugins': ['testplugin']})
>>> for plugin in get_plugins_by_capability('foo'):
...     print plugin.do_foo('Huhu')
...
Hello Huhu!
>>> from plugins import _instances
>>> _instances
{<class 'testplugin.ExamplePlugin'>: <ExamplePlugin ['foo']>}

done :-) You now should have your own plugin system in 38 lines of code ^^ Of course there is still much too do but it's enough for a very basic plugin system. I'll introduce you to the setuptools plugin way tomorrow :-)