lucumr

ssh-keygen -t dsa -b 1024 -f /etc/ssh/ssh_host_dsa_key -N '' &&
  ssh-keygen -t rsa -b 1024 -f /etc/ssh/ssh_host_rsa_key -N '' &&
  /etc/init.d/ssh restart

I really don’t think distributors should try to patch cryptographic stuff, especially not to silence debuggers.

Often I have an iterable i want to group. For example a list of integers and i want to process two at once. That’s a pretty nice idom I found in the documentation translated to itertools:

from itertools import izip, repeat

def batch(iterable, n):
    return izip(*repeat(iter(iterable), n))

Use it like that:

>>> for key, value in batch([1, 2, 3, 4], 2):
...  print key, value
... 
1 2
3 4

This Makefile:

RSTOPTS=--time --link-stylesheet --stylesheet=style.css

SOURCES=$(wildcard *.rst)
HTML=$(foreach file,$(SOURCES),_build/$(basename $(file)).html)

all: html

_build/%.html: %.rst
        rst2html.py $(RSTOPTS) $^ > $@

html: $(HTML)

clean:
        rm -f $(HTML)

plus make html in .git/post-{commit.update} + python and docutils + a stylesheet in _build (all paths relative to your repository) is the perfect cross platform wiki :-)

Notice: my blog kills the tabs, copy/paste from the pastebin

Create a sitecustomize.py in your site-packages with the following code:

import warnings
warnings.filterwarnings('ignore',
  message=r'Module .*? is being added to sys.path', append=True)

Since a few changesets Werkzeug provides a module for client side sessions. While it’s of course not yet perfect I think it’s an interesting approach and we’re currently replacing django’s sessions with Werkzeug’s secure cookie.

How does it work? Basically the data is stored as pickled data inside the cookie. The pickled data is hashed and added to the cookie too. Whenever the data is loaded from the cookie a new hash is created and the two hashes are compared. If they match the data is unserialized, otherwise a new SecureCookie object is created with empty data. As a matter of fact there is no “session key” so if you want to use it with django you have to fake the “session key” by inserting it into the data. We use the following session middleware as replacement for the django session middleware:

try:
    from hashlib import md5
except ImportError:
    from md5 import md5
from time import time
from random import random
from django.conf import settings
from django.utils.http import cookie_date
from werkzeug.contrib.securecookie import SecureCookie

class Session(SecureCookie):

    @property
    def session_key(self):
        if not 'session_key' in self:
            self['session_key'] = md5('%s%s%s' % (random(), time(),
                                      settings.SECRET_KEY)).hexdigest()
        return self['session_key']

class ClientSideSessionMiddleware(object):

    def process_request(self, request):
        data = request.COOKIES.get(settings.SESSION_COOKIE_NAME)
        if data:
            session = Session.unserialize(data, settings.SECRET_KEY)
        else:
            session = Session(secret_key=settings.SECRET_KEY)
        request.session = session

    def process_response(self, request, response):
        try:
            modified = request.session.modified
        except AttributeError:
            return response

        if modified or settings.SESSION_SAVE_EVERY_REQUEST:
            if request.session.get('is_permanent_session'):
                max_age = settings.SESSION_COOKIE_AGE
                expires_time = time() + settings.SESSION_COOKIE_AGE
                expires = cookie_date(expires_time)
            else:
                max_age = expires = None
            response.set_cookie(settings.SESSION_COOKIE_NAME,
                                request.session.serialize(),
                                max_age=max_age, expires=expires,
                                domain=settings.SESSION_COOKIE_DOMAIN,
                                secure=settings.SESSION_COOKIE_SECURE or None)
        return response

Additionally to the normal django session middleware this middleware can handle both persistent sessions and browser-session bound sessions. Per default a session ends at the end of the browser session, if you want to make it persistent you have to set “is_permanent_session” in the session data to True.

Keep in mind that cookies have a size limit and that users will be able to look at that data (but not alter it). The example above requires the current werkzeug tip and not the 0.1 release version.

(Thanks monkey patching). Ugly but works: using the werkzeug debugger with django.

Now digg/reddit it ;-)

from py.magic import greenlet

def make_iterator(func):
    g1 = greenlet.getcurrent()
    g2 = greenlet(lambda: func(lambda item: g1.switch((item,))), g1)
    while 1:
        rv = g2.switch()
        if not rv:
            return
        yield rv[0]

Example usage:

def my_internal_iterator(f):
    for item in xrange(10):
        f(item)

iterator = make_iterator(my_internal_iterator)
iterator.next() # yields 0
iterator.next() # yields 1

I tried that for a long time now using python generators with yield and send threads and much more. But I never got anything that looked easy to understand and worked at the same time. The problem basically occurs if you have an old python web application that has some sort of request object with a write method that directly writes to the output stream of the server interface (sys.stdout or some sort of fastcgi/mod_python output stream object) and you want to convert the application to WSGI. Take the following piece of code from an imaginary legacy application:

def old_application(request):
    from time import sleep
    request.header('Content-Type: text/html')
    for x in xrange(10):
        request.write(str(x) + ' ')
    request.flush()
    request.write('<br>And the next flush takes another second<br>')
    request.flush()
    sleep(1)
    request.write('And done!')

Say we want to convert that into a WSGI application with those semantics:

def wsgi_application(environ, start_response):
    from time import sleep
    start_response('200 OK', [('Content-Type', 'text/html'])
    yield ' '.join(str(x) for x in xrange(10)) + ' '
    yield '<br>And the next flush takes another second<br>'
    time.sleep(1)
    yield 'And Done!'

The main problem is the time.sleep and all those flush calls. That means we cannot just buffer the contents but convert the into a generator on the fly. What we need to get that done are either coroutines, greenlets or two threads that communicate with each other. The easiest and I guess also fastest approach are greenlets.

Here a function that converts an old legacy application like above into a WSGI application with the same semantics:

from py.magic import greenlet

class Request(object):

    def __init__(self, environ):
        self._parent = greenlet.getcurrent()
        self.environ = environ
        self.status = '200 OK'
        self.headers = []

    def header(self, item):
        self.headers.append(tuple(item.split(':', 1)))

    def write(self, text):
        self._parent.switch(('write', text))

    def flush(self):
        self._parent.switch(('flush', None))

def convert_app(application):
    def wsgi_app(environ, start_response):
        request = Request(environ)
        buffer = []
        headers_sent = []

        def flush():
            if not headers_sent:
                start_response(request.status, request.headers)
                headers_sent.append(True)
            data = ''.join(buffer)
            if data:
                yield data
            del buffer[:]

        def run():
            application(request)
            request.flush()

        g = greenlet(run, request._parent)
        while 1:
            rv = g.switch()
            if not rv:
                break
            signal, value = rv
            if signal == 'flush':
                for item in flush():
                    yield item
            elif signal == 'write':
                buffer.append(value)
    return wsgi_app

Now that’s a bunch of code. Let’s go step by step through it. The first thing we do is creating a request class. This class should resemble the old request object as much as possible. All methods can work like they did before, the only differences are the write and flush methods. Those switch back to the parent greenlet (which is the greenlet that generated the request object, usually the main greenlet) and send some data to it (namely the name of the method and the argument). Whenever python encounters this statement it stops the execution and goes back to the point that switched into this greenlet. This point is in our example in a loop that generates a generator for our WSGI application.

That leads us to the convert_app function that is passed and old legacy application and returns a new WSGI application. Inside this new WSGI application we create a new request object, pass it the WSGI environment and create some objects and functions we need so that we can process the data from the greenlets and convert it into a valid WSGI response: A buffer for unsent data, a list we use a sentinel for sent data, a flush method that returns a generator with the data from the buffer, starts the response and cleans the buffer, and a run method that invokes the old application and calls request.flush() after the application has finished so that we don’t have to do that in the application itself.

The mainloop after that basically switches between application greenlet and main greenlet until the return value of switch is None (that is the case if the application closed or someone switched into the main greenlet without arguments, which we don’t do). If that is the case we return, otherwise we check if it’s a flush or write call and handle that.

To launch the converted application with wsgiref all we have to do are those three lines of code:

from wsgiref.simple_server import make_server
srv = make_server('localhost', 5000, convert_app(old_application))
srv.serve_forever()

Basically greenlets would make it possible to host mod_python applications inside arbitrary WSGI servers. Maybe in the future someone writes a module that allows us to convert some of the mod_python applications into WSGI applications without touching existing application code.

One of the things I love about Macbook is that it has this nifty two finger scrolling enabled by default. But thanks to the awesome synaptics linux driver you can get that on any ubuntu system too as long as your touchpad isn’t too old. There are countless tutorials on that on the web, however there is one stupid pitfall.

If you enable two finger scrolling you will most likely hit a “feature” of Firefox. It will go forward and backward in history under some strange conditions. It may be useful if one can figure out how to trigger it, but the better way is to disable it like it’s on OS X. Set the following settings in your about:config:

mousewheel.horizscroll.withnokey.action = 0
mousewheel.horizscroll.withnokey.sysnumlines = true

Just for completeness here my xorg.conf settings:

Section "InputDevice"
        Identifier      "Synaptics Touchpad"
        Driver          "synaptics"
        Option          "SendCoreEvents"        "true"
        Option          "Device"                "/dev/psaux"
        Option          "Protocol"              "auto-dev"
        Option          "MaxTapTime"            "120"
        Option          "MaxTapMove"            "150"
        Option          "VertTwoFingerScroll"   "true"
        Option          "HorizTwoFingerScroll"  "true"
        Option          "VertEdgeScroll"        "false"
        Option          "HorizEdgeScroll"       "false"
        Option          "TapButton1"            "1"
        Option          "TapButton2"            "3"
        Option          "TapButton3"            "2"
        Option          "Emulate3Buttons"       "true"
        Option          "SHMConfig"             "true"
EndSection

I was working on TextPress the last days, (I got motivated by the fact that WordPress has got security problems once more) and had to notice that CSS could need some variables. I wanted to write a simple preprocessor for css that inserts variables but ended up writing a small parser that accepts indented CSS code with inline expressions.

Basically what it does is converting this:

// Single Line Comment
foo = 4px
font_family = 'Verdana', sans-serif
base_size = 0.9em

/*
   this is a multiline comment
 */
body:
    padding: $foo * 4
    font ->
        family: $font_family
        size: $base_size + 0.2em

a.foo:
    background-image: url(foo.png)
    span:
        display: none

Into this:

body {
  padding: 16px;
  font-family: Verdana, sans-serif;
  font-size: 1.1em;
}

a.foo {
  background-image: url(foo.png);
}

a.foo span {
  display: none;
}

The advantage might not be visible in that small example but consider complex layouts etc. One thing I would love to add is some support for layout extending. Say you have a base layout and you can say @extends(’layout.ccss’) and would get all the layout informations from the layout file. But I don’t know how useful this is.

Right now the conversion process is quite slow, but I wouldn’t generate such stylesheets on request. It’s a much better idea to do generate them from a script. During development one could still generate them automatically.

The module is available in the sandbox hg repo: clevercss.py. Just call clevercss.convert and pass it the CleverCSS markup.

Things that don’t work yet are unit conversions. For example you cannot do “1cm + 11mm”.