lucumr

So rails is ditching trac and subversion in favour of git and lighthouse. Additionally they won’t host git themselves but rely on github. Keeping in mind that the preferred development environment called OS X + TextMate, rails is now kneedeep in closed source land. Something I could care less about as I’m thankfully not using rails or any rails powered application. But what hit me was the discussion attached to that blog post. Apparently there are quite a few pissed of Windows users left. Gosh, they still exist!

David “Fuck You” Hansson could really care less as he stated in the past “that he would have a hard time hiring a programmer who was still on Windows”. Skimming through the blog you can see enough mac users telling the windows crowd to switch and I think you can nowhere find as many mac users as in the rails community. He really did a great job converting proprietary windows developers into even more proprietary Mac users. There are some really great quotes in that blog:

The proper move is to stop using Windows for development of RoR apps.

or

Windows users, please stop complaining about others using the right tool for their jobs and just start doing the same.

You can get rid of this shit and use something serious. Really. Others are doing it and nobody ever looks back. So you can do it, too.

Yeah stupid folks. Why in god’s name are you still using windows, that proprietary crap from redmond. DDH your Führer already told you to use OS X, do so and don’t ask questions. Interestingly though is that Linus himself is not that happy of OS X (the link goes to another post, I was refering to the reply, but that one is good too) as their filesystem handling is beyond broken, at least for git’s requirements.

So I’m asking. What exactly makes OS X better than Windows? Nicer hardware? I would count that as a major disadvantage of OS X as you’re bound to vendor. The Apple support here is very weak (where weakhere is Austria) as far as I’ve noticed. Sending a notebook in for two weeks for a single broken key on the keyboard is ridiculous. Additionally you’re paying a lot of money for it. I know that the design is the selling point of Apple computers but that doesn’t mean all PCs are looking bad. A Thinkpad or one of the more expensive Sony Notebook next to an Macbook Pro and it’s pretty hard to decide what looks better. Especially in terms of quality. My Macbook Pro is a bit more than half a year old and on the silver plastic you can spot where the palms of my hands are as the color is chipping off there. But even if the hardware was the selling point you could use other operating systems on it.

There is TextMate (which was the main reason why I personally bought one) but as it turns out, it looks better in the screencasts than in real world. If you’re used to a different editor it’s hard to switch. I tried multiple times and every once in a while i was cursing why :vs didn’t work out. Together with the braindead keyboard layout on the apple computers (probably an issue you only have on German macs) gives you a ridiculous feeling. When I switch between Windows and OS X (which happens quite a lot recently) I permanently mix things up. Windows after OS X locks me out every once in a while while writing mails (as alt+l is windows+l on a windows pc, which apparently is equivalent to “lock workstation”). The other way round I close my mailer as alt gr + q (which on a German layout gives you the at symbol) is command + q on a mac, which means “close application”. Can’t count the times I killed my vim/thunderbird that way.

One point where OS X shines is font rendering. I think for artists OS X is a reasonable good choice as operating system. The system itself has a good understanding of fonts and all that but for me as developer the rendering freaks me out. In a regular gvim with bright fonts on dark backgrounds it’s especially anonying as everything looks bold and there is no visible difference between bold and nobold. I know that Windows was flamed in the past dozens of times for aligning the fonts on the pixel raster but quite frankly, I prefer that over the OS X way. It might be true that Windows will block the 300dpi screen era for another ten years or so, but so will Linux and OS X. Most of the websites or applications still depend on pixel values, and even if web designers will switch to the SVG or better vector formats, you can’t ignore the old websites. So the problems with high resolution screens are clearly not the windows font rendering.
Beside that I want to point out recent ubuntu releases have got real good font rendering thanks to the turner patches for subpixel rendering and with some additional configuration and better fonts you get everything from the OS X to Windows ClearType like rendering.

But the real problem with OS X is that it tries to be Windows and Linux at the same time and fails miserably. For me as linux user the most important detail of linux is missing: the package manager. But at the same time the beloved setup.exes are missing. Dmgs with application bundles are a nice thing in theory but they don’t work out that well. There are neither uninstallation tools norr do they provide a user experience that makes sense. The OS X mouse behavior is ridiculous and installing applications via drag/drop is just crazy. The first thing I did when I got my Mac was dragging the Firefox out of the DMG into the Dock. Then I noticed that I want to have it in the applications too and draged it there. What happened? I deleted it. The same way I managed to delete a file when moving from my harddrive to a network share. I lost the wireless connection and the file was lost. That’s ridiculous and must never ever happen on an operating system. Application management on OS X is stupid anyways. To fully remove an application from your computer you better buy a shareware that is better as that. Seems like most functionallity you want to have on a computer comes as third party application on OS X.

And why in god’s name is there no cut/paste of files in the finder? I don’t want to start ranting about the finder as I think it’s OS X weakest point anyways but that’s a stupid limitation. Drag/Drop works but Cut/Paste not? Some 1337 terminal hackery later I got Cut/Paste support in Finder but then I had to notice why it’s not enabled by default. Because the Apple implementation of this simple but useful feature is broken. If you cut a file, and cut another, not only the latter is cut. No! The first one is moved to the trash. WTF? Not even Nautilus does that wrong, and Nautilus does tons of things wrong right now.

Then OS X is inconsistent. “:” in the Finder is “/” in the terminal and the other way round. Fortunately you don’t see location bars anywhere otherwise you would notice that. Unfortunately there are no location bars so you end up using the mouse all the time. Speaking of the broken mouse: mousing on OS X files like mousing through meter deep mud. Just google for “OS X mouse” and the first hit is “mouse acceleration problem” :-/. Oh and yes, you can fix that. Either buy a shareware or hack together an application that uses a deprecated interface in the IOkit. After one week of OS X mousing my hand hurt, something I don’t have on any other operating system, no matter how hard I try. I ended up using the touchpad of my Macbook as the mouse acceleration is better there.

But back to application management for a second as that goes back to the original topic: development environment. On ubuntu you have that cool GNU userland I’m used to. Not only that, you also have a kick-ass package manager that keeps my whole system up to date. On OS X you have a BSD userland which is irritating if you’re used to GNU tools. Many applications that seems to work out of the box on ubuntu and with a cygwin installation on windows too, work completely different on OS X. Like python. Why? Because of OS X bundles, frameworks and all that fancy stuff that you have to face when developing on OS X. For example on Tiger the python executable is in “/System/Library/Frameworks/Python.framework/Versions/2.3/bin/python2.3″ and the standard library in “/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages”. A non-framework Python comes via port which is installed to “/opt/local/bin/python2.3″. The cool thing: it’s a regular python executable like you know it from Linux or Windows. The problem? It’s not in a framework, so no wxPython for you. What frameworks do? No clue man. It’s a bundle, that’s what I know. Quite frankly I don’t want to know what’s the fucking difference, it’s just annoying. Teeworlds for example has a broken mouse behavior when started from a bundle. Why? No clue. But outside of a bundle it works. I won’t argue that it’s broken, it probably comes from the old Next times. But as windows or Linux user it’s irritating. Windows is easy to understand, linux is not much more complicated once you groked where the stuff is located, OS X is just irritating.

And the biggest problem of all for me: It’s slow. Freaking slow. Slower than ubuntu on my old notebook which was a 2GHz dothan with only 512 MB RAM. And GTK is slow, very slow. All kinds of gvims I tried on OS X are so slow that you can see the refreshing while scrolling, especially if more complex highlighting is activated. My python interpreter does something 0.3 seconds every startup, so does ruby. The network is slower here too. I don’t know if it’s the Wireless LAN chip or OS X in general, but if I start up my old notebook and the OS X one, transfering files is a lot faster. And by lot I mean I haven’t benchmarked it, but I can see the difference.

Then let’s come to security. Quite frankly I have no idea how secure the system really is, but the number of security updates is annoying. While I think it’s cool that they are patched, you’re downloading something like 300MB security updates per month I think and most of them require a reboot. Feels a lot like windows, just that I don’t have to reboot windows when a Windows Media Player update is installed.

But what’s the conclusion? It’s certainly not that OS X sucks. If you like it, feel free to use it. But telling other people that they are stupid because they can’t see the ingeniousness of all Apple products is just ignorant. OS X is just another proprietary operating system, and not the solution for all of your problems. It has it’s problems too and it doesn’t have any real advantages over Windows beside a nicer design and nicer application design and the fact that it has such a low userbase that you’re not the target of malware authors. At least not currently, let’s see if that changes the next four years. With such a homogeneous environment it will become a nice target for attacks at least.

Oh. And I don’t think that linux on the desktop will be the solution either. As long as patents exist or the kernel doesn’t allow binary drivers and KDE and GNOME people can’t settle on one architecture and HIG linux on the desktop is on the best way to become the most sucking operating system on the desktop for the regular user. But for developers? A good choice!

So let’s conclude: every OS sucks. And OS X is no exception. So don’t judge users by their prefered operating system / desktop environment. And don’t tell them to use something else just because you’re too lazy to adapt.

Georg already blogged about the new ast to code compilation feature of Python so I won’t cover that any more. Basically the old compiler package is nor surpassed by python’s real internal AST which is freaking awesome :)

The only thing that is missing is some sort of high level interface to it which makes manipulating and debugging it easier. The motivation behind it is that template engines such as Genshi, Jinja or Mako which all operate on the AST don’t have to write that much boilerplate code just to modify small pieces of the AST.

I don’t know if this module will make it into the standard library but even if not it will surely be available via the cheeseshop. So what does it do? It provides classes and utility functions to manipulate or traverse the AST in a way that is actually useful for real world applications. For example with the transformer it’s not possible to replace and remove nodes. Additionally it’s possible to generate python sourcecode from an AST to simplify debugging.

Here a small example of a transformer that brings Genshi like Loop behavior to a piece of Python sourcecode:

class GenshiSemantics(ast.NodeTransformer):

    def context(self, method):
        return ast.Expr(ast.Call(
            ast.Attribute(ast.Name('data', ast.Load()), method, ast.Load()),
            [], [], None, None
        ))

    def visit_For(self, node):
        self.generic_visit(node)
        return [self.context('push'), node, self.context('pop')]

    def visit_Name(self, node):
        self.generic_visit(node)
        return ast.Subscript(
            ast.Name('data', ast.Load()),
            ast.Index(ast.Str(node.id)),
            node.ctx
        )

When applied this piece of code:

seq = [1, 2, 3, 4, 5]
item = 42
for item in seq:
    print 'inside', item
print 'outside', item

becomes

data['seq'] = [1, 2, 3, 4, 5]
data['item'] = 42
data.push()
for data['item'] in data['seq']:
    print 'inside', data['item']
data.pop()
print 'outside', data['item']

The sourcecode of the module is in the Pocoo sandbox as usual. Unfortunately there is a bug in the initialization function of the ASTs right now so you will need a patch for it. And obviously this requires Python 2.6.

The full sourcecode for the example above can be found here.

So Python 3 is getting ready and looking at all the libraries I maintain I was really afraid of the final release. Looking at all those language changes it sounded like a horrible job to do. But there is that nice 2to3 script which should transform sourcecode from python2 to python3 in a semiautomatic manner. And what should I say? It kick ass.

Getting the current hg head of Jinja running on top of Python 3 was a matter of roughly 15 minutes. The only things I had to change was adding a missing sys import that came from the intern -> sys.intern translation and fixing a couple of sort() calls. Sort now requires you to use a key function. But that was easy to solve and it was only there for python 2.3 backwards compatibility anyways. The other minor thing I had to change was unicode behavior. Previously Jinja has had a env.to_unicode function that coerced bytestrings and unicode strings into unicode strings depending on the encoding setting on the environment. With Python 3 everything will use unicode internally so no need to have an environment charset anways so I replaced env.to_unicode with str everywhere. Theo the other unicode change was replacing file(foo, 'r').read().decode(charset) (pseudocode) with file(foo, encoding=charset).read().

That’s it! Well. I must admit Jinja is by far the easiest templating language to port from the trio of templating languages I use (Jinja, Mako, Genshi) as it has it’s own parser and does not relay on the now removed compiler module which was superseded with the new _ast with a different structure. So it clearly depends on what you’re using in your library how the quality of the conversion will be. For most libraries 2to3 will do a very good job.

Oh. What it also doesn’t convert are C extensions :) I don’t know if the Jinja traceback tools or speedups module work in Python 3 too as they are optional and I was too lazy to compile them, but that’s another thing you have to keep in mind. Also I was unable to run the Jinja testsuite to run everything as py.test doesn’t work on Python 3 so far.

But I’m very happy. If the process stays that simple my forecast of not using Python 3 before 2012 may be wrong. So, go on, port your libraries now, at least for testing purposes and let’s make some of the best available right with the Python 3.0 release which is scheduled for September. If the initial number of working libraries is high enough for some applications it improves Python’s changes for a quicker adoption a lot. I know it’s very unlikely for big projects to switch in a short timeframe but at least smaller projects will be able to benefit from Python 3 earlier.

And even if Guido disagrees: Python 3 is *the* change to break *small pieces* of the API. Don’t break them in a way that everybody is confused and doesn’t know how things work any more. But if there are some design flaws in the library you want to change adapt them with the change to Python 3 rather than between two Python 2 versions. (And of course document them!)

The TurboGears project has been accepted as a mentoring organization for the 2008 Google Summer of Code. Even if you’re not interested in TurboGears because your framework of choice is something else you might still be interested in that one as it includes two Genshi project ideas: Performance and Jython compatibility.

If you have a solid knowledge of XML/HTML, Python and you’re looking for a GSoC project that’s interesting, read Christopher’s blog post about it and go for it :-)

As mentioned in an earlier blog post I’m not a fan of full automatically generated documentation as I think it’s clearly the wrong way to solve documentation problems. Whenever I encounter epydoc generated documentation and I find out that that’s the only kind of documentation I run away :)

In the past there have been two kinds of documentation in the python world: handwritten documentation (Django for example) and full automated API documentation (Paste’s). Both of them have advantages and disadvantages for user and developer but none of them was perfect. At least for me. Django’s documentation is one of the best I ever encountered but writing such documentation yourself is painful. I tried to do something similar with Werkzeug and Jinja and it sucks. On the one hand because usually you start improving stuff and add the documentation later on, often forgetting about it. It’s not unlikely that the documentation is slightly different from the actual implementation because someone (usually me) forgot to “sync” them.

With Werkzeug 0.2 I was experimenting with combining those two things. I added some directives to docutils that pulled docstrings automatically from the objects and added them to the rst file. That way the documentation is perfectly in sync with the sourcecode and because the members are specified explicitly I can hide implementation details and private functions. However the Werkzeug documentation builder was and is a hack and was never meant to be used by anyone except the Werkzeug project, and even that one just for one release. The reason for that is that Georg Brandl spent the last couple of weeks rewriting and improving parts of the python documentation tool which powers the new Python 2.6 and 3.0 docs to support non cpython projects too.

The resulting library Sphinx is in my opinion the best general purpose documentation tool since sliced bread! It’s intended to be a tool for handwritten documentation that builds a documentation into standalone HTML files, CHM HTML files, LaTeX or pickles which can be used to display the documentation in a WSGI application.

It uses reStructuredText as markup language for all the documentation, supports syntax highlighting of code blocks via pygments, embedded doctests so that you can extend your testsuite with doctests from your documentation, has support for automatic object documentation by including the docstrings from objects listed (semiautomated documentation as I did with Werkzeug), automatic cross linking, index generation, changelog generators, many custom roles and directives for rst and much more.

While it’s the first release it’s already a very good documented and well tested library and used for the Python documentation. While I hate the word “framework” I think Sphinx could be that for documentation tools. The extension API can be used to add missing features and may also be used for more automated documentation generation in the future.

Unfortunately I wasn’t that active in the implementation of Sphinx so far, so I’m clearly the wrong person for further questions about the development direction of Sphinx but I’m sure Georg will answer your questions. You can contact him in #pocoo on irc.freenode.net or via E-Mail at georg guesswhichcharcomeshere python dot org.

Links:

• Sphinx Homepage
• Sphinx in the Cheeseshop
• Georg’s blog

Many of you probably already know that I’m one of those metalheads. But mainly because the number of concept albums is a lot higher compared to other genres and that most of the songs are very technical and tell a story. There is one metal subgenre I like most: progressive metal. There are tons of bands in said genre and some of them go into the direction of progressive rock (or the other way round of course). I would compare progressive metal more with Jazz and classical music than with thrash metal or death metal (both subgenres I like!) which are what people think of when hearing about metal.

But I think progressive metal is one of the not that extreme genres you usually don’t hear on the radio. The reason for this is probably that the average length of the song is far beyond the usual four minutes and because the songs don’t make that much sense if you don’t listen to them in the context of the rest of the album. Whenever I play selected songs to my pals that don’t listen to progressive normally the reaction is: “hmm. sounds not that bad, but I wouldn’t listen to that myself.”. And yes, progressive metal is not that kind of music you would listen to in the background only. It’s that kind of music you listen to like you read a book.

If you are a metal fan and you don’t know one of the albums below: visit the pirate bay, download one of them and listen to them (And if you like them, buy the CD/send the musicians money, whatever). The following list is my all time favorite collection of progressive metal albums.

Pain of Salvation — One Hour by the Concrete Lake tells the story of a stranger that works in the weapon industry who begins to have doubts about the morality of his profession. He realizes that he’s just a part of a big “machine”. The album then follows him on his voyage around the work that finally ends at Lake Karachay, the concrete lake, a lake in the former USSR where so much nuclear waste was dumped over the past fifty years that if one stood by the shore for one hour the radiation would be lethal.

For me it’s hard to say which of the POS albums is the best one and I really can’t say but “Concrete Lake” is one I can listen to over and over again. Both from a philosophical and musical point of view.

Dream Theater — Octavarium is a tricky beast. While it’s not a concept album that tells a story it certainly has a concept (a very complex one) behind it. It starts with the fact that countless things in the artwork, songs somehow have to do with the number eight and continues with ascending keys in the eight songs. The first song is in F, the negative time of the second one F#, the second one G, the negative of the third is G# and so on, all the way up to the next F which finishes the octave and the album. You can read a detailed analysis of the album at spatang.com.

It’s hard for me to select the Dream Theater album as all of them are incredible pieces of music but Octavarium is one of my personal favorites and every song on the album is totally different.

Opeth — Still Life is a great death metal – progressive metal cross-over concept album telling the story of a young Christ that discovers that Christianity isn’t exactly what he thought it would be and gets expelled from his home town. 15 years later he returns and is looking for his former love “Melinda” just to find out that she become a nun in the meantime. He asks her to come with him and she tells him that she still loves him but will not break her promise to the church. Later she’s killed by a soldier and the protagonist in rage kills the soldier and everyone he meets until he breaks down in total exhaustion. It ends with him being executed and right before his death he thinks he sees Melinda once more waving and watching him.

I think that album was the first real success of Opeth and if you like death metal elements it’s an incredible good one. If you like the story or not is a completely different thing but that shouldn’t stop you from enjoying the music.

Ayreon — 01011001 is one of the Ayreons albums, my personal favorite but probably not the best choice to start listening to Ayreon as the album somewhat continues the story from earlier albums. The album title is actually Y (1011001 is 89 which is the ASCII code for Y) tells the story of the planet Y and the seafaring ‘forever’ that lost their emotions and send their DNA to the earth in order to rediscover their emotions. Their comet extincts the dinosaurs and seed humanity. The experiment however fails as humanity becomes more and more depending on technology over time and in the end all tries to save them fail. The ‘forever’ leave the planet together with the migrator from a former Ayeron album.

Personally my favourite Ayreon album mainly because the synthesizers transport a very “spacey” sound and you can feel the story. Additionally the singers on the album (among others Hansi Kürsch from Blind Guardian, Daniel Gildenlöw from Pain of Salvation, Floor Jansen from After Forever and many more) are split up into man and forever and their duets and given roles match their characters and the characters in the story perfectly.

Blind Guardian — Nightfall in Middle-Earth is strictly speaking more power metal than progressive metal but an awesome concept album. The album is based upon Tolkien’s “The Silmarillion”, a book of tales from the First Age of Middle-earth. The album contains not only songs but also spoken parts narrating parts of the story. A true masterpiece and a great album to listen to while playing warhammer.

There are tons of good conceptual albums and progressive metal bands but time is rare and I could only pick a few of them. As I said earlier: if you like metal but never “tasted” progressive metal so far, give it a try :-)

Normally I try to avoid posting what is on digg because nobody is interested in that any more once it’s up there, but I could not resist this time. NIN’s Ghosts is available as download. What’s so awesome about it? It’s CC-NC licensed, Trent Reznor uploaded the first CD himself on the pirate bay, 5$ for the whole full freaking download in FLAC, MP3 or Apple lossless or 10$ for non download version. If you have more money you can buy the 75$ CD bundle that comes with the artwork, a DVD with the multitracks and a blueray disc in high-def stereo. And then there is the obligatory special edition for 300$ that includes four vinyls.

To quote the README in the torrent:

Nine Inch Nails: Ghosts I (2008)

This torrent is an official upload from Nine Inch Nails.

We’re very proud to present a new collection of instrumental music, Ghosts I-IV.
Almost two hours of music recorded over an intense ten week period last fall, Ghosts I-IV sprawls Nine Inch Nails across a variety of new terrain.

Now that we’re no longer constrained by a record label, we’ve decided to personally upload Ghosts I, the first of the four volumes, to various torrent sites, because we believe BitTorrent is a revolutionary digital distribution method, and we believe in finding ways to utilize new technologies instead of fighting them.

We encourage you to share the music of Ghosts I with your friends, post it on your website, play it on your podcast, use it for video projects, etc. It’s licensed for all non-commercial use under Creative Commons.

We’ve also made a 40 page PDF book to accompany the album. If you’d like to download it for free, visit http://ghosts.nin.com/main/pdf

Ghosts I is the first part of the 36 track collection Ghosts I-IV. Undoubtedly you’ll be able to find the complete collection on the same torrent network you found this file, but if you’re interested in the release, we encourage you to check it out at ghosts.nin.com, where the complete Ghosts I-IV is available directly from us in a variety of DRM-free digital formats, including FLAC lossless, for only $5. You can also order it on CD, or as a deluxe package with multitrack audio files, high definition audio on Blu-ray disc, and a large hard-bound book.

We genuinely appreciate your support, and hope you enjoy the new music. Thanks for listening.

http://ghosts.nin.com