Internationalized PyGTK Applications

written on Sun, 10 Jun 2007 10:22

Python is an awesome programming language, GTK is an incredible GUI toolkit. Python ships gettext, GTK supports gettext in the glade interface builder. So the logical conclusion is using gettext in your PyGTK application. This article will show you how to create and translate your PyGTK application in no time.

Prerequisites

Things you have to have installed on your computer in order to make the following examples work correctly:

Motivation

Although I have en_GB.UTF-8 as my locale and publish articles on this webpage in English my mother-tongue is German. As a result of that I love applications that are available in more than one language. And the OpenSource community has plenty of applications, libraries and online platforms for creating multilanguage GUI and web applications. When I tried to translate one of my applications some days ago I was surprised that there were no tutorials (or maybe I'm too stupid for googleing) that explained the issue. I then checked other projects and read some man pages.

So I hope that this article will make life easier for you.

Organizing the Application

The first (and hardest) part of any application is the organization. It's that hard because the organization should support development and production mode. Having to symlink or copy files into global folders makes things a lot harder and is usually a stupid idea. For my application I decided to not put the translation files into the locale folder but the application shared data directory.

My idea (or better, the idea of Leonard Ritter, who did a similar thing for his aldrin project) was that you have the whole folder structure in your development folder and just copy the files to prefix (usually /usr) when installing the application.

So basically the folder structure looks like this: 

bin/
  APPLICATION             the executable that starts the application
share/
  APPLICATION/
    glade/                the folder that contains all the glade files
    lib/
      APPLICATION/        the python package with the application data
        __init__.py       make it a package and add bootstrapping code.
        application.py    the actual application code.
        ...               other files.
      ...                 dependencies you maybe want to bundle
    i18n/                 contains all the compiled translations
    res/                  other resources (images etc)

The executable (bin/APPLICATION) now looks at it's own filename, and looks up the parent folder of the folder the executable is located in. So for the development mode this will be ., for production mode it will probably be something like /usr, /usr/local etc.

Then it calculates the absolute path to the lib folder and adds it to the pythonpath to import the main function from the package and call it: 

#!/usr/bin/env python
import sys
from os.path import dirname, join, pardir
sys.path.insert(0, join(dirname(__file__), pardir, 'share',
                        'APPLICATION', 'lib'))
from APPLICATION import main
main()

The main() function comes directly from the __init__.py file of the APPLICATION package located in the lib folder.

Bootstrapping

Now it comes to the second hard part. Before we can use all the libraries we have to do some bootstrapping to get the gettext system working in the python interpreter and the GTK system. This happens in the __init__.py file of the package. There we set up gettext for the whole python interpreter, get the path to the shared files and import the main function from the application submodule: 

from os.path import abspath, dirname, join, pardir

# here we define the path constants so that other modules can use it.
# this allows us to get access to the shared files without having to
# know the actual location, we just use the location of the current
# file and use paths relative to that.
SHARED_FILES = abspath(join(dirname(__file__), pardir, pardir))
LOCALE_PATH = join(SHARED_FILES, 'i18n')
RESOURCE_PATH = join(SHARED_FILES, 'res')

# the name of the gettext domain. because we have our translation files
# not in a global folder this doesn't really matter, setting it to the
# application name is a good idea tough.
GETTEXT_DOMAIN = 'APPLICATION'

# setup PyGTK by requiring GTK2
import pygtk
pygtk.require('2.0')

# set up the gettext system and locales
from gtk import glade
import gettext
import locale

locale.setlocale(locale.LC_ALL, '')
for module in glade, gettext:
    module.bindtextdomain(GETTEXT_DOMAIN, LOCALE_PATH)
    module.textdomain(GETTEXT_DOMAIN)

# register the gettext function for the whole interpreter as "_"
import __builtin__
__builtin__._ = gettext.gettext

# import the main function
from APPLICATION.application import main

Alright. The code above should be straightforward, what you probably not know is the __builtin__ module. Basically if python cannot find a name in the locale or global namespace (and also not in a closure) it will look for it in the __builtin__ module before giving up. This is the module where the stuff like float/dir etc. comes from.

By binding the gettext function to __builtin__._ we can use _('Hello World') to mark as translatable and translatate a string from every module in the python interpreter without having to import the function first.

Writing Internationalizable Code

i18n of Linux works pretty well because it's something gettext solves in a very good way. The only problem you have is that there are usually situations where something looks like it would work for your language but certainly not in another language. This usually affects pluralization. The rule "n != 1" for plural forms is something you have in English and German and some other languages but not in Slavic and also not in Asian languages.

Also it's possible that you have two strings that look exactly the same but mean different things in the source language. In that case you have to "annotate" them by adding additional information to it to have two differnent strings. In that case you will have to introduce an translation file for your source language too and cut of the annotated stuff. An example for that is "May" and "May" in Calendars. One of them is the three letter version, the other is the full version. Unfortunately both look the same in English because too short. Solution:

_('May::abbreviated') and _('May::full')

I won't cover plural forms here because that would make this article too long but if you have the situation that you must distinquish between singular and plural have a look at the ngettext function provided.

So what you have to do now is wrapping all translatable stuff in _() as explained here: 

# this:
foo = 'Hello World!'
# becomes:
foo = _('Hello World!')

# and this:
foo = 'Hello %s!' % bar
# becomes:
foo = _('Hello %s!') % bar

# because placeholders could move in other languages, this:
foo = 'Expected %d items, got %d.' % (expected, found)
# becomes:
foo = _('Expected %(expected)d items, got %(found)s.') % {
    'expected':   expected,
    'found':      found
}

Creating Internationalizable GTK GUIs using Glade

Alright. That's easy. It happens out of the box. What you can do additionally is leaving comments for translators or marking strings as untranslatable in the property editor but otherwise all labels etc are marked as translatable automatically.

Collecting Strings

Once the application is created we have to collect all the translatable strings and create so-called "pot" files (po templates). A translator can then use that pot file to translate the application and send it back with the translated strings in. A pot file with translations is called a "po" file.

For python there is a cool called pygettext in the distribution that helps collecting translatable strings and creating a pot file. For glade there should be a configuration to generate pot files too but I was unable to find that in my Glade version (3.0.2). But even if there was such a setting or button it would still generate two pot files which are more complex to merge (requires some POTFILES.in foobar and tools with manpages longer than I want to read) I wrote a small tool myself especially for PyGTK which scans all python and glade files for translatable strings.

You can find this tool here: generate_pot.py. Usage is quite simple: 

$ python generate_pot.py share/APPLICATION APPLICATION VERSION > APPLICATION.pot

Replace APPLICATION and VERSION with your application name of course.

Translating

So the process of translating an pot file is straigtforward. If it's a new translation (so there is now it.po or de.po) by now you can simply copy the pot file to it's target name and open it with your favorite translator application or text editor. The format is straightfoward, it's easily possible to translate it with vim or whatever editor you use. the msgid part is the source string, the msgstr part is the translated string.

If you already have an translated po file and you added or modified translatable strings in the application (and an updated pot file generated by generate_pot.py) you can use the msgmerge utility that comes with the gettext system: 

$ msgmerge -U de.po APPLICATION.pot

If you changed the casing of an source string or did another minor modification msgmerge resolves that and marks the string as fuzzy. Just have a look at the comment above the string.

Compiling Translations

Now you have to compile a mo file gettext can use. It's basically a binary version of the po file with the translated strings in. For that you can use the msgfmt application: 

$ msgfmt do.po -o share/APPLICATION/i18n/de/LC_MESSAGES/GETTEXT_DOMAIN.mo

As you can see you need a quite complicated folder structure for gettext so make sure that de/LC_MESSAGES exists in the i18n folder. and replace GETTEXT_DOMAIN with the domain you defined in the package's __init__.py.

Testing Translations

If you have a system with a different default locale and you don't want to logout and login with another locale just use the LANG variable to tell gettext to use another language: 

$ LANG=de_DE.UTF-8 ./bin/APPLICATION

Note that this requires that you have the German locales installed on your system because otherwise the setlocale call will raise an error.

Deployment

So now where the application is ready to distribute all you have to do is to create a Makefile (or what ever you want to use) and copy bin and share to the specified prefix.

Get the Code, Don't get Lost

I uploaded a small example application that uses the foundation explained above. You can download it here: pygtki18nexample.tar.gz

It contains a small application that just displays a window and connects some basic signals.

view page source