Django’s Problems and Why 2.0 is a Bad Idea
I stumbled about this thread on django-developers which proposes calling the Django 1.0 release Django 2.0. One the one hand version numbers say nothing. Just take Wine or Trac as two examples that are already very stable but still below the magical 1.0 release. Open Source software often takes some time until a 1.0 is released and that’s perfectly fine. However skipping a version number is purely a marketing trick IMO. Just think of Java which currently names 1.6 Java 6 whereas 1.4 still was Java 2.
With django it looks like the plan is to keep up with Rails which went to 2.0 a few days ago. While I love to see that django kicks ass and it’s moving toward a stable release I have a bad feeling naming it 2.0. Because there is currently a huge gap between rails and django unfortunately. Rails has gained really good integrated migrations, REST webservices, a debugger and many other things django is still lacking.
Django makes an incredible good framework if you get your problem into the use case of django. But as soon as you break out of it and need something that goes beyond what’s possible in django you wish you have chosen something else. The django ORM is far from optimal, the admin rocks but as soon as the number of users exceeds 10.000 users it’s impossible to use it (chose yourself in a dropdown of 50.000 users …) or becomes utterly complex. Complex data models also look awkward in the admin or become too complicated to manage. And if you want to stick with the admin you cannot replace the user model. Now what do you do if you have a forum and want to count the posts? Use a UserProfile module? And how do you want to display a list of users sorted by their number of posts?
Yes there are ways to hack around it but the more complex the application becomes the more you of django’s strengths become obsolete. The application I’m working on right now now is only using two more contrib modules. The auth and the admin, and it looks like we have to drop them too, due to the limitations. All applications in that project hack around ORM limitations, we have an incredible number of recreated base middlewares, we have to monkey patch the request object to hack in subdomain support.
I was talking with David Cramer from curse gaming about some of the issues and he told me that they have forked django at a given point and patched the ORM. The django template engine was replaced by Jinja (our application does the same) and they are caching the hell out of the application to scale it. Bryan McLemore from the curse team told me some time ago that some pages have up to 30 queries on a page.
I don’t want to say that django fails in what it’s doing. But it’s far from 2.0.
I usally say django is great for websites and webapps while rails is great for services and APIs.
The whole discussion with version number is just overkill, just get it released in stead.
Also, webdevelopment is evolving each day. Something new and great today could be deprecated tomorrow, verison number doesnt say a shit.
So just use the svn revision number, tag it with a flashy name (like ubuntu do on their releases) and just get it out! Real webdevelopers will use the trunk anyway, because as i said, webdevelopment is evolving.
Comment by Eric — Wednesday, December 12th, 2007 @ 11:59 amAlso, rails is lacking great things from django aswell, as form handling with newsforms for example. I think i like the way django handles models better though it would of course be great with some kind of migrations support
Comment by Eric — Wednesday, December 12th, 2007 @ 12:08 pmMore than 10,000 admins on a site? Sounds like too many cooks in the kitchen to me. Seriously though, at that point you are definitely at the upper bound of the use case for the admin app, so it starts to sound silly to complain. At some point you need to code your own app.
Regarding ordering by number of posts: denormalization is your friend. It would take any competant coder 5-10 minutes to create some signals and callbacks to update a num_posts field on a UserProfile. Someday when aggregation support is included in the ORM, this will become even easier, but still, todays solition is pretty darn easy.
Beyond those two (non)issues, you really didn’t make any points. The fact that you were able to easily swap in Jinja is actually a testament to the modularity of Django itself.
Comment by Eric Florenzano — Wednesday, December 12th, 2007 @ 12:39 pmYes. newforms is django’s killerfeature. It’s so far the best form validation system I found. Also that the django core is translated is a very good thing. While our project is not multilingual it’s not English and it’s a cool that django can output German error messages.
Our problem is not the updating of the num_posts fields in the UserProfile, our problem is the displaying of a user list paginated and sorted by the number of posts. Yes we can (And do) sort the user profiles by the number of posts and resolve the associated user object but still the process is a lot more complex than we want it to be :-/
And no, we don’t have 10.000 admins, just about 50.000 users and as many user objects :-) So we also have 50.000 items in a dropdown.
Comment by Armin Ronacher — Wednesday, December 12th, 2007 @ 1:01 pmHaha, I see what you meant about the 50,000 users now. That makes it a much more valid point :) Still though, the admin app is modular enough that if you have 50,000 users, you should be able to write some Javascript to code in an autocomplete box or something.
Comment by Eric Florenzano — Wednesday, December 12th, 2007 @ 1:17 pmThe Admin is an app made with django. It is not Django, it is nice to have it if you need to make a website. But if you make an app then you better roll out your own Admin. I can’t understand why people think the admin is Django and it should do al sorts of things. The Admin is Django + a admin application.
If you have 10.000 users you write your own user view.
Comment by Onno — Wednesday, December 12th, 2007 @ 1:19 pmI can see why developers would argue that version numbers aren’t important, but developers aren’t the only ones to decide which technology is used. (sadly)
Comment by Daniel — Wednesday, December 12th, 2007 @ 1:19 pmSo some business decider with a project with huge stakes hears two camps of developers, one supporting a 0.xx version technology and the other supporting a 2.xx version.
The camp with the 0.xx version technology has got some explaining to do!
(instead of discussing what should be the issue: the features etc.)
I’m not sure what the point of jumping to calling it v2. If they are departing, or breaking compatibility with previous versions, etc. then that would make sense. Or signal a major change, but if it’s just to be a higher number for comparison reasons, that’s kinda confusing.
Django, RoR seems to be most about convenience, than anything else. IMO Combining disparate pieces of software together leads to the most interesting software. Although everything isn’t all in one place (from the developers point of view, documentation, dependencies, etc.) the diversity can be healthy and keep things agile flexible, and avoid becoming bloated.
Comment by Braydon Fuller — Wednesday, December 12th, 2007 @ 1:27 pmYou should be able to solve the 50,000 users problem using the “raw_id_admin=True” flag on the ForeignKey field - that replaces the “select an object from this select dropdown” widget with a text field that you can enter an integer ID in along with a link that launches a search interface to help you find the ID.
If that doesn’t solve your problem, this is a serious bug in Django’s admin: it’s meant to be able to scale up to large numbers of related objects.
Comment by Simon Willison — Wednesday, December 12th, 2007 @ 1:29 pmHaving too many values in the drop down list is not a difficult problem to solve.
Our app (which used the automatic django admin interface to the fullest possible extent) had a purchase Order module in which the user keys in Item code. As you can imagine there are thousands of items in the item master table.
We got around this by using newforms and Javascript autocomplete.
Moving an app from automatic admin interface to newforms is pretty easy.
Comment by Pradeep Gowda — Wednesday, December 12th, 2007 @ 2:31 pmAn issue I bumped in to with Django was aggregate functions. There is not much support for SQL aggregate functions in Django.(http://code.djangoproject.com/ticket/3566) Rails wins here (count, avg etc) , by providing more aggregate functions and the ease with which you can use SQL directly (Whether its a good thing or not is another debate, but it helps getting things done).
Comment by neelesh — Wednesday, December 12th, 2007 @ 2:43 pmIt’s a funny thing though - there’s no group of developers that can avoid bikeshed discussion.
www.yellow.bikeshed.com
Comment by Konrad — Wednesday, December 12th, 2007 @ 4:27 pmI actually agree with some of your points about the Django admin interface not being optimal Wilson designed it, what, three years ago, and it hasn’t been touched since. It’s definitely in need of an update, and has been for some time.
But, how can you use problems with the totally-optional admin app as a way to say, “Django isn’t where Rails it at,” when Rails doesn’t include an admin application at all? If you don’t like the admin app, you can do exactly what you’d do in Rails: build your own. The Django admin is simply a tool available that you can use if it works for you. If it doesn’t, you’ll build one, just like you would in Rails.
Same goes for auth. If you don’t like it, don’t use it. But, you can’t say the optional auth app’s problems make Django worse than Rails, when Rails doesn’t include an auth app at all.
Right?
Comment by Jeff Croft — Wednesday, December 12th, 2007 @ 5:06 pmWoa. Quite a lot of response. Guess I have to straighten out some things.
Yes it’s true that django admin is not strictly part of the framework and yes, it’s an addon library. However very tightly integrated into the django system, especially the ORM. This is not a bad thing and there is huge potential. On the other hand it’s also true that having an administration application being part of the framework is a plus over rails.
But if we look at django the framework has four major elements: The object relational mapper, the form validation library and the WSGI/mod_python/HTTP modules and finally the template engine. The first element, the ORM is “crippled” compared to SQLAlchemy. One the one hand because there is no session system which becomes pain in the ass pretty soon if you see what the session system does in an SQLAlchemy powered application. However there are also strengths of the ORM, namely that it’s incredible user friendly. But zzzeek is working incredible hard on making SQLAlchemy easier and easier and recent versions are great fun to work with. And yes, you can of course use SQLAlchemy with django, but then you loose the admin and all the other great features that depend on the models.
The form validation and the admin system are certainly django’s biggest plus over alternative solutions. But as it turns out the admin becomes problematic for more complex data models and you have to create your own administration interface anyways.
Python has a huge number of great libraries, don’t forget about that. There is Genshi which, without doubt, is the so far best XML based template engine I found for an agile scripting language. There is SQLAlchemy which permanently blows my mind in what’s possible and how you can use the library. There is WSGI and libraries that built up on it (like Paste, Werkzeug, Webob…). There is a huge number of libraries you probably don’t know yet ;-) Those libraries combined together make (currently) a good scaling application if you know what you’re doing.
Django certainly leaves TurboGears out in the rain. It’s easy to integrate other libraries in django, even if others often claim that this is not the case. But you also lose all the strengths of django other frameworks don’t have (admin, auth). And if you lose all of them you are left with a (well performing) WSGI library plus a bunch of utilities you could have otherwise to.
So I really would say: Make that a 1.0 release and blow our minds with cool additions in the future (like migrations, an even better extensible administrator interface, row based permissions, subclass-able models) and revive the SQLAlchemy branch which would be the ultimate feature in django.
Regards,
Comment by Armin Ronacher — Wednesday, December 12th, 2007 @ 5:45 pmArmin
I think that one could just as easily make the case that Rails is immature compared to Django because of its (non-)handling of authentication, administration, Unicode and threading. Maybe Django should be at 3.0!
Comment by Paul — Wednesday, December 12th, 2007 @ 5:51 pmPaul: Rails is miles ahead of django so far. (Unfortunately). There are plugins that add auth/admin and other fancy stuff to it. And Rails does handle Unicode. And I wouldn’t say that django handles threads much better than rails. While Rails might not be threadsafe, Python has a GIL and performs better in a Forking Environment too most of the time (of course consuming more memory).
Comment by Armin Ronacher — Wednesday, December 12th, 2007 @ 6:04 pmI think you make some good points, but they should really be directed at any of the full stack “2.0″ web app frameworks. All ORM’s end up getting in the way at some point (IMHO), you always end up saying “this would be way more efficient in straight SQL.” I’ve been involved in projects that have addressed the issue in one of two ways, re-work the schema to fit the ORM or work around the ORM to get at raw sql. Both are good solutions depending on the situation. The real strength of Django, Rails, Turbogears, Pylons, and any other newish set of web tools is that they give you a set of usable defaults. However, there’s no way to anticipate every need, so they’ll all end up “failing” at some point. As long as they give you a reliable facility to fall back to some sort of “raw,” you’re in the clear.
So, given that limitation, my take on it is that the good frameworks are ones that perform well and are reliable. I’ve had to recommend Rails or Django in a few situations. Today, December 12th, 2007, I recommend using Django, as the base platform is easier to deploy, faster, and more reliable. The free stuff you get with it is a real bonus for the early stages of development. That said, I haven’t looked at Rails 2.0, or the latest versions of Pylons or Turbogears.
As for 1.0 or 2.0, I don’t give a crap.
Comment by Chris McAvoy — Wednesday, December 12th, 2007 @ 6:19 pmHi Armin, I couldn’t agree more with you. I’ve implemented Django and “suffered” the same things (mainly) with the ORM layer. On the other side, I think the template system is great, and the Jinja library is there to expand it.
http://jinja.pocoo.org/
Cheers!
Comment by Marcelo Fernández — Wednesday, December 12th, 2007 @ 6:55 pmMarcelo
Though I realize this is a Django discussion, I couldn’t resist trying a bit of Rails here. For those of you that never tried Rails, I’ve compiled a very very speed up “Rails 2.0 Screencast”:http://www.akitaonrails.com/2007/12/10/the-first-rails-2-0-screencast-english. In less than 30 min you can see some of the new features and what can be accomplished.
About Rails not having an Admin app, it can be an advantage or disadvantage depending on your point of view. If you’re mainly dealing with content management, this is a plus. Though mixing a Framework with a highly integrated App is arguably an optimal situation. I’d like to have Rails being the kitchen-sink and I can later add ActiveScaffold, for example, if I want very fast admin screens. Authentication can be done is several different ways and I also don’t think it belongs to the framework, as we have several plugins to tackle this matter as well.
That said, I also don’t think there only needs to be one framework, there is room for DJango, Rails, and many more.
Comment by AkitaOnRails — Wednesday, December 12th, 2007 @ 7:09 pm@AkitaOnRails: Python has more to offer than just Django. If one wants something like rails there is Pylons or a custom WSGI application which, thanks to SQLAlchemy, gives you a very good database access. Seriously. Replacing Django with Rails is like switching from one evil to the other. And even if Rails still leads the framework war, Django is doing great progress.
I just don’t think that it’s ready for 2.0 *yet* :-)
Regards,
Comment by Armin Ronacher — Wednesday, December 12th, 2007 @ 7:25 pmArmin
Rails is great and in my op is a littel more mature than django and other python frameworks. But the point is not rails vs django but rather python vs ruby. And as you all know, python is like poetry ; )
Comment by casseen — Wednesday, December 12th, 2007 @ 9:46 pm“But the point is not rails vs django but rather python vs ruby. And as you all know, python is like poetry ; )”
Ruby is cleaner (except for ‘end’, but python forces you to indent+’:’ so that is not a huge compensational win, since you cant choose to not have that) than python. The ambiguity cat.meow() is a joke, the variable mandatory “self” name in python is a joke too. (I am not saying python itself is a joke, python has some great things going for it, which you notice if you see that really many use python)
To be honest… i think both python and ruby lost the battle of the www to…
PHP
And this makes me sad. Php is a horrible language, but people use it.
Everyone talks about the www. Django and RoR are web FRAMEWORKs.
WEB! This is where PHP won, and both python and ruby should focus
in GOOD deal with the www. Www is all about interacting with other people.
We dont need a fancy erlang - we need software from people, for people. And
when i write people, I mean BILLIONS of people! Everyone is a user!
The www has won.
PHP won the first round, it is time to make it lose again!
Comment by herald — Thursday, December 13th, 2007 @ 1:12 am@AkitaOnRails: Django and Rails are both good web frameworks for the reason both of them are having a success. Since you’re a rails user. does rails has something like Django’s newforms? And I like what you can do to Django’s middleware very powerful :). The only room for improvement I think Django should do is for the templates. In the end I agree with casseen, it’s python vs ruby.
@Armin: Thanks for informing that SQLAlchemy is getting easier to use :)
Cheers
Comment by james — Thursday, December 13th, 2007 @ 4:51 amHave you tried monkey patching the User model itself, to add the number of user posts directly to User? It’s incredibly easy.
http://www.amitu.com/blog/2007/july/django-extending-user-model/
Comment by AndrewSK — Thursday, December 13th, 2007 @ 7:08 amGreat post. I think Django is great. I blog about it, I write about it, and I spend considerable time helping people learn it. That said I think it’s important to make a fair assessment about your strengths and weaknesses as a project, and constantly work to produce something that’s better.
I agree with you on the whole 2.0 version number thing. It’s nonsense.
Comment by Empty — Saturday, December 15th, 2007 @ 1:52 amThanks for your Post, AndrewSK. Your sollution is dirty but works :)
Comment by Benjamin Wiegand — Saturday, December 15th, 2007 @ 7:39 pm[…] Lucumr Cogitations » Blog Archive » Django’s Problems and Why 2.0 is a Bad Idea (tags: comparison django rails ruby webdev python orm curse fork development) […]
Comment by Extenuating Circumstances – links for 2008-01-07 — Monday, January 7th, 2008 @ 10:20 am