I've moved my blog to jmcneil.net. This is no longer being updated!

Friday, April 18, 2008

Choosing a framework with API support in mind?

For the last couple of years or so, I've maintained a system for managing and manipulating various web hosting accounts. The code consists of a few Python modules and some supporting C extensions where necessary.

Each module contains a series of XMLRPC calls that are made available to remote clients. Using these XMLRPC calls, it's possible to "do stuff" to a standard account. Accounts can be created, deleted, removed, and so on. It's a fairly powerful API in that it's possible to tickle just about every attribute associated with a user's account.

The application relies on a few thousand lines of boilerplate 'infrastructural code.' By infrastructural code, I generally mean things like logging, daemonization, startup and shutdown, and wrappers around Python's standard SimpleXMLRPCServer modules. They're actually subclassed in order to add support for some nifty little features, but that's not all that important.

We run multiple instances for each core module, each on a different port. For example, web might run on 8181, SQL on 8282, and ecommerce on 8383. The whole thing is fronted by Apache which provides URL mapping, authentication, and SSL capabilities.

I've really been hoping to get rid of all of that custom code. It's a headache. I don't want to manage PID files, session management, or SQL connection pooling manually any longer. Our group can be much more productive if we can simply focus on our API development rather than bugs in our frameworks.

I've spent a great deal of time researching the various Python application servers out there. I think I've narrowed it down to two: Zope 3 or Django.

With the exception of Zope 3, the problem I have with these frameworks in general is that they're clearly intended for web *site* development and not necessarily web-aware *API* development.

From a holistic point of view, I like Django. A lot. It's clean, pragmatic, and gaining a lot of popularity. I like the template system (I've seen a lot of horrible PHP). The documentation is outstanding. It took me almost no time at all to setup a basic site, grind out some extremely ugly templates, and wire everything up to a little MySQL database. That's about where it ends, though. We need to support XMLRPC calls. We don't need a UI. We don't even need a database for the most part.

Zope 3 on the other hand, looks to be exactly what we need. It's a very extensive framework. It's built on eggs, so we only need to install what we need. We can simply create XMLRPC/SOAP views and tie directly into our existing Python modules. Sounds great.

The documentation is god awful. I want to use it. I know it's very powerful. The Zope crew has just made the barrier to entry very difficult to overcome. It's not difficult to grasp once I find some type of authoritative documentation. It's finding that documentation that is the problem.

I'm going to learn it. It's clearly one of the most powerful utilities in the Pythonic tool chest.

All of that said, what is the preferred framework for API-driven development? Am I barking up the wrong tree with Zope and Django? Does it make sense to leave it as is? I don't need templates, URL routes, AJAX, or browser compatibility hacks. We're developing a large scale remote API, not a web site.

Lastly, I did look into using Twisted, but I'm not sure our current code base fits into the async. model very well. It would simply be an XMLRPC controller (for each module) firing off our current methods using threaded deferreds.


No comments: