Since January of this year I've been engaged in building what I guess many would call an enterprise web application in Django. I use the term enterprise in the sense that the project I've been tasked with building is large in nature of importance to the organisation and also in parts quite complex. It's been a very useful and educational experience, one which I would gladly repeat in Django. I'm going to break this down into three posts I think, the first being about some of our toolchain and possible improvements on it, second on django apps we used and some measure of how good they are and finally some advice on how to run similar projects and some pitfalls we experienced.
Toolchain
We developed on linux desktop machines in a variety of emacs, vim & nano. I know a bit hardcore, but we did make use of various plugins to help us, including the excellent vimmate plugins. Each developer ran a local instance of the django app, using the default server and sqlite3 for speed. We used virtualenv and pip requirement files to manage any external dependencies and used fabric to automate sections of the app deployment. We also worked with the internal systems team to move all of the Linux depenencies to Puppet as well, which will eventually give us a completely automatic build. We're not there yet as we still need to get postgresql database creation online and stable within our fabric / puppet coupling.
We made use of existing subversion hosting internally, we used tagging for releases, keeping trunk always in a runnable, non-broken state. We have a suite of tests for the django apps we wrote, written both as unit tests and functional tests where appropriate. We haven't got any in the browser tests using say selenium, but I'll cover that in the final post.
We used postgresql which for our domain is great, we don't do anything particularly hard with it (like GIS, real-time updates, etc) so we haven't had to tune it nor look at connection pooling for example. I've broken out our fabric script to be able to be able to setup our database on any host so that as the system grows the sysadmin team can migrate to a dedicated db server.
Aside from commenting code, we've used sphinx to create both enduser documentation and internal developer guides as we wanted to document our api and some other aspects of the system in a way that will be easy to pick up. For end-user documentation we took a demand led approach starting with cheatsheets and quickstart guides and fleshing out where questions come up.
Issues
Developing on Linux when you can't customise your working environment is a pain. A lesson I should of learnt early and acted on, was to lock down our target deployment environment (currently Fedora 12, and then RHEL 6 when it comes out of beta) so we could have set up a continous integration system such as Hudson. We have a test suite that is regularly run but could do with being automated. The ability and the need to be able to build a new instance of the system to test say a sandbox system integration cropped up repeatedly, slowing us down as we have to request a new VM to be built. What I'd do in future is maybe get a machine built using something like Ubuntu private cloud, that we could have had control over so as to remove a dependency on sysadmin support.
From a project management point of view we used basecamp with our UX designers, Trac with internal developers & RT with IT support desk. I'd say that basecamp worked well, but some way of digitally commenting on wireframes would have been really handy. They were produced in Axure which did a great job of automating documentation creation for the front-end developers.
Lessons learnt
- Staging environments that match production are a very good thing.
- Take the pain early, it'll save you over the development cycle.
- It's amazing what you can achieve just using vim
Next time: django apps - conventions, finding them & measuring quality.
