Intro: This is part one of an eight part series looking at the Elevator Explorer, a fun data interactive mostly coded between the hours of 10 PM to 2 AM during the week leading up to April Fools’ Day, 2013. I’m going to be looking at the things I learned, things I wish I could have done, and the reasoning behind my design choices. The code I’ll be referring to will be in this tagged release on github.

Rapid models.py development

Introduction

Every minute doing boilerplate at the beginning of a project is a minute you’re not developing. And the beginning of a project is when you really need that momentum to code ideas, not fiddle with settings.  This post goes into how I do my database boilerplate.

It is very common to use sqlite in early Django development because it’s so easy to migrate your database after editing your models. All you have to do is delete your old database file and run syncdb again. I even made a generic make command to find and delete all .sqlite files then run syncdb so I could reuse the same make resetdb command in every project. But what if you want to use another database like PostgreSQL? You could port to using dropdb/createdb, but you may find it annoying to set up for every new project.  Luckily, there is a generic way thanks to django-extensions and DJ-Database-URL, both of which I include on every project. In my settings.py, I still use sqlite by default with:

DATABASES = {'default':dj_database_url.config(default='sqlite:///' +
    project_dir('example_project.sqlite'))}

but in my virtualenv’s postactivate (or in your .env file), I have:

export DATABASE_URL='postgres:///tx_elevators'

so it actually uses PostgreSQL. And by using django-extensions’s reset_db management command, the modifications to the makefile actually end up making things simpler:

# old version:
#   $(foreach db, $(wildcard $(PROJECT)/*.sqlite),\
#      rm $(db);)
#
# new version:
resetdb:
    python $(PROJECT)/manage.py reset_db --router=default --noinput
    python $(PROJECT)/manage.py syncdb --noinput

https://github.com/texastribune/tx_elevators/blob/2013-april-fools/Makefile#L22-L24

Just remember to put django-extensions in your installed apps. And make sure you’re using a recent version of django-extensions (>= 1.1.0), because there was a bug in reset_db prior to then.

Now after every model change, you can still use make resetdb to reset the db, no matter what database engine you use. Well… as long as that database engine is either sqlite, MySQL, or PostgreSQL.

Why not South?

South is a pain. Even if you script away the repetitiveness, it is an overly-complicated, blunt instrument designed for established projects, not for rapidly developing pre-alpha projects. And wiping away the database is a good thing. It means you can’t accumulate baggage in your data that you’ll never be able to recreate. Another benefit is that bootstrapping development on a new machine is a breeze because you’ve make that process simple and repeatable. And if you do need to do a quick migration, my first choice is django-extensions’s sqldiff command. For example, let’s say I wanted to make the year_installed date null-able to indicate bad data, and to make Building.city a foreign key to a new City model. If I changed my original models.py to do that, the output of ./manage.py sqldiff tx_elevators becomes:

BEGIN;
-- Application: tx_elevators
-- Model: City
-- Table missing: tx_elevators_city
-- Model: Building
ALTER TABLE "tx_elevators_building"
    DROP COLUMN "city";
ALTER TABLE "tx_elevators_building"
    ADD "city_id" integer;
CREATE INDEX "tx_elevators_building_city_id_idx"
    ON "tx_elevators_building" ("city_id");
ALTER TABLE "tx_elevators_building"
    ALTER "city" TYPE integer;
ALTER TABLE "tx_elevators_building"
    ALTER COLUMN "city" SET NOT NULL;
-- Model: Elevator
ALTER TABLE "tx_elevators_elevator"
    ALTER COLUMN "year_installed" DROP NOT NULL;
COMMIT;

Which I can pipe into the database. You do have to know some SQL, because the SQL it produces is not always right, but it does get you 95% of the way there.

Afterwards

Once you do a release, you should abandon this and switch to using South. You can still use make reset_db and blow everything away, but you should at least be providing migrations once your project is stable.

Next time…

I’ll go over how I made importing data a one-liner.