Django MultiDB to the rescue!

Oct 12, 2012

#django#python

Warning: This post was written 4453 days ago, and may contain incorrect information, outdated opinions, and incorrect language due to my age at the time. Opinions in this post are not representative of who I am as a person today.

One of the key components of Auth is the ability to communicate with numerous other systems and manage their user authentication systems in situations were we can’t modify the application to use our authentication API. To achieve this we have the Services API, which is a generic interface designed for basic operations such as creating a user, disabling an account as so on. If we wanted to support a new system we write up a simple Python module with these functions and the API does all the required importing and abstraction out for Auth.

In previous versions of the API we handled databases in a very weird way, either by writing our own SQLAlchemy queries or boding the Django ORM to give us a basic cursor to work with, while it was far from perfect it allowed us to edit the databases of other applications without much hassle.

Recently, Django has updated into version 1.2 and with it came the MultiDB functionality which allows you to access multiple databases natively in the ORM. For our database layer this presents some new options not available to us in the old versions.

Using the Django database introspect you are able to generate a Model from a existing database schema, for this example we’re working with Mediawiki’s native database in MySQL. So first of all we need to define the database in our settings.

DATABASES = {
    'dreddit_wiki': {
        'NAME': 'wiki',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'wiki',
        'PASSWORD': 'passwordgoeshere',
    }
}

Next we fire off the inspect command to produce our database layout

$ ./manage.py inspectdb --database=wiki > wikimodels.py

After a short time you’ll have a fresh Python module with all your database models nearly ready to go, first thing to do would be to edit this file and change any foreign keys to the required Django ForeignKey() field. While inspect does as much as it can it can’t detect foreign keys.

Once your read to rock its a simple case of getting your shell out and giving it a test run.

$ ./manage.py shell
>>> from wiki.wikimodels import User
>>> User.objects.using('wiki').get(user_id=1).user_name
'Matalok'

Simple! No more hassling with db cursors, just simple ORM access without the hassle. The next big leap is defining the database connections at runtime, injecting into the DATABASES variable, by doing this I can remove the problem of having to manage the services’s database connection in the settings.py and instead have them defined on a per service basis.