Making Rails and Django Play Nice with the Same Data

Monday, March 12, 2012, at 04:49AM

By Eric Richardson

Old and new versions of the KPCC homepage, served out of Django (left) and Ruby on Rails (right).

Friday afternoon, KPCC opened up a public beta for our new web site, which finally gives me license to talk about some of what I've been working on over the past few months.

Accept the cookie on that beta page and you'll get a fresh new view of the KPCC site. In the process, you'll get handed off from our old Django site to a new Ruby on Rails application. Share a link with a friend? If they're not opted-in to the beta, they'll see the old site at the same URL.

That back-and-forth happens via some nifty conditional logic in nginx, a handful of carefully-applied MySQL views and custom session and caching backends that allow us to pass data and trigger actions between Django and Rails.

I'll explore those different pieces over a handful of posts, starting today with the MySQL views that enable polymorphic association and generic relations to play nice.

The Goal

As we started into the redesign project last fall, we knew that we wanted to be able to roll it out slowly and section-by-section. That meant we were likely to end up needing to support side-by-side apps regardless of what they were written in. At the same time, we had been looking at the work Rails was doing in 3.1 with the asset pipeline, and felt that the framework's approach and trajectory was more in line with where we wanted to be going forward.

One Saturday morning, I was sitting in a coffee shop and put together an email proposing an idea: What if we built the redesign as a Rails application that would use all the same database tables—buying us more time with a backend built around Django's built in admin interface—and fronted it with a thin wrapper that would allow us to route traffic to one backend or the other based on both path and cookie status?

With that email, an interesting project was born.

Enabling Interchange

For a simple database table, the only difference between what Django writes and what Ruby on Rails writes is likely to be the name. They both like auto-incrementing integers as primary keys, they both build attributes based on key names, and neither does anything particularly tricky for foreign key names or values.

What is different, though, is the way they handle polymorphic relations (Generic Relations, in Django terminology). Django maps models based on integer values created by its Content Type system, while Rails prefers to use the model name as a string in an _type field.

For instance, to create a polymorphic relation called content in Django, you would add something like this to your model:

content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content = generic.GenericForeignKey('content_type', 'object_id')

Both content_type (actually content_type_id in the database, matching any other Django foreign key) and object_id are integers: the former containing the id of the ContentType mapping, the latter containing the id of the object on the related model.

Rails would do it a bit differently. In your model, you would just say:

belongs_to :content, :polymorphic => true

That would look for two fields in your database table: content_type (a string containing the model name) and content_id (the id of the object on the other model).

So how could both frameworks work on the same data? MySQL views came to the rescue.

Basically, a MySQL view is a stored query mapping data from one or more tables. For us, that meant that we could create a view mirroring the underlying table, but replacing the fields for the Django generic relation with those that Rails would expect. Throw in one new table mapping content type ids to Rails classes, and we were in business.

The rails_content_map table

CREATE TABLE `rails_content_map` (
  `id` int(11) NOT NULL,
  `class_name` varchar(255) NOT NULL
);

INSERT INTO rails_content_map (id,class_name) VALUES(15,'NewsStory');

The MySQL view

  CREATE OR REPLACE 
  SQL SECURITY INVOKER
  VIEW rails_assethost_contentasset AS 
  select 
    a.id,
    a.object_id as content_id,
    m.class_name as content_type,
    asset_order,asset_id,
    caption 
  from 
    assethost_contentasset as a, 
    rails_content_map as m 
  where a.content_type_id = m.id

While life gets a little more complicated if you need to be inserting data on both sides, it turns out that Rails is perfectly happy to use a MySQL view as its "table" for reading. Just use self.table_name = "my_new_view" in the model, and Rails will think it's dealing with a well-formatted table underneath.

This is only necessary for tables that implement polymorphics. Simpler models require no cleanup work.

Who Creates the Data?

In our use-case, the data is being created in Django. Rails is acting purely as a consumer. That made it easy to know what to put where: Django writes to the real table, Rails gets the view. A piece of the application that has that workflow reversed might instead get a true table for Rails and a view for Django, though that's not something that I've had to implement thus far.

Fun with Permissions

MySQL does turn out to have some funky ideas when it comes to permissions on views.

First, while the Rails database user only needs SELECT rights to query data, it also needs the CREATE VIEW and SHOW VIEW rights to be able to migrate the view into place and query its structure.

Secondly, if you just call CREATE VIEW without specifying a permissions model, MySQL requires that both the definer of the view and its invoker have the select rights needed in order for it to execute. That can be annoying if you're transferring database dumps from system to system. For our needs, just saying that the invoker needed the rights was plenty and got away from some annoying headaches.

Tomorrow I'll take a look at sharing session information between the two apps.