Are Database Migration Frameworks Worth the Effort?

I’ve always used SQL scripts to manage database creation and migration. At my current company we have a database project with patch scripts that conditionally execute based on the database version. These scripts are simply run in sequence by an installer which stores the current database version as a database extended property. The approach relies on version based naming of SQL scripts and developers to remember to update the patch script whenever they make a structure or stored procedure change. Of course one could try to use a tool such as Redgate SQL compare to generate diff patch scripts but that won’t always suffice as that won’t capture refactoring such as column name changes or data migration (for more read The Problem with Database Diffs).

The use of SQL scripts is not without it’s problems. The most annoying of these is that a patch script may break as SQL will parse and validate your script against the existing table structure even if the statements are not going to run due to version check condition. This forces one to use nasty EXEC/sp_execute alter/create statements with double escaped strings (Yuck!). I find the following particularly problematic with this approach:

1. The string based SQL statements will always pass validation even though when they are executed they may be completely screwed
2. It is impractical to generate these by hand and the best way to create them is to use SQL Management Studio.
3. If your stored procedures are not wrapped in strings, to update the patch script requires you to script out the scripts from the database. Basically this increases development friction and discourages updating of the patch script.
4. It becomes difficult to determine when a structural change was made as one has to trawl through huge SQL scripts and it is not easy to use a regex search to find a change.

Hence I long for something a little more strongly typed, which is why the growing number of .Net based Database Migration frameworks have caught my eye. I want to be sure that there isn’t something obvious that I was missing out by not using one. Fortunately Ben Scheirman has done a fair amount of research already and wrote a fairly good round up of available .Net migration tools. Looking at the tools there are clearly two different types of frameworks available:

1. Structured SQL script execution– These tools provide a framework under which SQL scripts will be executed to create and upgrade a database to a specific version. Examples of these are: Tarantino and LiquiBase.
2. Rails like Active Record Migration– Commands to upgrade or downgrade the database schema and data can be coded in a .Net language, with the possibility of executing SQL statements if required. Tools that fall into this category are: RikMigrations, SubSonic Migrations, Machine.Migrations and Migrator.Net.

The obvious question that springs to mind when looking at these frameworks is whether or not the additional development friction (additional tasks that must be performed during the course of development) is worth the benefit. To be honest I don’t see a massive difference between the structured SQL script execution approach and what I’ve been doing all along. However the likes of LiquiBase do provide a functionality that nicely captures database refactorings and diffs, so maybe it could be used as more of a general toolset. However the Active Migration approach would be a more radical departure so I’d like to explore that more.

Here are my thoughts on Active Record Migration like frameworks:

1. Best paired with an ORM framework, such as NHibernate or Subsonic, where one is not writing a lot of custom SQL and hence the the upgrade classes can be kept clean and focus on structural and data migration. That is not to say that there will not be benefit to having them execute a lot of SQL statements to add/update stored procedures, it is just that it may get a little messy and possibly tedious.
2. Using C# code to do database change would provide an opportunity to use something like log4net to dump out progress information. I've always longed for a decent way of doing this that doesn't involve a custom proc that inserts into a table. I really like the idea of having a degree of compile time checking for the database changes. Of course with SQL scripts you could always run them on each build and if they break, you fail the build.
3. The whole idea of downgrading a database scares me because if someone accidently points an old version of an application (with automatic migration logic) to the database (think xcopy deployment) you could end up losing tables and data. Of course one should always backup a database before doing a deployment, but I’m talking about a situation where one doesn’t know it is pointing to the wrong database until it is too late. I hope these tools backup the database before applying changes. In my mind a downgrade should only be performed if a rollout fails and one has to revert to a backup taken before the upgrade was undertaken.
4. I really like the idea of using anonymous objects to insert data into the database (RikiMigration supports this) and avoiding the need to write insert statements.
5. Do you not run into potential development concurrency and structuring issues? So for example during a development cycle does one dump all changes into one class or into several?
6. The database vendor abstraction that is can provide is only attractive to a small number of projects. Furthermore one invariably will need to execute vendor specific statements and is best suited to a multi-vendor ORM such as NHibernate.
7. The approach may not fly in an organization where a DBA must review all SQL changes. Although I don’t see any reason why the frameworks couldn’t generate these for you.

I think if I was currently working on a project that used an ORM like NHibernate or LINQ to Entities, the migration frameworks would be a lot more appealing. However with other types of projects I’m worried that the additional effort isn’t really going to be worth the benefit.

Am I missing any other potential benefits (or problems)? I would really like to hear if there are other compelling reasons to adopt the approach.