Liquibase

Note to self: I think this (a) needs an outline and (b) wants to become a “how +to automate db upgrades for dummies” page. Also, this is really old (~2008) +and many things have changed: database migration tools are more +widely-available and mature now. On the other hand, I still see a lot of +questions on IRC that are based on not even knowing these tools exist.

Successful software projects are characterized by extensive automation and +supporting tools. For source code, we have version control tools that support +tracking and reviewing changes, marking particular states for release, and +automating builds. For databases, the situation is rather less advanced in a +lot of places: outside of Rails, which has some rather nice +migration +support, and evolutions or +South for Django, there are few tools that +actually track changes to the database or to the model in a reproducible way.

While I was exploring the problem by writing some scripts for my own projects, +I came to a few conclusions. You need to keep a receipt for the changes a +database has been exposed to in the database itself so that the database can +be reproduced later. You only need scripts to go forward from older versions +to newer versions. Finally, you need to view DDL statements as a degenerate +form of diff, between two database states, that's not combinable the way +textual diff is except by concatenation.

Someone on IRC mentioned Liquibase and +migrate4j to me. Since I was already in +the middle of writing a second version of my own scripts to handle the issues +I found writing the first version, I stopped and compared notes.

Liquibase is essentially the tool I was trying to write, only with two years +of relatively talented developer time poured into it rather than six weeks.

Liquibase operates off of a version table it maintains in the database itself, +which tracks what changes have been applied to the database, and off of a +configuration file listing all of the database changes. Applying new changes +to a database is straightforward: by default, it goes through the file and +applies all the changes that are in the file that are not already in the +database, in order. This ensures that incremental changes during development +are reproduced in exactly the same way during deployment, something lots of +model-to-database migration tools have a problem with.

The developers designed the configuraton file around some of the ideas from +Refactoring +Databases, +and provided an extensive list of canned +changes +as primitives in the database change scripts. However, it's also possible to +insert raw SQL commands (either DDL, or DML queries like SELECTs and +INSERTs) at any point in the change sequence if some change to the database +can't be accomplished with its set of refactorings. For truly hairy databases, +you can use either a Java class implementing your change logic or a shell +script alongside the configuration file.

The tools for applying database changes to databases are similarly flexible: +out of the box, liquibase can be embedded in a fairly wide range of Java +applications using servlet context listeners, a Spring adapter, or a Grails +adapter; it can also be run from an ant or maven build, or as a standalone +tool.

My biggest complaint is that liquibase is heavily Java-centric; while the +developers are planning .Net support, it'd be nice to use it for Python apps +as well. Triggering liquibase upgrades from anything other than a Java program +involves either shelling out to the java command or creating a JVM and +writing native glue to control the upgrade process, which are both pretty +painful. I'm also less than impressed with the javadoc documentation; while +the manual is excellent, the javadocs are fairly incomplete, making it hard to +write customized integrations.

The liquibase developers deserve a lot of credit for solving a hard problem +very cleanly.

+ + + +

+ + +comments powered by Disqus +

+ + + + + +