Friday, February 11, 2011

The latest Microsoft outrage

I am not a big fan of Microsoft. But I had been hearing relatively good things about Windows 7. I don't have a lot of use for Windows myself, but I do have a need on occasion to support others, so I figured it would be a good idea to upgrade my Fusion VM from XP to Windows 7, if for no other reason than to get some experience and learn where all the knobs moved.

So after checking the box very, very carefully to insure it was, I bought the box that included the Windows 7 Home Premium upgrade.

It is absolutely legitimate to upgrade from XP to 7.

While I was at it, I wanted to switch from 32 bit to 64 bit mode. The only way to do that is with a clean install.

Well, that's fine. When my Dad upgraded to Vista, his disk was unbootable, so we did a clean installation of Vista and called up the activation center and they engaged a special workaround that finessed it.

So a few days in, now, and Windows pops up the activation dialog. I put in the key in the box, and it complains that it's an upgrade key. So I call up Microsoft and they tell me to pound sand - that the only recourse is to reinstall XP and then install Windows 7 on top of that.

So, in short, once again, Microsoft takes something really, really simple, and makes it impossible, in a way that benefits absolutely nobody.

Sunday, February 6, 2011

ZFS and Cyrus

ZFS allows you to take snapshots of live filesystems, which is a great way to solve the 'oops, I deleted the wrong file' backup problem. 'zfs send' for snapshots allows you to effectively deal with the disaster recovery problem. Snapshots are smart in that they only make copies of new or modified files.

Where this breaks down, unfortunately, is things like databases, where relatively large files wind up being treated as modified and copied into each snapshot. It's much better to use the database's own tools to generate backups, which tend to be much smaller. They also are safer, as a snapshot of a running database may wind up not being cleanly recoverable (obviously they're supposed to be, if the database is fully ACID compliant, but there is always a difference between theory and practice).

Fortunately, zfs makes it cheap and easy to create separate filesystems for data that has different needs.

Cyrus IMAP, unfortunately, by default mixes two different flows of data together - each mailbox directory has one message per file, plus a handfull of opaque database files that are always changing as the content of the mailbox changes. What's more, these database files are entirely reconstructible, so backing them up is unnecessary.

The problem is that you want to snapshot the filesystem where the partition is, but the snapshot will also backup these index files, which is a pointless waste of space.

Fortunately, Cyrus 2.4 has added the ability to separate the meta-data from the partition. This way you can create snapshots of just the mail itself. The snapshots will wind up being much smaller, as a single message will only be present on the disk once (because with cyrus, once a message is written to disk, it's not touched after that).

Before I began, the imapd.conf file had in it:

partition-default: /home/imap-spool

I added to that

metapartition-default: /home/imap-meta

metapartition_files: header index cache expunge squat lock

I created a new zfs filesystem for /home/imap-meta, and chown'd and chmod'd it to match imap-spool. I then shut down the cyrus system.

At this point, there are two choices to migrate. I chose the safer path, which was to simply run 'reconstruct' and then 'find imap-spool -name cyrus\* -delete'. Unfortunately, this resulted in all of the messages being marked as unseen.

The other possible choice would have been to replicate the directory structure under imap-spool to imap-meta, and then move all of the files that don't match the pattern [0-9]*\. from imap-spool to imap-meta.

With either of these paths taken, you should be able to restart Cyrus and see that everything is basically unchanged and still works.

But having done this, you don't have to set up imap-meta for snapshots or backup.