1:00 AM
El DiPablo
I have spent the better part of the last few months testing out some homegrown SAN options. I have been writing about them here almost exclusively during this time. I originally started off with Openfiler 2.99 using DRBD and Corosync. That blew up in my face with some really sweet kernel panics. I thought it might be a problem with Openfiler, so I thought I would build my own SAN using Ubuntu and iSCSI Enterprise Target, and setup failover using DRBD and Heartbeat. That didn't kernel panic on me but was really slow, and completely unusable with VMware.
Since DRBD seems to be the gold standard for open source failover, I would say this pretty much means that an HA setup with a homegrown SAN is not possible at the time of this writing. I decided to abandon the HA setup with my homegrown SAN, and guess what? By itself, without DRBD, Ubuntu with iSCSI Enterprise Target really does work flawlessly. Even with SATA disks, in VMware, and with 5TB of storage allocated for backups from Microsoft Data Protection Manager I never see I/O Wait times greater than 1%. With DRBD I/O Wait was anywhere from 15% to 30% constantly. I think the proof is in the pudding.
Reading on forums, wikis and blogs about these issues, it seems that the only people experiencing similar issues are people who are trying to setup a failover cluster using DRBD. In almost all the documentation on configuring Openfiler, or iSCSI Enterprise Target for failover includes using DRBD and LVMs. It turns out that LVMs over DRBD just don't work all that great. In fact, I believe it was DRBD causing the kernel panics in my Openfiler setup.
If you are hoping to setup your own failover cluster storage server, you might want to wait a while. It's just not there yet. If you just need a lot of cheap storage, then any of the current technologies are pretty good. Openfiler, FreeNAS, and iSCSI Enterprise Target will suit you just fine. In fact, I would say that iSCSI Enterprise Target on Ubuntu works better than Openfiler and FreeNAS, and has less overhead. Especially if you follow these best practice suggestions: (IET VMware Best Practice)
Do you know of a better way to setup failover in Linux other than DRBD? Is it easy? Can you point us to some documentation? Let us know in the comments!
del.icio.us tags: drbd, ha, ubuntu, iscsi enterprise target, iet, openfiler, flaky, slow performance
