Monitor Progress When Using DD in Linux ~ Bauer-Power Media

Aug 16, 2011

Monitor Progress When Using DD in Linux

1:00 AM

El DiPablo

For those following me on Twitter and Facebook, you first know that I built a redundant SAN using Openfiler. Well, last week that son of a bitch gave me kernel panics 5 times in a row. The only thing I could do was reboot it. Not sure why, it wasn't under that heavy of a load. It had about 8 VMs running on it, and some backups. Not even 1TB of space was being used, but out of nowhere it would choke and die on me. I spent all night last Thursday until about 4:30am baby sitting it while our dev guys did a deployment. Well, that's all I can take with Openfiler.

I was talking with a former boss of mine, and the owner of a very successful managed services/consulting firm here in San Diego. He was telling me that at his colo facility, he hosts his clients on home grown iSCSI SANS himself. The difference? They run on Gentoo Linux using iSCSI Enterprise Target. Not only that, in his very own words, it works "flawlessly". Sign me up for that! Well, all except the Gentoo part. I mean come on, who are we kidding here? I'm not a sadomasochist. Nope. Ubuntu is my distro of choice for this.

Anyway, it turns out you can configure a similar failover cluster as my original Openfiler setup using Ubuntu, iSCSI Enterprise Target, DRBD, and Heartbeat. I figured, why the hell not? I just need to do something about the existing data on my running Openfiler.

What I did was take the offline Openfiler node down, and installed Ubuntu Server edition, and the above programs. I then created a 5TB LUN on my new Ubuntu SAN, then began to do a DD copy over SSH from my old online Openfiler LUN to my Ubuntu LUN. Nice and all, but if you've ever used DD it's not really good at letting you know it's progress. There is a cool trick to work around that though. Here's what you do:

Open an SSH session on the server you are copying to.
Open an SSH session on the server you are copying from.
On the server you are copying from, run your DD command. This is what I ran:
dd if=/dev/data/LUN01 | ssh root@100.100.10.12 dd of=/dev/VG1/LUN01
On the server you are copying to run the following to get the PID number of the DD process:
pgrep -l '^dd$'
On the server you are copying to run the following (Change the below number to match the number you got from the above command):
watch -n 10 kill -USR1 8789
Now on server you are copying from, you should see a status update on the DD copy every 10 seconds!