To get everyone in context I’ll say that I had some Xen VMs which used LVM volumes as disk images living in one machine and I suddenly needed them on another one. After searching for some tried and tested solution I found none. So the only option I could see was to do some kind of dump (probably dd involved with bzip2) and bring them over to the other machine. Be it by network, DVD or external disk. Haven’t done it before, and it sure would have taken some effort on my part. Downsides: dangerous precise movements required and nothing new learned. This was a no-op

Luckily an alternative came to my mind. I could use AoE [1] (ATA over Ethernet not Axis of Evil nor Age of Empires!!!) and some LVM wizardry. Upsides: dangerous precise movements required and cool stuff learned. This was the way to go!

Just for the sake of clarity some definitions and requirements first. All the stuff below was done on two Debian Etch 4.0 boxes. Both of them having Xen for virtualization and LVM2 for storage administration. You’ll need an AoE enabled kernel (Debian’s kernels are from 2.6.11+). You’ll also need the aoetools package in the destination machine at least. Let’s call ORIG and DEST the machines and VG = Volume Group, PV = Physical Volume and lv = Logical Volume. If that’s mumbojumbo to you then keep away from the next commands!

Now to the recipe:

Enable AoE server on ORIG

root@orig-host# vbladed 0 0 eth1 /dev/md1

That should export any block device you want over eth. In my case a whole RAID device (md1) was exported through eth1 with major 0 and minor 0. Consider that Ethernet protocol is layer 2 and is non-routable. Another thing is that AoE provides no security inherently, so I enabled it manually for this and disabled it afterwards. If you are going to use it otherwise make sure you design a security scheme to go with it.

Enable AoE support on DEST

root@dest-host# modprobe aoe

Then search for AoE devices and make sure it found some

root@dest-host# aoe-discover
root@dest-host# aoe-stat

That should output something like “e0.0 250.000GB eth0 up”. At this point you should have a new block device available on the client box named /dev/etherd/e0.0. You can do anything to that block device as if it was a locally attached device. Cool!

Now we have to disable the ORIG-VG and make it unknown to the ORIG LVM system

root@orig-host# vgchange -an ORIG-VG
root@orig-host# vgexport ORIG-VG

Tip from experience: it’s better to have a dedicated VG for storing VMs images so we don’t have any downtime in the real box. I had the root filesystem on the same VG and had to switch to runlevel 1 just in case :???:

Search for new PVs on DEST and add them to LVM

root@dest-host# pvscan

Merge the two volume groups into one

root@dest-host# vgmerge DEST-VG ORIG-VG

Move LVs from ORIG-PV to DEST-PV. This is the step that’s gonna take forever or not depending on your network speed and the amount of data to move. I don’t have my exact figures but I recall it taking a couple of hours.

root@dest-host# pvmove -n some-lv /dev/etherd/e0.0 /dev/md1

Note this time /dev/md1 is the destination RAID device which belongs to DEST-VG, not the one on ORIG box stated before.

Once it finishes we want to make things go back to normal. So we disable the VG so we can do the split (James Brown style baby!) Then we re-enable the VG

root@dest-host# vgchange -an DEST-VG
root@dest-host# vgsplit DEST-VG ORIG-VG /dev/etherd/e0.0
root@dest-host# vgchange -ay DEST-VG

Now we have to disable the ORIG-VG and make it unknown to the DEST LVM system

root@dest-host# vgexport ORIG-VG

Back to the ORIG box we re-add the “lost” VG and re-activate the VG

root@orig-host# vgimport ORIG-VG
root@orig-host# vgchange -ay ORIG-VG

And that should be it! I’ll stop writing now since it’s a long post already. Hope somebody finds this useful and/or inspiring. Good luck!

[1] http://www.howtoforge.com/ata_over_ethernet_debian_etch

• Sony Pictures
• Warner Brothers
• Universal
• 20th Century Fox
• Walt Disney Pictures
• Paramount
• MGM (bought in 2005 by capitals in part from Sony)

:arrow: These corporations control more than 95% of the U.S. film business :!:

Even more, these corporations are part of even larger conglomerates:

• Sony Pictures → Sony
• Warner Brothers → TimeWarner
• Universal → General Electric
• 20th Century Fox → News Corp.
• Walt Disney Pictures → The Walt Disney Company
• Paramount → VIACOM
• MGM → Sony

:arrow: These last conglomerates own more than 90% of all media in the U.S. :!:

PS: Watch the movie. It’s a documentary about how the MPAA accomplishes censorship in the land of freedom in the most open and declared way. One more and counting…

Note: I apologize to Luke Welling for taking this from his blog and never citing the source. An anonymous comment made me realize about my mistake.