Version 5 (modified by Ross, 7 years ago) (diff)


Moving guests from one server to another using bdsync

bdsync is a synchronizing tool designed to work on block devices, as opposed to rsync which is designed to work on file systems.

It works like this:

  • Generate a diff file that records the differences between two block devices (typically one is local and the other is on a remote machine).
  • Apply the diff file to the remove machine's block device so that they are in sync.

Using bdsync to move virtual guests on MF/PL infrastructure =

bdsync is not (yet) in debian. So, it has to be compiled. Fortunately it is a small program that is easy to compile.

On the source machine

  • As root, install dependencies:
    apt-get install make gcc libssl-dev libc6-dev
  • As root, Create a bdsync user to compile the program:
    useradd bdsync
  • Become that user
    su - bdsync
  • Check out the source and cd into the source directory:
    git clone && cd bdsync
  • Make:

You should end up with a bdsync binary in your current directory.

On the target machine

  • If the source and target machines are running the same version of debian and have the same architecture, just copy the bdsync binary to the target machine. Otherwise, repeat this process on the target machine.
  • If you haven't already, create a bdsync user on the target machine:
    useradd bdsync
  • Add the source server's root user to the monkeysphere authorized user ids file. Replace $TARGETSERVER with the name of the target server, e.g. "chavez":
    echo "root@$" >> /home/bdsync/.monkeysphere/authorized_user_ids
  • Update monkeysphere credentials
    monkeysphere-authentication update-users bdsync

Create first diff

On the target machine, figure out which device is the one being used, e.g.:

0 cleaver:~# ls -l /dev/mapper/vg_cleaver0-fuller 
lrwxrwxrwx 1 root root 7 Dec 11 19:39 /dev/mapper/vg_cleaver0-fuller -> ../dm-3
0 cleaver:~#

Then, chgrp it so that the bdsync user can read it:

chgrp bdsync /dev/dm-3

On the source machine execute the following command. Replace $LOCALDEV with the path to the block device on the local machine and $REMDEV with the path of the block device on the remote machine (e.g. /dev/mapper/vg_kiyoshi0-chavez). Replace $TARGETSERVER with the name of the target server (e.g. "chavez"):

Running via ionice ensures that it doesn't kill all disk i/o on the machine.

ionice -c 2 -n 7 bdsync "ssh bdsync@$ bdsync --server" $LOCDEV $REMDEV | gzip > DEV.bdsync.gz

If you have problems, check syslog on the target server.

Now, copy the diff to the target machine.

scp DEV.bdsync.gz bdsync@$

Apply first diff

On the target server, apply the diff:

gzip < DEV.bdsync.gz | bdsync --patch=$REMDEV

Final sync

After running once, shutdown the source machine and repeat to get the final diff. Then repeat the application of the diff.