Version 12 (modified by Daniel Kahn Gillmor, 11 years ago) (diff)


How to Setup an MF/PL Xen server

This document describes how to set up a debian Xen dom0 with the associated hypervisor for a standard May First/People Link server. It assumes you're starting with a minimal stock debian install.

Install the hypervisor

  • Install with (change the version number of xen-linux-system-2.6.18-6-xen-686 to the version you need; also on amd64 machines you don't libc6-xen):
    aptitude install xen-hypervisor xen-linux-system-2.6.18-6-xen-686 libc6-xen
  • limit the dom0 to 128M of RAM, and make sure both the hypervisor and dom0 kernel use the serial console. A convenient way to do this on debian is to edit /boot/grub/menu.lst so that the following lines are contained within the AUTOMAGIC KERNELS LIST section:
    ## Xen hypervisor options to use with the default Xen boot option
    # xenhopt=dom0_mem=131072 com1=115200,8n1
    ## Xen Linux kernel options to use with the default Xen boot option
    # xenkopt=console=tty0 console=ttyS0,115200n8
  • run update-grub and reboot

Set up the dom0's network environment

  • Install bridge-utils:
    aptitude install bridge-utils
  • Tell Xen to bridge the primary network interface across all domUs -- you can do this by applying the following diff to /etc/xen/xend-config.sxp:
    --- xend-config.sxp.orig        2008-03-13 11:23:56.000000000 -0400
    +++ xend-config.sxp     2008-03-13 17:40:57.000000000 -0400
    @@ -68,7 +68,7 @@
     # use
    -# (network-script network-bridge)
    +(network-script network-bridge)
     # Your default ethernet device is used as the outgoing interface, by default.
     # To use a different one (e.g. eth1) use
    @@ -84,7 +84,7 @@
     # two fake interfaces per guest domain.  To do things like this, write
     # yourself a wrapper script, and call network-bridge from it, as appropriate.
    -(network-script network-dummy)
    +#(network-script network-dummy)
     # The script used to control virtual interfaces.  This can be overridden on a
     # per-vif basis when creating a domain or a configuring a new vif.  The
  • Restart xen
    /etc/init.d/xend restart

Prepare for convenient domU creation and maintenance

  • install xen-tools:
    aptitude install xen-tools
  • Edit /etc/xen-tools/xen-tools.conf to adjust for three different things:
    • new domUs should be created with debootstrap, since we want debian domUs
    • domU block devices should be pulled from the primary volume group of the domU
    • domUs should use the same kernel as the dom0.

Here's the example diff from gramsci:

0 gramsci:/etc/xen-tools# diff -u xen-tools.conf.orig xen-tools.conf
--- xen-tools.conf.orig 2007-05-27 19:59:38.000000000 -0400
+++ xen-tools.conf      2007-05-27 20:01:15.000000000 -0400
@@ -35,6 +35,7 @@
# lvm = skx-vg
+lvm = vg_gramsci0

@@ -61,7 +62,7 @@
# copy = /path/to/pristine/image
-# debootstrap = 1
+debootstrap = 1
# rpmstrap = 1
# tar = /path/to/img.tar
@@ -95,7 +96,7 @@
swap   = 128Mb    # Swap size
# noswap = 1      # Don't use swap at all for the new system.
fs     = ext3     # use the EXT3 filesystem for the disk image.
-dist   = sarge    # Default distribution to install.
+dist   = etch     # Default distribution to install.
image  = sparse   # Specify sparse vs. full disk images.

@@ -154,8 +155,8 @@
# Default kernel and ramdisk to use for the virtual servers
-kernel = /boot/vmlinuz-2.6.16-2-xen-686
-initrd = /boot/initrd.img-2.6.16-2-xen-686
+kernel = /boot/vmlinuz-2.6.18-6-xen-686
+initrd = /boot/initrd.img-2.6.18-6-xen-686

#  The architecture to use when using debootstrap or rpmstrap.
1 gramsci:/etc/xen-tools#
  • make sure there's enough room in the dom0's filesystem for the domUs to all have their state saved to disk -- this is what happens at a normal shutdown when /etc/init.d/xendomains stop gets invoked. In particular, the domU state gets stored in /var/lib/xen/save, and the size of the state files is roughly the size of the memory (plus a little bit extra for the state of the processors). A decent rule of thumb is to allocate the total size of physical memory + 10% for a separate partition on /var/lib/xen. Here's the situation on fred for example:
    0 fred:~# xm info | grep ^total_memory
    total_memory           : 8188
    0 fred:~# df -h /var/lib/xen
    Filesystem            Size  Used Avail Use% Mounted on
                          8.9G  149M  8.3G   2% /var/lib/xen
    0 fred:~#