Version 21 (modified by Jamie McClelland, 2 years ago) (diff)


Server Installation

Initial steps

  • Plug in monitor and keyboard
  • Enter Bios/Setup. Specify that Bios should output to serial console and make our standard bios changes
  • Ensure that server skips errors like no keyboard attached
  • Reboot and select Boot Options and choose to boot to network from F12 menu
  • Either pxe boot from telehouse or xo.
  • Choose Expert install
  • For DNS servers, use:
  • When prompted for component to use, include all of them to be safe (but be sure to include the SSH server one)
  • When given the option - choose to continue your installation via ssh - this will give you the ability to easily set and record pass phrases

Drive partioning/Disk setup

Our servers are now coming with four hot swappable disks.

The big picture is

  • All partitions should be Primary
  • One approximately 512MB partition on all four disks: configured as RAID1, used as boot partition
  • The rest of the space on each disk: configured as RAID10, used as encrypted disk
  • Encrypted disk: used as physical volume for LVM
  • Create on volume group: vg_${server_name}0
  • Create standard partitions as logical volume

Details - Using the Installer or manually via debootstrap

Since we are using disks at are 2TB or bigger, we need to ensure that the partitions are created properly. Also, we leave 100MB at the end in case a replacement disk is a smidge smaller (thanks bug magnet!). Open a new ssh session and choose to switch to console.

parted /dev/sda -- mklabel gpt
parted /dev/sda -- unit s mkpart biosboot 8192 16383 
parted /dev/sda -- set 1 bios_grub on 
parted /dev/sda -- unit s mkpart boot 16384 1015807
parted /dev/sda -- set 2 raid on 
parted /dev/sda -- unit s mkpart pv 1015808 -196608 
parted /dev/sda -- set 3 raid on 

Repeat for sdb, sdc, and sdd.

When done, type:

parted /dev/sda -- unit s p

And you should get:

0 sittingbull:~# parted /dev/sda unit s p
Model: ATA Hitachi HUA72303 (scsi)
Disk /dev/sda: 5860533168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start     End          Size         File system  Name      Flags
 1      8192s     16383s       8192s                     biosboot  bios_grub
 2      16384s    1015807s     999424s                   boot      raid
 3      1015808s  5860533134s  5859517327s               pv

0 sittingbull:~#

You should be able to return to the graphical installer to do the rest.

Using the Installer

Choose manual partition in the Debian Installer. Then:

  • Choose "Congifure software RAID" (skip step if one disk system)
    • Choose "Create MD Device"
    • Choose RAID1 for boot partition and RAID10 for big partition
    • Number of active devices: 4
    • Number of spare devices: 0
    • Now select the second partition on each device.
    • Click Continue. Repeat for third partition on each device but select RAID10 instead of RAID1.
    • When you are done, click Finish. Now you are back at the partition menu.
  • Scroll down to the raid devices (or apply straight to your partitions if it's a one disk system). Modify the 512 MB RAID device: Filesystem: ext3, mount on /boot
  • Encrypt the larger, remaining device. Save passphrase in keyringer.
  • Choose "Congifure LVM" (selecting the device encrypted in the previous step)
  • Create a volume group called vg_nameofserver0
  • Create logical volumes in this volume group based on your needs. Suggestions: 3GB for /, 1GB swap.
  • After returning to the main disk config menu, click on each logical volume that you create and specify how it should be formatted and mounted.
  • Enter root password. Save in keyringer.
  • Do not install the server package or the base package - deselect all of them.

Views of Disk Configuration

  • all partitions set to use physical volume for RAID
      ┌────────────────────────┤ [!!] Partition disks ├─────────────────────────┐
      │                                                                         │
      │ You are editing partition #1 of SCSI1 (0,0,0) (sda). No existing file   │
      │ system was detected in this partition.                                  │
      │                                                                         │
      │ Partition settings:                                                     │
      │                                                                         │
      │               Use as:         physical volume for RAID                  │
      │                                                                         │
      │               Bootable flag:  off                                       │
      │                                                                         │
      │               Copy data from another partition                          │
      │               Delete the partition                                      │
      │               Done setting up the partition                             │
      │                                                                         │
      │     <Go Back>                                                           │
      │                                                                         │

You can also install everything manually using debootstrap

Picking up from where we left off partitioning the disks...


mdadm --create --raid-devices=4 --level=1 --metadata=1.0 --verbose /dev/md0 /dev/sd[abcd]2 
mdadm --create --raid-devices=4 --level=10 --metadata=1.0 --verbose /dev/md1 /dev/sd[abcd]3

Now cryptsetup:

cryptsetup luksFormat /dev/md1
cryptsetup luksOpen /dev/md1 md1_crypt

pvcreate /dev/mapper/md1_crypt
vgcreate vg_nameofserver0 /dev/mapper/md1_crypt
lvcreate --name swap --size 1GB vg_nameofserver0

Then, create fileystems:

  • For the boot partition
mkfs -t ext4 /dev/md0
  • Repeat for each logical volume
    mkfs -t ext4 /dev/mapper/vg_nameofserver0-nameofpartition
  • Then, mount all partitions in /mnt
    mount /dev/mapper/vg_cleaver0-root /mnt
    mkdir /mnt/{boot,var,proc,dev,sys}
    mount /dev/md0 /mnt/boot
    mount /dev/mapper/vg_cleaver0-var /mnt/var

Note: Don't mount /proc, /sys, and /dev before running debootstrap - you will end up with a broken installation (apt won't be available for one thing).

  • Run debootstrap
    debootstrap stretch /mnt
  • Bind mount kernel directories:
    mount -o bind /proc /mnt/proc
    mount -o bind /sys /mnt/sys
    mount -o bind /dev /mnt/dev
  • chroot
    chroot /mnt
  • Add and populate/modify the following files (use an existing system to compare format etc.)
    • /etc/network/interfaces.d/eth0
    • /etc/fstab
    • /etc/crypttab
    • /etc/default/grub
    • /etc/hosts
    • /etc/hostname
  • Install a lot of necessary packages for booting:
    apt install mdadm lvm2 cryptsetup grub-pc linux-image-amd64 bridge-utils
  • Generate /etc/mdadm/mdadm.conf file. Generate with:
/usr/share/mdadm/mkconf > mdadm.conf
  • Set root passwd:

Additional Steps

  • Create a DNS host from control panel.
  • Get ssh access - Install bridge-utils and configure /etc/network/interfaces to look like this (make changes as needed to IP Address information.
0 florence:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto br0

iface br0 inet static
 bridge_ports eth0
0 florence:~# 
  • As of May 16, 2013, we still have a MAC Address bug (see #5743). The first created guest will need to have the MAC Address created manually by editing the /etc/sv/kvm/GUEST_NAME/env/MAC file and creating a manually randomized MAC Address.
  • This is an example of a broken MAC Address (the middle four columns should not be zeros, instead change them to numbers and letters.
    0 baubo:/etc/sv/kvm/yippie/env# cat MAC
    0 baubo:/etc/sv/kvm/yippie/env#