Version 2 (modified by 8 years ago) (diff) | ,
---|
How to reboot a physical server and guests
On occasion we need to reboot a physical server and all of it's guests (often due to a kernel upgrade). This is an intensive process and can take a long time as many servers need to run fsck on reboot which demands intensive disk activity. This page documents the currently recommended process for such reboots.
Make sure all guests on a host are updated
- run package updates
apt-get update apt-get -s dist-upgrade
- if the server needs upgrades
apt-get dist-upgrade
- if the server needs upgrades
Check available disk space
- Make sure there is available space on the logical volumes, and take appropriate steps if not, see extend-disk-on-kvm-guest
- if you added disk space from the host, reboot the guest with
shutdown -h now
- if you added disk space from the host, reboot the guest with
Determine if the guest will fsck on reboot
- On kvm guests or vserver hosts check when the next fsck is scheduled for all logical volumes with:
for i in $(ls /dev/mapper/vg_* | grep -v swap); do tune2fs -l $i | grep Next; done
- e.g.
0 malcolm:~# for i in $(ls /dev/mapper/vg_* | grep -v swap); do tune2fs -l $i | grep Next; done Next check after: Fri Oct 11 12:12:47 2013 Next check after: Fri Oct 11 12:12:28 2013 Next check after: Thu Aug 29 23:16:36 2013 Next check after: Fri Oct 11 13:07:33 2013 Next check after: Fri Oct 11 13:08:17 2013 Next check after: Fri Oct 11 13:08:40 2013 0 malcolm:~#
- e.g.
- If a guest will fsck on reboot (i.e. the date from the above is earlier than the current date). Reboot a kvm guest from root@GUEST with. You can watch reboot progress with
ssh -t guest@host screen -x
shutdown -r now
Do this for all guests on the host.
Take down the host's guests
FIXME: Add better information for vservers, this is currently just about kvm
- From
root@host
check running services:cd /etc/service for x in *; do sv status $x; done
- Take down each guest's service:
sv down GUESTNAME
- Make sure the guest has gone down, this may take a few seconds.
- If the guest has not gone down in a short time, you may need to manually restart from
root@guest
with:shutdown -h now
- If the guest has not gone down in a short time, you may need to manually restart from
- Make sure the guest has gone down, this may take a few seconds.
- Check to make sure all guests are down, repeat the status command above. You should see a line that starts with
down: GUESTNAME
for all guests.
Reboot the host
- Assuming all the guests are down, you should be able to restart the server.
- Get access to the serial console
- Telehouse use telehouse_serial_access
- XO use xo_serial_access
- Shut down the system from
root@host
withshutdown -r now
- Watch the reboot process from the serial console and enter the encrypted password when requested (accessed via keyringer)