wiki:faq/backup-setup

Version 10 (modified by Jackrabbit, 11 years ago) (diff)

--

If you are running a linux system and you'd like an offsite server to back your data up to, you can use one of May First/People Links' shared backup servers. If you are using an OS X server, you will not be able to use these directions. You will be able to do backups using rsync (with: rsync+hfsmode), however, rdiff-backup uses the rsync libraries, it does not call rsync directly. So getting the proper HFS file system support will require using rsync+hfsmode directly). If you are using Windows, we'd strongly encourage you to setup a linux file server prior to trying any type of automated backup system.

Having said that...You are free to use whatever backup software you like, provided that it can copy files via ssh using an ssh public/private key pair.

We recommend that you use backupninja and rdiff-backup. Below are instructions for how to get started with these two programs.

If you are not running Debian, please check out the respective web sites for backupninja and rdiff-backup to download the programs (http://dev.riseup.net/backupninja and http://www.nongnu.org/rdiff-backup).

The directions below assume that you are running Debian linux.

  • Create an ssh key pair which does not require a passphrase while logged in as the root user on your server (if you haven't already):
ssh-keygen -t rsa
  • Post a ticket, with your public key attached, asking to have your public key installed on our server. On your server, the public key should be stored in /root/.ssh/id_rsa.pub.
  • You will be told which of our backup servers to use (for example, b.backup.mayfirst.org or c.backup.mayfirst.org, etc.). This information will be used below.
  • Test the public key (after you've received confirmation that it has been installed) by trying to ssh into our backup server while you are root (replace b.backup.mayfirst.org with your backup server).
ssh MEMBERNAME-sync@b.backup.mayfirst.org
  • Install backupninja and rdiff-backup

NOTE: You'll need to install the same version of rdiff-backup that we have on our server. Our version runs the version of rdiff-backup in Debian Lenny. If you are running Sarge or Etch, you'll want to install rdiff-backup from backports.org. As of December 30, 2009 we are using rdiff-backup version 1.2.5.

aptitude install backupninja rdiff-backup
  • Create a file in /etc/backup.d called 90-mfpl.rdiff with contents along the lines of:
#options = --force

######################################################
## source section
## (where the files to be backed up are coming from)

[source]


# an optional subdirectory below 'directory' (see [dest]).
# This should be set to the name of the particular server you're backing up
# in order to distinguish between multiple server backups.
label = SERVERNAME

# only local type is currently supported
type = local

# how many days of data to keep
keep = 5

# files to include in the backup
# (supports globbing with '*')
# You can add additional lines to back up additional files or directories.
include = /etc
include = /var/log
include = /var/www
include = /home

# files to exclude from the backup
# (supports globbing with '*')
exclude = /etc/dnscache/*

######################################################
## destination section
## (where the files are copied to)

[dest]

# only remote type is currently supported
type = remote

# the machine which will receive the backups
# REPLACE WITH YOUR BACKUP SERVER
host = b.backup.mayfirst.org

# put the backups under this directory
directory = backups/

# make the files owned by this user
# note: you must be able to ssh backupuser@backhost
# without specifying a password
user = MEMBERNAME-sync
  • Edit /etc/backupninja.conf

Schedule the back up to start sometime between 11:00 pm and 6:00 am America/New_York time.

  • Test with:
backupninja -t -n -d

(t is test and tells backupninja to simply test the backup procedure without actually backing up the files; n is now, which tells backupninja to ignore its regular schedule and run right now; d is debug, which tells backupninja to give detailed feedback on what it's doing and any errors it encounters.)

NOTE: in it's current release backupninja has no internal way to limit bandwidth (bandwidth limiting will be available in future releases). There are two good reasons for setting bandwidth limits: so that your internet connection's bandwidth isn't completely monopolized while the backup is running, and so that you don't monopolize May First's bandwidth and make it harder for others to do backups at the same time as you. Therefore, you should run it through trickle. dkg explains it with:

Say you want to limit backupninja 50Kbps down and 70Kbps up. Instead of running

backupninja -n

you can run

trickle -d 50 -u 70 backupninja -n

Since you won't usually be running backupninja manually, you should also change backupninja's cron job to use trickle. By default in Debian, the cron job will be found in /etc/cron.d/backupninja. You should change the cron job to look something like this:

# /etc/cron.d/backupninja -- cron tab entry for package backupninja

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# run backupninja every hour on the hour
0 * * * * root if [ -x /usr/sbin/backupninja ]; then /usr/bin/trickle -d 50 -u 70 /usr/sbin/backupninja; fi

A few additional thoughts from abh:

If you're getting funky errors, definitely ask for help here, but try making sure you're backing up to the right directory. I wasn't!

And - hints for setting up a nullmailer to handle emailing: here.