Opened 3 months ago

Closed 2 months ago

#14276 closed Bug/Something is broken (fixed)

Ensure ListServ lists are backed up

Reported by: https://id.mayfirst.org/wolcen Owned by: https://id.mayfirst.org/jamie
Priority: High Component: Tech
Keywords: Cc:
Sensitive: no

Description

Hello,

Looking at morales, it appears there is an rsync job that would cover listserv (from /srv), however I do not see a backup ninja task for rdiff backups. Does this mean that a single backup is performed here? If, for example, a mailing list blew up somehow - we'd have 24 hours to detect and restore? Going on the logs, it appears the rsync does complete successfully (just with some files being removed). Granted, I can't see those/don't want to - but I have faith they are on iz :)

Please confirm, if you would, and please add some additional backup history, if necessary/possible. The most critical files, I believe to be the ones in /srv/listserv/home/

Thanks!

Change History (5)

comment:1 Changed 2 months ago by https://id.mayfirst.org/clayton

  • Priority changed from Medium to High

Hi MayFirst,

Can we get a response to this?

Thanks!

comment:2 Changed 2 months ago by https://id.mayfirst.org/wolcen

This system has a larger DB that should be being backed up as well, so please confirm that ahead of pending changes for the SSD.

comment:3 Changed 2 months ago by https://id.mayfirst.org/jaimev

  • Owner set to https://id.mayfirst.org/jamie
  • Status changed from new to assigned

I can confirm the contents of /srv are being backed via rsync to iz. rdiff-backup is no longer running for morales. I believe the new strategy for secondary backup is to perform daily borg backups of morales backups from iz to minnie. Copying jamie for more input.

Where is the database stored?

comment:4 Changed 2 months ago by https://id.mayfirst.org/wolcen

My apologies - seems I can contribute most to confusion. Mumia != Morales, as was the thinking in my head at that moment.

Sorry :-/

The large DB of concern is just on mumia, of course. This system has no surgery scheduled, so don't let me swap patients.

comment:5 Changed 2 months ago by https://id.mayfirst.org/jamie

  • Resolution set to fixed
  • Status changed from assigned to closed

No problem - I always appreciate an opportunity to audit our backups.

Based on our new backup strategy, data is rsync'ed from morales to iz every night, overwriting the previous backup.

During the day, iz performs an incremental borg backup to minnie, our offsite server. And those seem to be intact:

0 borg-iz@minnie:~$ borg list servers/morales/
20181208                             Sat, 2018-12-08 11:00:58
20181209                             Sun, 2018-12-09 11:00:25
20181210                             Mon, 2018-12-10 11:00:14
20181211                             Tue, 2018-12-11 11:00:16
20181212                             Wed, 2018-12-12 11:01:23
20181213                             Thu, 2018-12-13 11:00:34
20181214                             Fri, 2018-12-14 11:00:57
20181215                             Sat, 2018-12-15 11:00:34
20181216                             Sun, 2018-12-16 11:00:15
20181217                             Mon, 2018-12-17 11:00:26
0 borg-iz@minnie:~$

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.