wiki:wheezy-jessie-upgrade

Wheezy Jessie Final Upgrade

Thanks to the great effort made on Saturday March 5, we now have:

163 Total servers in puppet:

  • 16: N/A (we don't have access or already turned off but not yet purged from puppet)
  • 13: Scheduled to be retired in coming months
  • 4: squeeze
  • 12: wheezy
  • 118: jessie

Only 16 more to upgrade!

Steps to upgrade

For a Mosh: read mosh upgrade page, more complicated.

For other servers:

  • Read what the server does in puppet - test services before upgrading
  • Delete all sources and preferences files:
    rm -f /etc/apt/preferences.d/* /etc/apt/sources.list.d/*
    
  • Edit source.list to change to squeeze/jessie
    nano /etc/apt/sources.list
    
  • Update and upgrade
    apt-get update && apt-get dist-upgrade
    
  • Test services again

If the server is running squeeze, upgrade to wheezy first.

The more detailed upgrade process

Here's my list of servers to upgrade:

General

  • assata:wheezy (jaimev)
  • barron:wheezy AWAITS REBOOT (jaimev)
  • cleaver:wheezy AWAITS REBOOT (jaimev)
  • clr:wheezy AWAITS REBOOT (jaimev)
  • florence:wheezy AWAITS REBOOT (jaimev)
  • fuller:wheezy problem booting (jaimev)
  • howard:wheezy (steve)
  • iz:wheezy (greg)
  • jojobe:wheezy (jaime: UK server)
  • keller:wheezy (greg)
  • leslie:wheezy (greg)
  • malaka:wheezy (greg/jamie)
  • mcchesney:wheezy (nat)
  • parsi:wheezy (nat) AWAITS REBOOT
  • peltier:wheezy (nat)
  • pietri:wheezy (nat)
  • revueltas:wheezy (airtime) (retired see #8172)
  • robideau:squeeze (steve)
  • sankara:wheezy (nat)
  • stallman:wheezy (roundcube) (steve)
  • toussaint:wheezy (nat)
  • vilma:wheezy (nat)
  • zimmermann:wheezy (steve)
  • ranciere:wheezy (old telehouse-based authoritative DNS server, nothing live on it now) (jamie)

Moshes

  • hashmi:squeeze (jaime: keep PHP pinned to current version, ensure apache files all end in .conf)
  • jones:squeeze (jaime: remove PHP pinning, ensure apache files all end in .conf)
  • roe:squeeze (ensure apache files all end in .conf)
  • slaapbeen:wheezy (jaimev)
  • sojourner:wheezy (jamie to notify agaric)

Special Care

These should be handled carefully because they are dedicated to members or handle mission critical services

Sahara Reporters (Ross)

  • acholonu:wheezy - Ross scheduled for July 15, 2017
  • ngugi:wheezy
  • saadawi:wheezy
  • deadeasy:wheezy
  • franz:wheezy

PTP (jamie)

  • augusto:wheezy
  • dorvalina:wheezy
  • marti:wheezy

Yes men

  • banksy:wheezy
  • baubo:wheezy

Varnish (Ross?)

Known issues with migration path.

  • dolores:wheezy

Rochester IMC

  • douglass:squeeze (ben)

Laneta (Jaime)

Misc

  • barghouti:wheezy (jamie to contact CIVIC)
  • JAMIE morales:squeeze (Portside - listserv)
  • JAMIE hay:wheezy (careful! running members.mayfirst.org, id.mayfirst.org)
  • gamiz:wheezy (authoritative DNS) (jamie)
  • kennedy:wheezy (authoritative DNS) (jamie)
  • kramer:squeeze (Philly Fight - running koha) (jaime)
  • moses:wheezy (running support.mayfirst.org - dkg?)
  • rustin:wheezy (bulk mail relay) (jamie)
  • cleveland:wheezy (bulk mail relay) (steve)
  • woodhull:wheezy (jamie to contact NLG - file sharing)

Here's my list of servers that need upgrades but will be retired during xo -> telehouse transition so it's not worth the effort:

dee:squeeze
axiom:squeeze
ottorene:squeeze
roy:squeeze
swartz:squeeze
luisa:squeeze
negri:wheezy
achebe:wheezy
bouazizi:wheezy
herman:wheezy ([[wiki:decommission_kvm|retire]]) (steve)

Exceptional Cases

Zimmerman. apt-get dist-upgrade proceeded to

Selecting previously unselected package dmeventd.
Preparing to unpack .../dmeventd_2%3a1.02.90-2.2_amd64.deb ...
Unpacking dmeventd (2:1.02.90-2.2) ...
Setting up init-system-helpers (1.22) ...
Setting up dmeventd (2:1.02.90-2.2) ...
Failed to parse message: No such device or address

and then hung. Unresponsive from ssh session I used for the upgrade; wouldn't accept new ssh connections. We ended up killing zimmermann's kvm process, restarting, and continuing the upgrade from the console.

Later error:

Setting up sks (1.1.5-3) ...
debian-sks uid check: ok
debian-sks homedir check: ok
db5.3_upgrade: BDB0671 /var/lib/sks/DB/DB_CONFIG~: unrecognized file type
db5.3_upgrade: DB->upgrade: /var/lib/sks/DB/DB_CONFIG~: Invalid argument
dpkg: error processing package sks (--configure):
 subprocess installed post-installation script returned error exit status 1
Setting up ca-certificates (20141019+deb8u1) ...

Removed the emacs backup file, then dpkg --configure -a. Next error

0 zimmermann:~# dpkg --configure -a
Setting up sks (1.1.5-3) ...
debian-sks uid check: ok
debian-sks homedir check: ok
cp: error writing '/var/backups/sks/20160305-143824/DB/key': No space left on device
cp: failed to extend '/var/backups/sks/20160305-143824/DB/key': No space left on device
dpkg: error processing package sks (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 sks

This was harder to sort out

cd /var/lib/sks/DB
db5.1_recover -v complained about an unrecognized file format
db5.3_recover -v ran recovery successfully

/var/lib/sks/berkeley_db.active contained "5.1". I changed this to "5.3", since the files seem to be readable by 5.3 rather than 5.1.

After that, dpkg --configure -a succeeded.

Stallman. Convert postgres 9.1 -> 9.4

pg_dumpall --verbose > pg91.out

# Verify roundcube databases not in 9.4
psql -p 5433 --list

# load dump into 9.4
psql -p 5433 < pg91.out

# check
psql -p 5433 --list

Next, stop all postgres instances, and switch 9.4 from the alternate port to the main port.

systemctl stop postgresql@9.1-main.service
systemctl stop postgresql@9.4-main.service

Set port = 5432 in /etc/postgresql/9.4/main/postgresql.conf

systemctl start postgresql@9.4-main.service
systemctl disable postgresql@9.1-main.service

0 roundcube@stallman:~$ psql -U roundcube roundcube
psql (9.4.6)
Last modified 7 years ago Last modified on Jun 28, 2017, 8:36:25 PM