#12862 closed Task/To do item (fixed)

switch owncloud to nextcloud

Reported by: https://id.mayfirst.org/jamie Owned by: https://id.mayfirst.org/jamie
Priority: High Component: Tech
Keywords: lucious.mayfirst.org owncloud nextclout Cc:
Sensitive: no

Description

We are currently running version 8 of owncloud.

Nextcloud is a fork that is more in lines with the values of our organization (http://karlitschek.de/2016/06/nextcloud/ - no dual licensing, more transparent).

We should upgrade to the current nextcloud version.

Change History (14)

comment:1 Changed 18 months ago by https://id.mayfirst.org/jamie

I started by installing nextcloud 1.4.2 for android (via fdroid) and tried to connect to our existing owncloud installation and unfortunately, it crashed. Which suggests we may need to upgrade our server before asking people to upgrade their clients.

comment:2 Changed 18 months ago by https://id.mayfirst.org/jamie

  • Owner set to https://id.mayfirst.org/jamie
  • Status changed from new to assigned

comment:3 Changed 18 months ago by https://id.mayfirst.org/jamie

I've completed an installation to https://share.dev.mayfirst.org/.

At the moment it is an exact replica of the live install (except with a separate database, data directory etc).

I'll be adding content to this site and then the next step will be to practice upgrading it to next cloud.

comment:4 Changed 17 months ago by https://id.mayfirst.org/jaimev

Today we talked about trying to look at the logs in the current owncloud install to get an idea of what dedicated clients people are using to connecting to the existing service. This might tell us how much we have to prepare them for the change to nextcloud.

comment:5 Changed 17 months ago by https://id.mayfirst.org/jamie

I've prepared a wiki page describing how the nextcloud server will be setup. It optimistically anticipates switching from apache to nginx/php5-fpm.

I'm also relying heavily on this nextcloud page on migration which suggests we should upgrade to Nextcloud 9 next (which is compatible with owncloud 9). Then move to nextcloud 10, 11 and 12.

And, here is a list of steps for the upgrade:

Pre Upgrade

  • Install docker and start collabora docker image (see nextcloud admin page)
  • Prepare the following directories and configuration files
    • /var/lib/nextcloud/{apps-local,data,themes}
    • /etc/nextcloud/config.php
    • /var/www/nextcloud
  • Download nextcloud version 9.0 into /var/www/nextcloud and prepare the symlinks
  • Download the latest nextcloud releases of apps into /var/lib/nextcloud/apps-local
  • Prepare a Postgres database named nextcloud and ensure www-data has full write access to it
  • Prepare /etc/apache2/sites-enabled/nextcloud.conf that points to /var/www/nextcloud
  • Install php5-fpm and prepare the php configuration per the nextcloud wiki page
  • Upload /root/nginx/nextcloud.conf and /root/nginx/collabora.conf files - so they are ready to be moved into place when we switch apache for nginx
  • Prepare new versions of the following scripts: /usr/local/sbin/mf-owncloud-purge-user, /usr/local/sbin/mf-owncloud-list-users and /usr/local/sbin/mf-owncloud-purge-deleted-users

Upgrade

  1. Put owncloud in maintenance mode
    su - www-data
    cd /var/www/owncloud
    ./occ maintenance:mode --on
    
  2. Make a dump of the database
    su -c "pg_dump owncloud" www-data > /root/upgrades/backups/owncloud.pre.nextcloud.9.backup.sql
    
  3. Umount /var/lib/owncloud/data and remount it as /var/lib/nextcloud/data
    umount /var/lib/owncloud/data
    vim /etc/fstab
    mount /var/lib/nextcloud/data
    
  4. Drop existing nextcloud database and import the dump into the nextcloud database
    su - postgres
    psql nextcloud
    DROP SCHEMA public CASCADE;
    CREATE SCHEMA public;
    GRANT ALL ON SCHEMA public TO postgres;
    GRANT ALL ON SCHEMA public TO public;
    quit
    exit
    su - www-data
    cat /root/upgrades/backups/owncloud.pre.nextcloud.9.backup.sql | psql -f - nextcloud
    
  5. Delete /etc/apache2/sites-enabled/owncloud.*
    rm /etc/apache2/sites-enabled/owncloud.*
    
  6. Enable the nextcloud versions:
    a2ensite nextcloud.conf
    a2ensite nextcloud.ssl.conf
    
  7. Run ./occ upgrade
    su - www-data
    cd /var/www/nextcloud
    ./occ upgrade
    
  8. Login as mfpl-admin (external auth won't be enable-able until we reach nextcloud 12) and enable bookmarks, calendar, contacts
  9. Run ../occ upgrade (same as above)
  10. Move /var/www/nextcloud to /var/www/nextcloud.9 and and Move /root/upgrades/nextcloud.10 to /var/www/nextcloud and repeat steps 5, 6, and 7. Then to it again for 11 and 12.
  11. Execute the following SQL statements to prepare the new external auth app:
        INSERT INTO oc_users_external SELECT 'mfpl' AS backend, uid, displayname FROM oc_users WHERE oc_users.uid != 'mfpl-admin';
        DELETE FROM oc_users WHERE oc_users.uid != 'mfpl-admin';
    
  12. Login as mfpl-admin and enable user_external and mfplauth and richtext/collabora apps
  13. Test the mf-nextcloud-purge-deleted-users script in debug mode.
  14. Edit the mf-nextcloud-purge-deleted-users-cron to enable it and purge the mf-owncloud-* cron and scripts.

Post Upgrade

  • Stop apache2
  • Install nginx
  • Move nginx files into place

Delete all things owncloud (/var/lib/owncloud, /var/www/owncloud, postgres owncloud database)

Last edited 17 months ago by https://id.mayfirst.org/jamie (previous) (diff)

comment:6 Changed 17 months ago by https://id.mayfirst.org/jamie

All the pre-steps are done. Ready to go!

comment:7 Changed 16 months ago by https://id.mayfirst.org/jamie

I've just updated the nextcloud.12 directory to include the recently released 12.1 update.

comment:8 Changed 16 months ago by https://id.mayfirst.org/jamie

Upgrade complete!

I'm still working on the last few cleanup steps.

comment:9 Changed 16 months ago by https://id.mayfirst.org/jamie

Mostly successful however, unfortunately, our docker image providing the collabora libre office code s failing.

We seem to be afflicted by both:

The capabilities thread recommends changing docker to use the devicemapper storage device for jessie and overlay2 for stretch. So I think that's a good place to start (tomorrow).

comment:10 Changed 16 months ago by https://id.mayfirst.org/jamie

Switching to devicemanager is difficult to configure and ultimately not our best move (it seems that overlay2 is the direction we should be going to).

However, overlay2 requires a linux kernal of 4.0 or higher and we are running 3.16.

So, my proposed plan is:

  • Upgrade to linux kernel 4.9 (from jessie backports) this friday night
  • Re-configure docker to use overlay2

And go from there.

comment:11 Changed 16 months ago by https://id.mayfirst.org/jamie

We're running the new kernel - and docker is happily using the overlay2 storage engine and I rebuilt all images and... still crashing.

comment:12 Changed 16 months ago by https://id.mayfirst.org/jamie

Seems to be working at last.

I finally got it to work by downgrading docker from the last version from the docker apt repository (17.06.0~ce-0~debian) to the ancient version of jessie-backports (1.6.2~dfsg1-1~bpo8+1).

It's using the devicemapper storage driver with loop back - which is the worst performing combo - but fortunately, it shouldn't be doing much writing.

comment:13 Changed 16 months ago by https://id.mayfirst.org/jamie

Now trying the debian version in unstable... 1.13.1~ds1-2

comment:14 Changed 16 months ago by https://id.mayfirst.org/jamie

  • Resolution set to fixed
  • Status changed from assigned to closed

Not going to work with debian jessie (maybe stretch). Reverting back to old version of docker. Letting it run in terminal... will take an hour or so to build the images...

We should revisit how we run docker when we upgrade to stretch.

For now, however, I will consider this upgrade complete (or at least in an hour when this rebuild finishes).

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.