wiki:transfer_site_between_servers

Version 17 (modified by Jamie McClelland, 4 years ago) (diff)

--

How do I move a hosting order from one server to another server?

This help page is intended for May First/People Link administrators. Root access to the servers in question is required.

To help distribute load evenly between our standard servers, it is sometimes necessary to move a hosting order from one server to another. These steps document a process for making a move that allows you to copy and test the move before making a final commitment.

Check the mail logs for user accounts not checking mail.mayfirst.org

In order to minimize email access breaking, you can check if any of EXAMPLE.ORG's email accounts are not checking mail.mayfirst.org. Replace the ALL CAPS portion of this command with the appropriate details.

for foo in $(grep EXAMPLE.ORG /etc/postfix/virtual_alias_maps | cut -f2 | grep -v @ | sort | uniq); do echo $foo; zgrep "imapd-ssl: LOGIN, user=$foo" /var/log/mail.log* | egrep -v "240.253.234.209|216.66.23.48" | wc -l; done | tee MEMBER-mail-logins.txt

Create the appropriate items in the Members Control panel

The script below creates a new hosting order pointing to the old server, migrate the old hosting order to point to the new server and creates duplicate copies of all the items from the old hosting order with the status of transfer-limbo on the new hosting order. The new hosting order, pointing to the old server, is suffixed with "(temp for host move)". The new hosting order is to be deleted once the migration is final.

The first step is to run the transfer-red-items-to-new-host script from hay.mayfirst.org:

  • Login as root:
    ssh root@hay.mayfirst.org
    
  • Become the www-data user (which has the ability to notify hosts of item changes)
    su - www-data
    
  • Execute the script which asks you to confirm whether you want to run this part of the migration or not. The confirmations allow you to skip certain steps in case you need to skip them if you run the script more than once. Below is the transcript of the stesp you'll take with itemized descriptions at each point.
    0 hay:~$ cd /usr/local/share/red/ui/sbin/ 
    0 hay:~$ ./transfer-red-items-to-new-host 
    
  • Enter the domain of the hosting order being moved and you'll see information about the hosting order including the current server.
    Please enter the domain name of the hosting order to move? apen.ourpowerbase.net
    Hosting Order ID: 1000117
    Member: Progressive Tech
    Current host: julia.mayfirst.org
    
  • Next enter the fully qualified domain name of the new host
    Please enter the full domain name of the new host. ella.mayfirst.org
    
  • Turn the domain TTL down for the final migration.
    Crank down time to live for DNS to 300 seconds? [Yn] y
    Cranking down time to live for DNS to 300 seconds
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for apen.ourpowerbase.net
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for www.apen.ourpowerbase.net
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for mail.apen.ourpowerbase.net
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for ibew25.mayfirst.org
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for apen.ourpowerbase.net
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    DNS record updated for advantage.apen.ourpowerbase.net
    
  • Create the new hosting order that points to the old server. The nomenclature of the script is confusing, but the new hosting order is actually a copy of the old one and will be deleted. The new hosting order points to the old server. The old hosting order is updated to point to the new server.
    Create new hosting order? [Yn] 
    Creating new hosting order
    New hosting order created (1000145)
    Using temp hosting order id: 1000145
    
  • Move all the existing red items to the new hosting order (old server) and change their status to transfer-limbo.
    Transfer relevant red items to new host (1000145)? [Yn] 
    Moving red_items to newly created hosting order and setting as transfer-limbo
    Done
    
  • Change the old hosting order (new server) with updated server name.
    Update old hosting order with new host value? [Yn] 
    Updating hosting order with new host (ella.mayfirst.org)
    
  • Duplicate the red items from new hosting order (old server) to the old hosting order (new server)
    Create new red_items on new host? [Yn] 
    Making duplicate red_items on new host
    
  • Validating the new red items will likely produce a lot of errors, particularly email addresses. When running just validation, you will get errors about how certain items did not validate because the related user account doesn't exist. That's normal because when validating, you don't actually create the user account. This output could prove unhelpful depending on the size of the hosting order. Regardless the old hosting order (new server) should be manually checked to make sure that user accounts, email addresses, databases, DNS records, web configuration, etc. don't show any errors.
    Would you like to validate new red items? [Yn] 
    Working on item: 23103 (User Account)
    validate'd
    Working on item: 23104 (Server Access)
    You can only specify a user account that exists.
    Errors on validation: 
    Working on item: 23105 (Web Configuration)
    You must specify a user account that exists. You entered: opbapen.
    Errors on validation: 
    Working on item: 23301 (Email Address)
    You must either specify a valid email address or a user account that exists. You entered: opbapen.
    Errors on validation: 
    Working on item: 23384 (Web Configuration)
    You must specify a user account that exists. You entered: opbapen.
    Errors on validation: 
    
  • Commit the changes to the old hosting order (new server).
    Would you like to commit new red items? [Yn] 
    Working on item: 23103 (User Account)
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    commit'd: 23822
    Working on item: 23104 (Server Access)
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    commit'd: 23823
    Working on item: 23105 (Web Configuration)
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    Warning: DocumentRoot [/home/members/progressivetech/sites/apen.ourpowerbase.net/include/nonhttps] does not exist
    commit'd: 23824
    Working on item: 23301 (Email Address)
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    commit'd: 23825
    Working on item: 23384 (Web Configuration)
    Pseudo-terminal will not be allocated because stdin is not a terminal.
    commit'd: 23826
    0 hay:/usr/local/share/red/ui/sbin$
    

When you are done, there will be two hosting orders - one that is in transfer-limbo (new hosting order; old server) and one that is active (old hosting order, new server). You can only edit the latter. Editing the former will result in a validation error because any changes made there won't be preserved. The new and old hosting orders are exactly the same, but immediately after the script has completed, the transfer-limbo hosting order is current. The only thing that makes the transfer-limbo hosting order current is DNS. Once the DNS is changed to the active hosting order, all email, web traffic, etc. will go to the new server and that will become the current one.

Before changing the DNS we need to copy the file system data.

Copy data from the current to the new host

Next, copy the data from the current host to the new host.

Start by logging into the new host and granting root access to the current host (you will delete this access when you are done).

ssh root@NEW.MOSH

Edit /root/.monkeysphere/authorized_user_ids, add:

root@CURRENT.MOSH

Run:

monkeysphere-authentication update-users root

Now, log into the current host as root:

ssh -A root@CURRENT.MOSH

Ensure you can access the new mosh:

ssh root@NEW.MOSH

Then, from the CURRENT.MOSH, run the /usr/local/share/red/node/sbin/copy-data-to-new-host script, passing the new host as the first argument and the path to the hosting order as the second. For example:

/usr/local/share/red/node/sbin/copy-data-to-new-host NEW.MOSH /home/members/mayfirst/sites/mayfirst.org

This script will rsync all the data, preserving permissions and ownership, to the new host. Then, it will search for Drupal, Wordpress or CiviCRM databases, and if it finds one, creates it (and the associated usernames) to the new host and then dumps the data. Finally, it will search for crontabs on the current host, re-creating any it finds on the new host.

You can safely run this script as many times as you like.

Note: the script does not detect databases other than those used by Drupal. If the hosting order is using a non-Drupal database you will need to transfer it by hand.

Test

The next step is to test. You can test by modifying your hosts file so that the domain name used by the site your are moving resolves to the IP address of the new host.

Switch

If all goes well with the test, you can finalize the move by modifying the DNS records to point the domain name(s) to the IP of the new host. While you are changing the IP records, be sure to change the time to live back to a reasonable number.

Clean up

Once the transfer has happened, and the site is confirmed to be working and completely intact, you will need to remove the transfer-limbo hosting order:

  • From the control panel, find the control panel in the state "transfer-limbo". Delete it.

NOTE: It's critical that you delete the transfer-limbo site immediately. As long as that site exists, our imap proxy server will use it for directing IMAP logins (see #7178 for more details).

Moving from FCGID server to mod_php

If you receive errors when updating hosting order web configurations, you may need to take an additional step and modify the /usr/local/etc/red/red_node.conf file and remove the line:

$red_backends['red_item_web_conf'] = 'apache2_suexec_fcgid';

And add this line:

$red_backends['red_item_web_conf'] = 'apache2';

Otherwise, you will not be able to update apache config files.