Opened 11 years ago

Closed 11 years ago

#1013 closed Bug/Something is broken (fixed)

Move workersliberty to new server

Reported by: Jamie McClelland Owned by: Jamie McClelland
Priority: Urgent Component: Tech
Keywords: Cc:
Sensitive: no

Description

Due to excessive loads on malcolm, we will be moving workersliberty.org to a new server (that will be setup and called albizu).

The target timeline will be this weekend (May 24/25).

Change History (23)

comment:1 Changed 11 years ago by ekes

I should be online this weekend in case there is anything that needs help with. So no probs there.

comment:2 Changed 11 years ago by Jamie McClelland

Great - let's shoot for saturday 11:00 am Americas/New_York - 3:00 pm UTC.

I'll be on the mayfirst channel (/join mayfirst) on our silc server (silc.mayfirst.org).

One question: how are email accounts handled by workersliberty.org? People will need to be notified that their primary server is changing from malcolm.mayfirst.org to albizu.mayfirst.org (either in their desktop applications, or when they choose their mail server when logging in via webmail). I want to be sure they have enough advance warning.

comment:3 Changed 11 years ago by Martin Thomas

I'll contact all the people with @workersliberty.org e-addresses - Martin

comment:4 Changed 11 years ago by Martin Thomas

Will people using Imap email have the same messages on albizu after the switch as on malcolm before the switch? I suppose so, but can you confirm? Thanks - Martin

comment:5 Changed 11 years ago by Jamie McClelland

Yes - your imap messages will be preserved. Here's my plan for the switch:

  • Today:
    • Re-create account on new server. I will run the transfer script. This script:
      • re-creates all of your account information on the new host (albizu). It retains the settings on your current host, so nothing has changed yet in the way your site and email works.
      • Reduces the "time to live" for your domain name records from 1 day to 5 minutes. This tells the world: starting now, please check back every 5 minutes to see if our IP address has changed. This way, when we do the switchover tomorrow, the new server switch can be propagated to all servers on the Internet relatively quickly.
      • Finally - the script does put your current settings in "transfer-limbo" - which means you will be unable to change a password, add a new email account, etc. until the transfer is complete tomorrow.
    • Copy data to new server. I will also run the copy data script, which will copy all of your info to the new host (albizu). At this point, you can login to either the new or the old host (for testing purposes). No new email will be sent to the new host - all email will still go to your existing host.
  • Tomorrow: I will change your DNS settings to the new IP address of albizu and then re-run the copy data to new server script so you have the most recent data on your live server.

Let me know if that works for you and if you have any other questions or suggestions for making it a smoother transition.

Thanks!!

Jamie

comment:6 Changed 11 years ago by Jamie McClelland

Uh oh - problems.

I did the transfer successfully (including changing the time to live to 5 minutes - and the data is currently being dumped.

However, I just realized that workersliberty.org is not being handled by May First/People Link's domain name servers :(. So - my turning down the time to live has no effect.

The domain name seems to be handled by ns0.serve.co.uk. Do you have access to the domain name settings on that server? Can you turn down the time to live to 5 minutes (300 seconds)?

comment:7 Changed 11 years ago by Martin Thomas

Nick, who has access to the domain name settings, is off on a camping holiday at present. He texts me: "Can't turn down ttl. But can alter dns if someone sends me details". What details should I send him?

comment:8 Changed 11 years ago by Martin Thomas

Jamie, would you email the dns changes required to Nick at nick@…. He can get email on a Blackberry, but has been unable to log in to mayfirst. Thanks - Martin

comment:9 in reply to:  8 Changed 11 years ago by Jamie McClelland

Replying to https://id.mayfirst.org/workersliberty:

Jamie, would you email the dns changes required to Nick

Thanks Martin - I just emailed him and will await his response.

comment:10 Changed 11 years ago by Jamie McClelland

The DNS change has been made. I'm not doing a final synch of all data (the web site may be a bit wonky over the next 30 minutes or so (it's a large db dump)).

comment:11 Changed 11 years ago by Jamie McClelland

Resolution: fixed
Status: newclosed

We're done! Thanks for your support and help.

I'm going to close this ticket since that transfer has been made - but please feel free to open a new ticket if you encounter any problems with the new setup.

A couple other house cleaning notes:

I made a complete backup of your data on malcolm (including a fresh database dump) and saved it here:

malcolm.mayfirst.org:/home/members/workersliberty/sites/workersliberty.org.delete.after.2008.06.24.tar.gz

If we somehow missed any data in the transfer, we can get it back. We'll delete that file in a month.

And - we still haven't been issued a corporate-signed SSL certificate for albizu.mayfirst.org. That means that if you use a program like Thunderbird to check your email, you may get warnings saying that the certificate authority that signed the certificate is not trusted. That's pretty funny. Once we have the certificate signed by a capitalist authority, it will be "trusted" - however, since it's signed by a radical technology nonprofit it's not trusted. Sigh. If you want those messages to go away, you can enable your software to trust MFPL signed certificates. If you want to follow our progress on getting a corporate signed certificate, you can subscribe to ticket #999.

And for the record - once the transfer was complete, albizu proceeded to run out of memory and crash. I just doubled the memory (1 GB to 2 GB) and now it seems perfectly content.

comment:12 Changed 11 years ago by Martin Thomas

Resolution: fixed
Status: closedreopened
Type: Task/To do itemBug/Something is broken

Three problems with my email (IMAP, using Thunderbird) after the change.

(1) When I send myself a test message, it isn't delivered back to me. Looks like no incoming email at all is being delivered to me (see (2) below).

(2) At the point of sending, I get an error message "There was an error copying the message to the Sent folder". (There is also a message saying that the email has been successfully sent. That's true; I've checked by sending to another address. So I guess the test message I've sent to myself on Mayfirst really has been successfully sent; looks like problem (1) is a matter of *no* incoming email being delivered to me).

(3) When I try to use webmail, the log-in fails (the error message is just "Login failed", no further detail).

comment:13 Changed 11 years ago by Martin Thomas

Another problem: when I tried to look at www.workersliberty.org I got "Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 30720 bytes) in /home/members/workersliberty/sites/workersliberty.org/web/sites/default/modules/site_map/site_map.module on line 338".

comment:14 Changed 11 years ago by Martin Thomas

Yet another symptom (reflecting the same basic problem, I guess). Tried to move an (already received) message from Spam to Trash, got "Unable to connect to your IMAP server. You may have exceeded the maximum number of connections to this server..."

comment:15 Changed 11 years ago by Jamie McClelland

Hi Martin - thanks for the feedback.

Do you still have malcolm.mayfirst.org as your incoming and outgoing mail server in your Thunderbird configuration? If so - those should both be changed to albizu.mayfirst.org. I checked your mailbox on albizu - and you have (as of now) over 100 new messages.

comment:16 in reply to:  13 Changed 11 years ago by Jamie McClelland

Replying to https://id.mayfirst.org/workersliberty:

Another problem: when I tried to look at www.workersliberty.org I got "Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 30720 bytes) in /home/members/workersliberty/sites/workersliberty.org/web/sites/default/modules/site_map/site_map.module on line 338".

I just increased your allocated memory (using [wiki:change_php_ini_settings these directions). Let me know if you still get that out of memory error.

Jamie

comment:17 Changed 11 years ago by Martin Thomas

Thanks: www.workersliberty.org is back up. But I still have exactly the same problems with email.

Yes, I have changed the server name to albizu.

I can send (tho' not copy messages to the Sent folder); I can't receive any mail; I can't log into webmail.

100 new messages sounds about right - and I'm getting a bit anxious about getting to them!

comment:18 Changed 11 years ago by Jamie McClelland

Hi Martin - I'm really sorry for leaving you hanging this long!

I just checked and found a problem - I didn't properly install the new security certificate (our IMAP server requires both the key and certificate in the same file). I just fixed it - can you try it again?

comment:19 Changed 11 years ago by Jamie McClelland

I just created a test account and tested the login and it seems to be working. Now - apologies for the stress!

comment:20 Changed 11 years ago by Martin Thomas

Resolution: fixed
Status: reopenedclosed

Yes, IMAP now fixed. I can log in to email and receive messages. Thanks - Martin

comment:21 Changed 11 years ago by Martin Thomas

Resolution: fixed
Status: closedreopened

Now another email problem. (Connected? I don't know). I seem to be *receiving* email all right, but now when I try to *send* email to (as far as I can see) any address, I get the error message: "An error occurred while sending mail. The mailserver responded: 5.7.1 <addressee e-address>: Relay access denied. Please check the message recipients and try again".

I'm using Thunderbird on IMAP.

comment:22 Changed 11 years ago by Martin Thomas

Further to the above: I can still log in to webmail and send from there.

comment:23 Changed 11 years ago by Martin Thomas

Resolution: fixed
Status: reopenedclosed

Oops. Sorry. It was a mistake in my Thunderbird set-up which had crept in somehow. Not a Mayfirst problem.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.