MOSH Server Split

Overview

Over the last several years, many May First/People Link members who are hosted on our shared, MOSH servers, have begun running web sites that require more resources than we are able to easily provide or that we are able to predict in advance. As a result, we have frequent periods in which any given standard server runs slowly or even crashes.

There are three problems with our infrastructure that hamper our ability to address these situations:

• Our MOSH servers provide both web and mail services, preventing us from optimizing servers for one or the other task
• The mail services require members to customize their desktop email clients based on the particular standard server they are using (e.g. chavez.mayfirst.org or viewsic.mayfirst.org). Therefore, moving mail services from one server to another to re-distribute load requires a level of member coordination that makes it infeasible in most cases

Proposals

The following steps are a proposal for addressing these problems:

• Abstract the use of all mail configuration so that all members use the same settings
• Move all mail services to dedicated mail servers, leaving current standard servers as dedicated web / mysql / shell servers

There are three mail settings in use by MFPL that require a server's domain name:

• Relay: all members who relay mail through May First/People Link must specify their primary server in their desktop email client settings (the "outgoing" server)
• MX records: all member domains that use MFPL for email, have a DNS MX record set to their primary server.
• POP/IMAP: all members who access their email via POP or IMAP must specify either in the desktop client or when they visit ​https://members.mayfirst.org/ to check their email via the web the name of their primary server

Rather than forcing our users to specify their primary servers for these settings, each one should have a organization-wide address used by everyone.

• relay.mayfirst.org: Currently, we have authenticated mail relay on each of our standard servers. If we setup one or several servers that had a login for every MFPL user, we could easily transition to the use of a single domain name for all mail relaying.

• incoming.mayfirst.org: Using ​nginx or ​perdition, we could setup one or several IMAP/POP proxies that similarly redirect requests to the appropriate server based on a database / table lookup.

• mx records should stay configured to the individual server. This way, our DNS records can provide a canonical way to identify the real mail server for any given domain.

Implementation

The transition would need to be implemented so that there is a long overlap period whereby the current approach to mail settings work as do the new approach.

Implementing relay.mayfirst.org is the easiest of the three, since we can use the same software and configuration we are currently using - with the exception of implementing it on a dedicated server. Once in place and tested, we can begin using it immediately. It will, however, require users to change their own desktop email settings.

incoming.mayfirst.org requires us to explore new software, making it the most complex step. Furthermore, it will also require users to change their local settings.

Probably best to make all changes first, and then request that users update their settings. Through our mail logs we can accurately assess exactly who has made the changes and who hasn't.

Only after everyone has made the changes can we really put this new setup to use by moving users around and splitting off our standard servers into web and mail servers.