Changes between Version 1 and Version 2 of standard-server-split


Ignore:
Timestamp:
Jun 10, 2011, 3:27:08 PM (13 years ago)
Author:
Jamie McClelland
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • standard-server-split

    v1 v2  
    1 = Standard Server Split =
     1= MOSH Server Split =
    22
    33== Overview ==
    44
    5 Over the last several years, many May First/People Link members who are hosted on our shared, standard servers, have begun running web sites that require more resources than we are able to easily provide or that we are able to predict in advance. As a result, we have frequent periods in which any given standard server runs slowly or even crashes.
     5Over the last several years, many May First/People Link members who are hosted on our shared, [wiki:mosh MOSH] servers, have begun running web sites that require more resources than we are able to easily provide or that we are able to predict in advance. As a result, we have frequent periods in which any given standard server runs slowly or even crashes.
    66
    77There are three problems with our infrastructure that hamper our ability to address these situations:
    88
    9  * While a series of bash scripts has eased the deployment of new standard servers, the process is still cumbersome, taking several hours.
    10  * Our standard servers provide both web and mail services, preventing us from optimizing servers for one or the other task
     9 * Our MOSH servers provide both web and mail services, preventing us from optimizing servers for one or the other task
    1110 * The mail services require members to customize their desktop email clients based on the particular standard server they are using (e.g. chavez.mayfirst.org or viewsic.mayfirst.org). Therefore, moving mail services from one server to another to re-distribute load requires a level of member coordination that makes it infeasible in most cases
    1211
    … …  
    1514The following steps are a proposal for addressing these problems:
    1615
    17  * Complete the transition to Puppet to ease the work involved with deploying new standard servers
    1816 * Abstract the use of all mail configuration so that all members use the same settings
    1917 * Move all mail services to dedicated mail servers, leaving current standard servers as dedicated web / mysql / shell servers
    20 
    21 The middle task is the most complicated, so I will explain more here.
    2218
    2319There are three mail settings in use by MFPL that require a server's domain name:
    … …  
    3127 * relay.mayfirst.org: Currently, we have authenticated mail relay on each of our standard servers. If we setup one or several servers that had a login for every MFPL user, we could easily transition to the use of a single domain name for all mail relaying.
    3228
    33  * mx.mayfirst.org: We could setup one or several mx servers with minimal postfix customizations, that could accept ''all'' mail intended for MFPL and then route mail to the particular server that houses the account. By keeping a list of all valid email accounts on all mx servers, we would also be able to reject mail for unknown mailboxes and we even deal with viruses, etc. before email reaches the server housing the actual mailbox
     29 * incoming.mayfirst.org: Using [http://www.debianadmin.com/howto-install-nginx-webserver-in-debian.html nginx] or [http://www.debianhelp.co.uk/perdition.htm perdition], we could setup one or several IMAP/POP proxies that similarly redirect requests to the appropriate server based on a database / table lookup.
    3430
    35  * incoming.mayfirst.org: Using [http://www.debianadmin.com/howto-install-nginx-webserver-in-debian.html nginx] or [http://www.debianhelp.co.uk/perdition.htm perdition], we could setup one or several IMAP/POP proxies that similarly redirect requests to the appropriate server based on a database / table lookup.
     31 * mx records should stay configured to the individual server. This way, our DNS records can provide a canonical way to identify the ''real'' mail server for any given domain.
    3632
    3733== Implementation ==
    … …  
    3935The transition would need to be implemented so that there is a long overlap period whereby the current approach to mail settings work as do the new approach.
    4036
    41 All three changes would require changes to the Members Control panel, so that the creation of an email account triggers action not only on the user's primary server (to preserve backward compatibility), but also on the newly added servers.
    42 
    4337Implementing relay.mayfirst.org is the easiest of the three, since we can use the same software and configuration we are currently using - with the exception of implementing it on a dedicated server. Once in place and tested, we can begin using it immediately. It will, however, require users to change their own desktop email settings.
    44 
    45 mx.mayfirst.org requires us to explore a new postfix configuration. However, again, not significantly complicated or new. Furthermore, once this change is in place, we can update DNS entries without relying on users to make the change.
    4638
    4739incoming.mayfirst.org requires us to explore new software, making it the most complex step. Furthermore, it will also require users to change their local settings.
    4840
    49 Probably best to make all three changes first, and then request that users update their settings. Through our mail logs we can accurately assess exactly who has made the changes and who hasn't.
     41Probably best to make all changes first, and then request that users update their settings. Through our mail logs we can accurately assess exactly who has made the changes and who hasn't.
    5042
    5143Only after everyone has made the changes can we really put this new setup to use by moving users around and splitting off our standard servers into web and mail servers.