Changes between Initial Version and Version 1 of red-mosh-reorganization

Jul 6, 2018, 9:18:21 PM (3 years ago)
Jamie McClelland



  • red-mosh-reorganization

    v1 v1  
     1= Red/Mosh Re-organization =
     3The red control panel and filesystem layout of our MOSH servers limits us in several critical ways.
     5This proposal for re-organization has the following goals:
     7 * Prepare MOSH'es to handle a network file system as an intermediate step in [wiki:container-infrastructure our container infrastructure project]
     8 * Make it easier to move user accounts and web sites from one membership to another
     9 * Eliminate the confusing "hosting order" - so user accounts, DNS records, email lists, etc are linked directly to a membership, rather than being part of a hosting order collection
     11== Phase one: File system re-organization ==
     13The first step is to change the file system layout on MOSH'es.
     15Currently, user home directories are stored in `/home/members/<member-short-name>/sites/<hosting-order-identifier>/users/<username>` and web sites are stored in `/home/members/<member-short-name>/sites/<hosting-order-identifier>/{web,include,logs}`.
     17Since the membership name is part of the home directory and web paths, moving a web site or username to a different membership requires a tedious and error prone process of updating the user account home directory, web site DirectoryRoot, etc.
     19In addition, web sites and user paths are grouped together, making it harder to separate user accounts used for email from those used to secure FTP to update a web site.
     21=== New Layout ===
     23The proposed new layout would be:
     25 * Home directories: `/home/members/users/<username>`
     26 * Web directories: `/home/members/sites/<hosting-order-id>/{web,include,logs,conf/apache,conf/php}`
     28A web site's apache configuration would live in conf/apache and `/etc/apache2/sites-enabled/<hosting-order-id>.conf` would be a symlink to it. A web site's php pool configuration file would live in conf/php and `/etc/php/7.0/fpm/pool.d/<hosting-order-id>.conf` would be a symlink to it.
     30=== Transition ===
     32Since all MOSH servers have /home as a single partition, moving web site and user directories around should take minimal disk i/o.
     34As part of the transition, we could both:
     36 * perform a sed replacement for any instance of the old path with the new path (so php configuration files are properly updated)
     37 * leave a symlink from the old location to the new location in case anything was missed
     39=== Goals ===
     41Once completed, this step would achive the goal of making it easier to move users and web sites between memberships since the only record of what membership a user or web site belongs to would be in the control panel database.
     43== Phase two: eliminate the hosting order ==
     45The hosting order would be renamed "web site" (and the hosting-order-id in the path above would simply be renamed web-site-id in the database). We would still keep the following items grouped together:
     47 * web configuration
     48 * web application
     49 * mysql database
     50 * mysql username
     51 * schedule job (cron job)
     53However, the following items would be moved up to the member level:
     55 * Email address
     56 * Email list
     57 * High volume email list
     58 * DNS
     60We would eliminate Hosting order access - we would no longer support access control based on a web site.
     62=== Goals ===
     64With this step done, we would no longer have hosting orders
     66== Phase three: network file system ==
     68Once we have a network file system available, the various network drives would be mounted in /media, e.g. `/media/fs/<id>/{users,sites,databases}`.
     70We could transition each MOSH on a case-by-case basis by simply moving all data from /home/members to their respective locations on the network file system and then replacing the directories in /home/members with symlinks to /media/fs.
     72Depending on the network file system chosen, we could store MySQL databases and maintain symlinks in /var/lib/mysql.
     74== Phase four: central authentication and authorization ==
     76The last phase before deploying containers is to replace /etc/passwd, /etc/shadow, and /etc/group with a central authentication/authorization system (NIS with ldap or mysql).
     78With this step in place, every MOSH would have every login available on it.
     80Now, moving a web site between MOSH'es would require:
     82 * Creating symlinks on the target MOSH (home directories, web config, php config, databases) and reloading necessary services. Now everything is available on both servers at the same time. NOTE: the web files can live on both servers at the same time, but the database has to be removed from one server before it is started on the other to avoid data corruption.
     83 * Update DNS (or preferably the internal routing mechanism so it can be instant).
     84 * Remove symlinks from old MOSH
     86== Phase five: containers and beyond ==
     88Once these pieces are in place, switching to a container based system is relatively simple as user and web directories can easily be mounted inside a container.