Changes between Version 4 and Version 5 of red-mosh-reorganization


Ignore:
Timestamp:
Jul 9, 2018, 10:02:28 AM (2 months ago)
Author:
https://id.mayfirst.org/jamie
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • red-mosh-reorganization

    v4 v5  
    1818To get all databases on SSD, we should allocate two virtual machines on each of our three physical hosts with SSD and allocate all remaining SSD space to these machines. These will become our MySQL servers for all members.
    1919
    20 We will setup an internal network (e.g. 10.0.0.0/24) so all database ports are shielded from direct access over the Internet.
     20We will setup an internal network (e.g. 10.12.53.0/24) so all database ports are shielded from direct access over the Internet.
    2121
    2222We will also setup several MySQL proxies (that will respond via round robin DNS to mysql.mayfirst.org). Each proxy will properly route the database request to the proper back end host, so members will all simply configure their web apps to use the host mysql.mayfirst.org.
     
    3030We could signficantly reduce this load by instead running an rsync backup to our onsite backup server, then running an incremental backup from the onsite backup server to the offsite server during the day (and perhaps switching from rdiff-backup to borg-backup to see if we get better performance (see [ticket/12677#comment:15]).
    3131
    32 == Phase one: File system re-organization ==
     32=== Goals ===
     33
     34Immediatley relieve disk i/o contention.
     35
     36== Phase two: File system re-organization ==
    3337
    3438The first step is to change the file system layout on MOSH'es.
     
    6266Once completed, this step would achive the goal of making it easier to move users and web sites between memberships since the only record of what membership a user or web site belongs to would be in the control panel database.
    6367
    64 == Phase two: eliminate the hosting order ==
     68== Phase three: eliminate the hosting order ==
    6569
    6670The hosting order would be renamed "web site" (and the hosting-order-id in the path above would simply be renamed web-site-id in the database). We would still keep the following items grouped together:
     
    8589With this step done, we would no longer have hosting orders
    8690
    87 == Phase three: network file system ==
    88 
    89 Once we have a network file system available, the various network drives would be mounted in /media, e.g. `/media/fs/<id>/{users,sites,databases}`.
    90 
    91 We could transition each MOSH on a case-by-case basis by simply moving all data from /home/members to their respective locations on the network file system and then replacing the directories in /home/members with symlinks to /media/fs.
    92 
    93 Depending on the network file system chosen, we could store MySQL databases and maintain symlinks in /var/lib/mysql.
    94 
    9591== Phase four: central authentication and authorization ==
    9692
     
    105101 * Remove symlinks from old MOSH
    106102
    107 == Phase five: containers and beyond ==
     103== Phase five: network file system ==
     104
     105Once we have a network file system available, the various network drives would be mounted in /media, e.g. `/media/fs/<id>/{users,sites,databases}`.
     106
     107We could transition each MOSH on a case-by-case basis by simply moving all data from /home/members to their respective locations on the network file system and then replacing the directories in /home/members with symlinks to /media/fs.
     108
     109Depending on the network file system chosen, we could store MySQL databases and maintain symlinks in /var/lib/mysql.
     110
     111== Phase six: containers and beyond ==
    108112
    109113Once these pieces are in place, switching to a container based system is relatively simple as user and web directories can easily be mounted inside a container.