= Support Team Face to face 2012-11-03 = 10:00 (America/New_York) - Jamie's House [[TicketQuery(keywords=~f2f,format=table,col=resolution|summary|owner|reporter)]] Disaster (in)Tolerance discussion __Targets__ * https://support.mayfirst.org * offsite backups * service advisories * single point of sending/updating that is colo independent * multple upstream routes fail over options * bring back all servers without physical visit to the colo or data compromise __Solutions__ * multiple upstream routes * GSM (or other mobile) modem in each colo * second uplink crossconnect * direct interconnect between cabinets * SMO * db and config replication for SMO, with manual cutover and cutback scripts * read-only, non-login copy of SMO with modified template that makes it clear that trouble is a-brewin' * on-failure-display * backups * big beefy disk array servers * member CPE to provide other backup services * get mf/pl cable link at the lair * move sittingbull to the lair * UPS-backed low-power machine as console server, switch backed as well.