Infrastructure Report

Hardware

We currently have over 20 phsyical server (over 90 virtual servers) in 6 locations.

Our two main hosting facilities continue to be Telehouse (25 Broadway, NYC) and XO/Highwinds (Bandcon was purchased by Highwinds) at 111 8th Ave, NYC.

Telehouse is close to capacity, whereas Highwinds is at about 50% capacity, meaning we have plenty of room to grow.

At the previous Leadership Committee I had mentioned the idea of combining these two facilities to save money, however, given our current financial situation, I think we should try to continue with both facilities for the time being. Given the room to expand at XO/Highwinds, I don't expect us to need to purchase more space over the next year.

Our hardware strategy continues to be retiring servers over 3 - 4 years. This approach is wasteful from the perspective of hardware, but dramatically cuts down our electricity costs and resources by using the most energy efficient servers available.

Support Team

The most important infrastructure development of the year has been the support team. In September we held our first ever face-to-face meeting of techies who are interested in supporting MFPL. Since then, we've built a team of 12 hard-working and dedicated activist techies. We've had a total of 4 face-to-face meetings and in the process made enormous progress in spreading the knowledge and skills needed to maintain the servers throughout the team.

Challenges

With a solid bank account and the best support team in our history, we have a lot going for us in 2011. However, there are some significant challenges facing us as well.

• Operating system upgrades: This month, the latest Debian release become stable, an event that happens every 2 years or so. Debian is the operating system running on all of our servers. When we were running a dozen servers or so, a new Debian release was not a major issue. Now, with 92 servers, it has become quite a project to upgrade all of them.

• Drupal upgrades: last fall, the newest version of Drupal (Drupal 7) was released. That means that Drupal 5 is no longer supported. We have over 100 members running web sites in Drupal 5. While upgrading Drupal is widely known among our membership as being a member responsibility, it's also not regularly done since it's complicated and time consuming. However, since Drupal 5 is no longer supported, security patches are no longer being released, which means these sites make our servers more vulnerable.

• Resource intensive web sites: Over the last several years, the resources needed to run web sites has gone up. Only a few years ago, typical web sites used very little server resources. Now, as Drupal has become more popular and more complex, even relatively simple sites with relatively few visitors can have a tremendous impact on our servers. As a result, we've experienced more slow downs on our shared servers due to a single web site being overloaded then in previous years.

• Downtime: In addition to slow downs, over the last year we've had some very critical periods of downtime. Most outages have involved obscure technical snafus that we've addressed. However, just one member with one experience of downtime or a slow down can be unbearable for the member and deadly for our reputation. The same member with more than one such experience creates a situation that can be extremely difficult to overcome. The planned projects for 2011 involve major changes to our infrastructure to transition us from technology designed with a handful of servers and one system administrator to technology designed to scale and be managed by a team. As we move forward with this transition, we may encounter a few more bumps in the road. Maintaining good communication and trust with our members will be a critical challenge through this period.

Projects for 2011

As we continue to develop the support team, we are focusing on the following major projects:

• Puppet. Puppet is a system for managing multiple servers as a group rather than individually. We've been experimenting with puppet for over a year and a half. It should greatly assist us as we do the Debian operating system upgrades by ensuring that all servers have the same standard configurations. It will also allow us to more quickly deploy new servers.

• Standard server split. I've written a proposal to the support team to conduct a significant change in our infrastructure to address the challenges of the resource intensive web sites. It involves splitting up the mail and web functions, allowing us to tune our servers better for each task. It will also make it much easier to re-located web sites between servers

• Drupal upgrade project. When Drupal 6 was released, Alfredo, in his role as a consultant, contacted most of the people running Drupal 4.7 and arranged to upgrade their sites for a fee. We will need to do something similar now - but with a bigger team.

Leadership

The biggest advancement in our organizational leadership has been the development of the support team. It's the most significant change in day-to-day decision making that we've had since our merger in 2005.

In addition, as reported in the finance report, are proposing the addition of our first paid staff person, which will have another impact on the day-to-day decision making of our organization.

Given these changes, and the fact that the LC has now been in existence for over three years, we would like to propose that the leadership committee take a more active role in several areas of the organization:

• Expansion sub-committee: up to now, new leadership committee members have been selected by Alfredo and Jamie and approved by the leadership committee. Moving forward, we'd like to propose a committee to evaluate this process and make recommendations on how we can move toward a more democratic process of leadership that involves our membership more. This process will be a long term process, but one that can be started now, even with small steps.

• Recruitment sub-committee: help organize and develop the 5 organizations per LC member outreach drive mentioned in Alfredo's main report.

• Program sub-committee: consult with the co-directors on program work and decisions, including setting our travel and conference agenda and focus