Understanding Colo Issues

This wiki page is being written to help explain the concepts being discussed in ticket #1098


There are hundreds of networks to choose from. Networks provide the bandwidth and (in most cases) an IP range to use.

Types of Networks

Tier one networks (see Wikipedia's article) are networks that connect to the rest of the Internet soley through "Settlement Free Interconnection" (meaning that they can directly connect to the other Tier one networks - called "peering," not relying on a intermediary provider). Wikipedia lists some interesting reasons why these carriers are less reliable because they often only have one route to each other network. Examples of Tier One networks are Global Crossing, Level Three, and AT&T.

Tier two networks (see Wikipedia's article) rely on purchasing transit from one or more Tier one networks.

In addition, I've heard people refer to carrier's like Hurricane Electric as "blending" connectivity from many other providers.

In addition, there are public exchanges like InterNAP, PAIX, and Mizma. Getting an exchange connect provides multi-homed peer arrangements, vs. single homed ones like via one provider (say Level 3, UUnet, etc., but I understand that even depending on an exchange connect is risky)... I'm not exactly sure how this works. InterNAP is expensive though, and I've read that mixing our own combo of Level3, Savvis and Peer1 can get the same performance for a fraction of the cost of Internap.

Network pricing

Typically pricing is in Mbits/per second. That literally means that over a 5 minute period your throughput is measured Mbits and divided by 300 seconds. This happens every 5 minutes for a month. At the end of the month, the highest 5% of 5 minute increments are discarded and the remaining ones are averaged, which produces a 95 percentile.

Usually you are charged dollars/Mbit with a certain "commit" and a certain allowed burstable amount.

The commit means you will pay for that many Mbits regardless of your actual usage. The burstable refers to the highest Mbit/second you can reach.

For example, we are currently charged by Hurricane Electric $50/Mbit, with a 10 Mbit commit and our line is burstable to 100 Mbits. That means we always pay at least $500/month (for the 10Mbit commit). We pay $50/Mbit if we go above 10 Mbit. We cannot transfer more than 100Mbit/second.

Connecting to a network

Unless we become a Tier One network provider (not realistic), we are going to purchase bandwidth from someone.

The simple approach:

Some colocation facilities offer their own bandwidth. That means they have figured out how to connect to one more more networks, they have their own routers setup to route the traffic, and you can just connect to their network and off you go. This approach is convenient and easy. It has the downside that if either the colocation facility or the provider goes out of business, we're totally screwed.

Two middle approaches:

  • Colocate in a carrier neutral facility (in which multiple network providers are present). This is the approach we are taking now in Telehouse with Hurricane Electric as our provider. We pay Telehouse a "cross connect" fee to connect our rack to Hurricane Electric.
  • Pay for a connection (called transport) between a carrier neutral facility to the facility we are in. Depending on the distance, this option can be very pricey.

In either case we have some independence from the colocation facility.

The extreme approach: We can negotiate directly with one or several networks or network exchanges, setup our own BGP routers, and operate as a Autonomous System (AS). This provides ultimate independence. It gives us the option to both move locations and re-negotiate with different providers while still keeping our IP range. It's kinda complicated and expensive though.

In addition, there are "meet-me" rooms which allow you to establish connections with other carriers (??).

Colocation Facilities

There seem to be three major physical locations in the NYC area where "peering" occurs - in other words, where the Tier 1 networks inter-connect. They are:

  • 25 Broadway (a facility run by Telehouse)
  • 60 Hudson
  • 111 8th Avenue

Who runs the other two locations?

Last modified 11 years ago Last modified on Jun 18, 2008, 10:53:35 AM