| | 1 | == Intro == |
| | 2 | |
| | 3 | By many estimates (see [http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.html Cisco White Paper] and [http://www.nytimes.com/2008/03/13/technology/13net.html?_r=2&pagewanted=1&hp NY Times article]), video traffic on the Internet will continue to increase over the next few years, becoming a dominant form of content and driving up bandwidth usage every where. |
| | 4 | |
| | 5 | Corporations are both fueling and controlling this process by providing unsustainable and capital intensive ad-supported services, like YouTube, that have unacceptable content ownership policies, support the use of proprietary video formats, and are politically unreliable to trust with sensitive content. |
| | 6 | |
| | 7 | In short, the world needs a better way to host and delivery multi-media content. |
| | 8 | |
| | 9 | == Global Cache Nework == |
| | 10 | |
| | 11 | The Global Cache Network is a proposal for creating such a system. The proposal has been developed by May First/People Link based on [wiki:internet-video-distribution-paper meetings and discussions with our members] who work heavily with multi media content. |
| | 12 | |
| | 13 | The Global Cache Network would provide: |
| | 14 | |
| | 15 | * Servers for uploading and transcoding multi-media content from widely used proprietary and non-proprietary formats to free formats. |
| | 16 | |
| | 17 | * Servers for storing and delivering the media content |
| | 18 | |
| | 19 | * Servers designed to cache the content, providing fast and reliable access. |
| | 20 | |
| | 21 | == Principles and Content Policy == |
| | 22 | |
| | 23 | The Global Cache Network would be based on the following principles: |
| | 24 | |
| | 25 | * Distributed: Nearly all aspects and servers used in the network will be distributed. Anyone who demonstrates the ability to technically install the appropriate software on a server can contribute a server to the network, thus allowing it to grow infinitely. The only centralized aspect of the network will the domain name system, which will control which servers are authorized to be in the network (and will allow for the removal of servers that are unreliable or proven untrustworthy). |
| | 26 | |
| | 27 | * Anonymous: all submissions are anonymous. Authors are free to link to uploaded content from their own sites and assert ownership, however, no ownership is internally linked to any content in the network. By extension of anonymous uploads, no user-initiated edits or deletes are allowed. Once content is uploaded, the uploader cannot request that it be removed or replaced (there will be no mechanism to know who has authority to authorize such edits or deletes). Content pruning will happen via automatic deletion of material that is deemed "minimally accessed" over a given period of time (pruned content can be re-uploaded). |
| | 28 | |
| | 29 | * Free to share and re-use: All content is licensed under the Creative Commons [http://creativecommons.org/licenses/by-nc-sa/3.0/us/ Attribution-Noncommercial-Share Alike 3.0]. |
| | 30 | |
| | 31 | Any content is allowed provided it meets the following Content policy: |
| | 32 | |
| | 33 | * No erotic content. |
| | 34 | |
| | 35 | * No content that advances an agenda of hate, discrimination or oppression |
| | 36 | |
| | 37 | * No commercials. Content that directly advances a for-profit venture is prohibited. |
| | 38 | |
| | 39 | * No initial uploads greater than 2 GB (subject to change) |
| | 40 | |
| | 41 | == Technical Overview == |
| | 42 | |
| | 43 | The technical details are based on a central domain name of gc.domain (obviously this will change). |
| | 44 | |
| | 45 | The network would be broken into several pieces: |
| | 46 | |
| | 47 | * Entry servers, providing upload and transcoding services (a.entry.gc.domain, b.entry.gc.domain, c.entry.gc.domain etc.) also available via round robin DNS as entry.gc.domain. |
| | 48 | |
| | 49 | * Caching servers (responding to a.store.gc.domain, b.store.gc.domain, c.store.gc.domain) and pulling from the corresponding primary storage servers (see below). |
| | 50 | |
| | 51 | * Primary storage servers (primary.a.store.gc.domain, primary.b.store.gc.domain, primary.c.store.gc.domain, etc.). |
| | 52 | |
| | 53 | * Backup storage servers (0.backup.a.store.gc.domain, 1.backup.a.store.gc.domain, 2.backup.a.store.gc.domain, 0.backup.b.store.gc.domain, 1.backup.b.store.gc.domain, 2.backup.b.store.gc.domain, etc). |
| | 54 | |
| | 55 | * Indexing and searching server (anyone can troll the content servers and create their own indexes and searching mechanisms) |
| | 56 | |
| | 57 | Users begin by entering entry.gc.domain into their browsers and, via round robin DNS, they are directed to one of many entry servers. The entry server re-writes the domain name to the entry server's canonical domain name (e.g. a.entry.gc.domain). The user begins an http file upload. Once the initial file is uploaded, the server provides the user with a URL where the user can check back on the status (e.g. a.entry.gc.domain/?status=hJlw24lF814Jqm9z). Then, the server begins transcoding the file. When complete, the server publishes the transcoded file in a pre-defined directory (e.g. a.entry.gc.domain/data/hJlw24lF814Jqm9z/). |
| | 58 | |
| | 59 | Meanwhile, primary storage servers, on a scheduled job, iterate over all available entry servers asking for content. When queried, if the entry server has content that has not been passed to a storage server, it hands it to the storage servers and logs which storage server it handed it off to. |
| | 60 | |
| | 61 | Once the storage server copies the content, it responds to the entry server saying it's complete. At that point the entry server publishes, via the status URL, the canonical URL of the uploaded content (e.g. a.store.gc.domain/hJlw24lF814Jqm9z/item.ogg) to the uploader and deletes the content from the entry server. |
| | 62 | |
| | 63 | Storage servers, on entering the system, will publish their maximum capacity. When they reach capacity, a storage server admin may turn off their scheduled job to stop additional downloads. Storage servers will advertise their status via a public URL (e.g. a.store.gc.domain/status). |
| | 64 | |
| | 65 | Backup servers can choose a primary storage server to mirror based on the published storage capacity. Backup storage servers run a scheduled job to mirror the content from the primary storage servers. The backup storage servers only exist to recover from a primary server going down. They do not delivery content to the public. |
| | 66 | |
| | 67 | Caching servers provide load and bandwidth balancing for the primary servers. |
| | 68 | |
