November 21, 2006

Webscale Papers

I had a nice chat with an old friend tonight about Snapvine & recent stuff he's come across at MSR. This inspired me to go look up a few really interesting papers from MSR & google labs. If you haven't had a chance to look at the papers on google labs, check it out: http://labs.google.com/papers/

Of particular interest are GFS, Chubby & BigTable.

Paxos algorithm for electing a master in a fault tolerant system
http://research.microsoft.com/users/lamport/pubs/paxos-simple.pdf

This is all quite far beyond anything we need to implement at Snapvine, but it's interesting reading about how you go from 100s of servers to 10,000s of servers.