Jacques Chester has an interesting idea to deal with the Digg effect: Chester essentially wants to use Bittorrent for HTTP caching to help websites struggling with sudden surges of traffic, and he proposes to add an X-Torrent header to HTTP to do so. From Chester's post:

"The thinking behind the X-Torrent header is that whenever a web server returns any headers over HTTP, it includes an X-Torrent header pointing to a torrent tracker for that document. When a website becomes heavily loaded, browsers would use the BitTorrent protocol to obtain the requested document or resource from their peers, distributing the load amongst the audience as well as on the server."

This does sound like an interesting idea, and there's definitely a need to deal with these types of issues. Bigger websites can oftentimes deal with traffic spikes by offloading some content to CDNs, switching to static, cached content as opposed to dynamically rendered pages and things like that, but your average blogger with a shared hosting account is usually just screwed if his site goes down.

However, there are some reasons to be skeptical about this proposal. First of all, most browsers still don't support Bittorrent, but that of course could change in the future. But I'd also argue that the X-Torrent header model has a fundamental flaw in that it still depends on the very server that is in trouble in such a situation.

Sure, some of the data would be offloaded to the clients, but it that doesn't mean that the server won't still have to deal with lots and lots of requests, which could eventually overload it. New users won't get hold of the P2P-cached document either in such a case, even if it's readily available in the cloud.

I think a more decentralized system that only kicks in if websites are not or barely reachable is much more promising. Howie Vegter suggested something like this called Notorrent a couple years back, but it still hasn't caught on either. Maybe we really do need to wait for a ubiquitous browser implementation of Bittorrent?

Tags: , , , , ,