DotSlash - Creating Content Distribution Networks on Demand

Abstract

Traditional content distribution networks, such as Akamai, are well-suited for static web services that routinely experience large traffic volumes. They are unsuited for active content, i.e., content generated by scripts from databases, and web sites that are unlikely to receive significant number of requests. However, a few such sites will invariably experience their "fifteen minutes of fame", typically by being mentioned on a high-volume news site such as SlashDot or CNN. Such flashcrowds or "slashdot effect" will routinely cause single-server websites to collapse. We have designed and prototyped an autonomic web replication system, called DotSlash, that drafts rescue servers fully automatically, without user intervention. The system discovers suitable rescue servers via wide-area service location, either among peer servers or from a dedicated pool of rescue servers, allocates them for temporary use and redirects requests to them. The system is completely transparent to clients and does not require URL rewriting or other client modifications. We have designed two versions. The first, an Apache extension, deals only with static content, e.g., HTML pages or media objects. The second version can also replicate and execute scripts remotely. We have prototyped the system for the common LAMP (Linux, Apache, MySQL and PHP) configuration and shown that a common benchmark for bulletin boards can be replicated without code changes, yielding capacity increases bounded only by the database server. Since many such systems, including most blogs, are bottlenecked by the web server, our system can significantly increase capacity and works even for extremely rapid load increases. We are currently investigating how such systems can be further extended by increasing the database capacity of read-mostly systems with loose consistency constraints.