Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo

 

TR10: HashCache

Vivek Pai's new method for storing Web content could make Internet access more affordable around the world.

5 comments

Closing the divide: Students surf the Web at Ghana’s Kokrobitey Institute, a conference center with an Internet connection only about four times as fast as dial-up. The link is enhanced by Princeton’s low-cost, low-power HashCache technology, which stores frequently accessed Web content.
Olivier Asselin/WPN

Throughout the developing world, scarce Internet access is a more conspicuous and stubborn aspect of the digital divide than a dearth of computers. "In most places, networking is more expensive--not only in relative terms but even in absolute terms--than it is in United States," says Vivek Pai, a computer scientist at Princeton University. Often, even universities in poor countries can afford only low-bandwidth connections; individual users receive the equivalent of a fraction of a dial-up connection. To boost the utility of these connections, Pai and his group created HashCache, a highly efficient method of caching--that is, storing frequently accessed Web content on a local hard drive instead of using precious bandwidth to retrieve the same information repeatedly.

Despite the Web's protean nature, a surprising amount of its content doesn't change often or by very much. But current caching technologies require not only large hard disks to hold data but also lots of random-access memory (RAM) to store an index that contains the "address" of each piece of content on the disk. RAM is expensive relative to hard-disk capacity, and it works only when supplied with electricity--which, like bandwidth, is often both expensive and scarce in the developing world.

Advertisement

HashCache abolishes the index, slashing RAM and electricity requirements by roughly a factor of 10. It starts by transforming the URL of each stored Web "object"--an image, graphic, or block of text on a Web page--into a shorter number, using a bit of math called a hash function. While most other caching systems do this, they also store each hash number in a RAM-hogging table that correlates it with a hard-disk memory address. Pai's technology can skip this step because it uses a novel hash function: the number that the function produces defines the spot on the disk where the corresponding Web object can be found. "By using the hash to directly compute the location, we can get rid of the index entirely," Pai says.

To be sure, some RAM is still needed, but only enough to run the hash function and to actually retrieve a specific Web object, Pai says. Though still at a very early stage of development, HashCache is being field-tested at the Kokrobitey Institute in Ghana and Obafemi Awolowo University in Nigeria.

Video

Next Page »

  • Page
  • 1
  • 2

David Talbot

Chief Correspondent

I’m MIT Technology Review's chief correspondent, keeping an eye most often on the world of information and communication technologies—and asking my kids when I don’t...
See full bio »

Related Articles:

Spare Some Bandwidth?

Pakistani scientists have a way to boost download speeds.

Mason Inman

Net Neutrality: Far from a Done Deal

Legislation and industry concessions won't guarantee equal access to the Internet.

Wade Roush

The Internet Reborn

A grass-roots group of leading computer scientists, backed by Intel and other heavyweight industrial sponsors, is working on replacing today's Internet with a faster, more secure, and vastly smarter network: PlanetLab.

Wade Roush