Wayback Machine stowed in shipping container

By Dave Bailey on Mar 31, 2009 6:16 AM
Filed under Hardware

Non-profit organisation the Internet Archive has crammed its entire history of the internet into a shipping container.

The Internet Archive moved from its own custom hardware and storage system, which hosts its Wayback Machine, to an optimised Sun Modular Datacentre (Sun MD) platform – also known as the "data centre in a shipping container".

Sun has agreed to host, manage and optimise the facility at its Santa Clara campus in California, also providing power, cooling and network connectivity.

"As more of the world's most valuable information moves online and data grows exponentially, the Internet Archive will serve as a living history to ensure future generations can access and continue to preserve these important documents over time," said Internet Archive founder Brewster Kahle.

The Internet Archive has constructed a library of internet sites and catalogued digitised cultural artifacts in audio, image, text and video form.

Access to the general public and academic researchers is free.

By the end of 2008 the amount of information stored was more than three petabytes (three million gigabytes), with a growth rate predicted to be about 100 terabytes a month.

The platform, tweaked by Sun to house the Internet Archive's library, uses Sun Fire x4500 servers running Solaris 10 and stored under the ZFS file system.

As well as wanting to take advantage of cloud computing developments, the Internet Archive saw it as important to maintain and own the information contained in the Archive.