AARNet completes major upgrade to mirror

Australia's Academic Research Network (AARNet) has upgraded the SAN (Storage Area Network) used for its mirror service from 11.4 Terabytes to 55 Terabytes.

The SAN is used by Australia's academic community as a local mirror site for the downloading of important software such as updates to various flavours of the Unix and Linux operating systems.

A mirror (sometimes called a cache), is a local store of popular data users can draw on to prevent multiple users from attempting to download the same sets of data across international networks.

According to Steve Maddocks, director of operations at AARNet, the mirror contains "most of the major public domain Unix software" available, including Ubuntu, Fedora, Debian and FreeBSD. The mirror automatically downloads all updates to these systems - such that when systems within Australia's academic community download an update, they download it from a local source.

Maddocks said there are two goals to the mirror. The first is to allow better performance for users on AARNet - as they will receive files faster if they are stored closer to them.

"It is quite frustrating to download large objects from sites overseas," he said.

Second, the mirror saves AARNet and its customers in terms of reducing their reliance on international traffic.

"We have met these objectives," Maddocks said. "We save the cost of international transmission, of users downloading the same files over and over again across the international networks."

AARNet also uses its mirror to provide these same files to the general public.

"The mirror serves tens of thousands of ordinary Australians, with 50,000 to 60,000 people using it every day, downloading around 1.3 Terabyte of data," he said. "Of all the content on the mirror, two-thirds goes to Australian internet users."

Maddocks said it is of incremental cost to provide these files to the public.

AARNet uses Hitachi Data Systems (HDS SMS100) storage arrays to store the data, whilst the front-end of AARNet's mirror is a 1.5 Terabyte SSD (solid state drive) array to provide fast access to popular content.

"Its very useful if the mirror is very sporadic or bursting," Maddocks explained. "If a new version of Ubuntu or Fedora comes down, the hits on our storage go through the roof."

Maddocks said AARNet provides a 10Gbps link to the mirror. Strictly speaking, the Mirror SAN storage architecture has 28 Gbps of bandwidth. AARNet engineers have demonstrated ther mirror delivering 7.6Gbps successfully with no observable degradation of service.

New content

Maddocks said AARNet is "desperately trying to get the Sourceforge collection" of open source software onto the mirror in the coming months.

But the research network is not interested in providing a mirror for commercial software vendors, he said.

Maddocks said AARNet has "dabbled" in cacheing updates of commercial software, but has learned that it is too much pain for little reward.

"Almost always it ends in some proprietary copyright process - its just too difficult to manage," he said.