Inside CHOGM's IT network design

Solving the uplink crises of Commonwealth delegates one at a time.

It has only been four days since the Commonwealth Heads of Government Meeting (CHOGM) closed in Perth and the last of 56 cubic metres of IT equipment has been boxed for shipping back to Canberra.

For most of the 53 IT contractors from services firm ASG charged with the setting up and running the CHOGM network, it is a chance to relax after 12 months of planning.

"I'm still trying to catch up on my sleep," ASG's CHOGM senior project manager, Peter Yin, said.

Planning for the event - a get-together of Commonwealth leaders held every two years - started in November 2010.

The IT logistics were complex. Most of the hardware was configured and tested in Canberra before being shipped to an undisclosed "security-cleared facility" in Perth.

Once there, it became Yin's responsibility. After a "quick test to make sure it all [still] worked", the kit was repackaged with security tape and tamper-resistant seals to assure security in the Perth supply chain.

Yin physically sorted and shifted the load into containers for the various venues - the Perth Convention and Exhibition Centre (PCEC), the Hilton Hotel and Kings Park, where a leadership retreat was held.

Although desktop computer hardware was shipped from Canberra, it was imaged in the staging area at ASG's Bentley data centre in Perth.

"We built in total 304 PCs at Bentley over two-to-three weeks in September," Yin said.

"Some of the services guys who didn't have much to do, particularly during the night, were given the dubious honour of building all those PCs up with a standard build."

The computing environment was relatively locked down. Computers supplied for CHOGM were imaged with an email client, Office 2007, Internet Explorer and Adobe Reader.

Delegates that chose to use these PCs - rather than their own - did not have permission to install other programs.

Network design

The CHOGM network was functionally partitioned into three sub-networks using virtual LANs.

One sub-network, designated for the Prime Minister, ran through a secure network back to the Department of Prime Minister and Cabinet in Canberra. The other sub-networks were set up for media and delegates respectively and supported BYO devices.

The main venue at the exhibition centre was connected with a 100 Mbps primary link supplied by Amcom. Backup 20 Mbps links were secured from Telstra and Optus, although in the end they went unused.

User internet connections ran through a Websense proxy server.

CHOGM used barely any of the network infrastructure already at Perth Convention and Exhibition Centre. What it didn't use, it disconnected.

"The only thing we really used from their infrastructure was their patch panels. Everything else was ours," Yin said.

"We had some time to set up [our] rack in the comms room in the PCEC, put in the Websense devices and get the main switches put up. Then we took down the existing PCEC network infrastructure and switched it across to [ours]."

The rack was configured to ensure there was no single point of failure.

"We knew how we could switch things across," he said. "If we had to do it in a hurry starting with the 100 Megabit link we could quickly switch across to one of the 20 Megabit ones we had".

The network capacity was not pushed even at the conference's peak. Up to 3000 delegates were expected but ASG declined to put a figure on the number of concurrent users at any one time.

"The most we saw that 100 Megabit link go up [to] was 30 Megabits," Yin said.

Yin - who provided on-ground IT support at the convention centre - did find one case of a user trying to saturate the uplink.

"We had one guy who was complaining his 100 Gigabyte upload to his FTP server was taking so long," Yin said.

"I had to explain to him that 100 Gigabytes was going to take a long time on a 100 Megabit link.

"You have to be polite but you have to point out that 100 Gig is a big upload. And it wasn't us that was slow.

"We were quite happy to give him a whole 100 Megabit [per second]. His FTP wouldn't take 100 Megabit."

In total, internet access at CHOGM was provided from over 500 access points. The points used Cisco/Linksys hardware while core network switch infrastructure was Cisco.

There were floor pits in the exhibition centre every ten metres. Each provided only one network access point.

"We had to put an either 8- or 16-port switch under each of them to provide for the [IP] phone and the access point connectivity [for delegates]," Yin said.

Desks for delegates were fitted with a supplied computer and an Avaya IP handset. Most allowed only national calls though a few were set up for international direct dial.

Read on to page two for the tale of Microsoft updates left in a build.

Challenges

The CHOGM network challenges were logistical and service-level, rather than technically - oriented.

With regional attention focused on the outcomes of the meeting, Yin conceded part of the challenge was the pressure to keep the network up.

"I was there for the [leadership] retreat physically to make sure all was OK," he said.

The ASG team ran morning and evening checks on the network to make sure everything went smoothly.

"I came in one morning and found everything had rebooted itself because somebody had forgotten to stop all the Microsoft updates in that build," Yin said.

"That was on the day setting up for the conferences so we fixed that up by six o'clock before anyone came in."

Being responsible for IT at an event like CHOGM also means addressing user issues in real-time.

"If a user had a problem we can't get back to them in our usual four to six-hour type SLA structure," an ASG spokeswoman said.

"Everything had to be handled quickly there."

Yin said that the CHOGM taskforce - the organisers of the meeting - were given priority during the event itself.

"They were probably the most important customers at that time because they were highly stressed," Yin said.

"We made sure they were always looked after well".

Delegates and media with IT problems called a helpdesk, answered by support staff in Canberra who, if necessary, escalated to ASG staff on the ground in Perth, including Yin and ASG's other CHOGM project manager, Cheryl Goodall.

Yin said a third-level support team in ASG's Perth office stood ready to assist.