Census puts business model under review

The Australian Bureau of Statistics’ census team has put its business model, software development and infrastructure outsourcing contracts under review after collecting almost a third of this year’s population survey responses online.

Some 2.6 million households used the bureau’s web-based eCensus form last month, up from 778,000 in 2006.

Paul Lowe, head of the ABS population census program, hoped use of eCensus would double to reach 60 percent of Australian households by the next population census in 2016.

“As an organisation, we’re looking to expand our e-collection across a range of collections,” Lowe said, highlighting the introduction of a web-based form for its agricultural census in July.

With the $400 million population census under its belt, Lowe said the bureau was researching how it might build and host future electronic census forms.

The Australian Financial Review reported last month that the bureau had saved up to $80 million by reducing the number of staff it needed to collect and process paper forms this year.

The 2011 and original 2006 versions of the population census application were developed and hosted by IBM.

Last year, the bureau’s director of infrastructure Tony Marion said the 2011 eCensus would be the last to be outsourced to IBM, with future applications to be hosted on an in-house, virtualised server farm that burst onto third party clusters when needed.

But Lowe said the bureau had an “absolutely fantastic” relationship with IBM and would consider it alongside in-house development and hosting and other outsourcers for the next population census.

The bureau hoped to decide on the most cost-effective solution by mid-2012, start building systems that year and begin testing the population census application in 2014.

It was also considering rolling out portable devices – including iPads and smartphones – to future census collection staff to gather “real-time management information” and improve the team’s efficiency.

“At the moment, we have a really antiquated system,” Lowe said. “We’re looking at our business requirements, then will go to market.

“[The population census is] a huge exercise … We deliberately start early; you only get one shot at it every five years.”

Lessons learned

About two weeks before census night, the Australian Bureau of Statistics sent letters to 550 internet service providers (ISPs) and network operators to prepare them for a “considerable” spike in network traffic caused by eCensus.

Lowe said it was a proactive attempt to avoid the bureau’s 2006 problem, when certain ISPs “throttled back their speeds because they thought it was a denial of service attack”.

According to IBM, this year’s eCensus delivered 100 percent availability during the busiest period, between Saturday 6 August and Sunday 14 August.

It handled a peak load of 200,000 households at 8.43pm on Tuesday 9 August, returning 85 eCensus submissions a second from IBM’s data centre in Baulkham Hills in north-western Sydney.

Lowe said the eCensus application was designed to handle a peak of 120 submissions a second but did not know exactly how much traffic the system could handle because the team could not “break the system” during testing.

“We tried to break the system but we couldn’t; we couldn’t generate enough traffic so we don’t really know what the limit is when it would fall over.”

Although the application delivered average page-load response times of less than one second, the bureau’s internally hosted website struggled to handle the volume of visitors seeking information from its online eCensus guides.

About 679,000 households that had completed online census forms also failed to hit the ‘submit’ button on census night, sparking a public relations campaign by the bureau.

Lowe said the bureau “probably didn’t go as far as we would have liked” in performing usability tests and planned to put the next version of eCensus under more testing.

Feedback from the public indicated that this year’s eCensus had met its objective of being “fast and easy”, he said.