Govt to work up big data pilot projects

The Federal Government is hoping to build its big data capability through a series of pilot projects between now and July 2014.

A draft whole-of-government big data strategy was put on the table late Friday of last week after a period of industry consultation. (pdf)

The strategy is light on detail and sets the deadline to develop anything more substantial — such as "better practice guidance", a report on "barriers to big data" and a "guide to responsible data analytics" — at between March and July next year.

The main action item involves identifying and supporting an undisclosed "number of big data pilot projects" across government between now and July 2014.

The scope covers "existing projects that take advantage of big data analytics as well as the initiation of new big data projects to be led by selected Government agencies."

Convincing agencies to pilot big data projects could prove challenging. Former human services minister Kim Carr raised that prospect last month, saying "there are far too may people deeply suspicious of what big data is all about".

The draft strategy outlines a "number of specific issues around privacy" that will need close attention.

They include:

• better practice in linking together cross-agency data sets
• better practice use of non-government data
• de-identification and the "mosaic effect" ("the concept whereby data elements that in isolation appear anonymous can lead to a privacy breach when combined")
• the necessary considerations to make before releasing open data; and
• data retention and cross-border flows.

The strategy indicates these issues are being thought about but provides no indication of when it might have answers for concerned agencies.

"Forthcoming guidance will be produced to increase agencies understanding of these issues and provide advice and best practice for addressing these," the strategy states.