The Attorney-General's Department was told to keep a mandatory data retention regime to a maximum of six months before it initiated public consultation proposing metadata retention for "up to two years".
The recommendation is contained in a preliminary privacy impact assessment of proposed reforms to the Telecommunications (Interception and Access) Act, conducted by Information Integrity Solutions and submitted to the department in December last year (pdf).
The document, released publicly for the first time in late August under freedom of information laws, recommended the Attorney-General "limit the non-content data retention requirement to a short period (6 months) unless there is strong evidence relevant to Australia of the utility of a longer period".
It also suggested that any regime be legislated, rather than regulated and that the nature of the data marked for retention be "clearly defined".
Nicola Roxon, who became Attorney-General in the same month— likely after the report was commissioned — opened the data retention proposal for public consultation and a Parliamentary inquiry in May this year.
But she has rejected current coverage of the proposal, which was initially based on a two-line inclusion in the discussion paper on the reforms released by her Department, as well as knowledge of closed-door discussions held with ISPs up to two years ago.
The department, and Roxon, have also relied heavily on the existing European data retention model first mandated six years ago, despite moves since to mark the directive as unconstitutional in several member countries, as well as evidence that many countries who did implement the regime applied a six-month ceiling on most datasets.
Australian law enforcement agencies have indicated the two-year retention model came as the result of a compromise during government discussions, in the face of proposals for five-year or even indefinite regimes.
But the assessment shows clear concerns voiced to the Attorney-General's Department that the proposals could heavily impact on the privacy of Australian citizens, without proper assurances to "protect retained information from misuse, loss or other unauthorised (or new) uses".
"Access to non- content data is becoming more sensitive as the extent and nature of the data held expands and the power to analyse and draw inferences from personal information increases," the impact assessment states.
"Requiring service providers to hold non-content data for long periods will add to the pool of data available (to the service provider as well as other agencies) and hence the sensitivity."
The Attorney-General's Department did not reply to questions at the time of writing.
Honey for hackers?
Telcos have warned of the significant cost involved in establishing such a regime, placing a preliminary figure of $500-700 million on acquiring the systems required to keep the metadata, depending on what is required for retention.
The privacy impact assessment, too, weighed into those concerns, suggesting the central storage of metadata could provide a "honey pot" to potential hackers that outweighed other concerns associated with the proposal.
It also recommended the retention of metadata be restricted only to those datasets that telcos already created or kept for billing purposes.
Major carriers including Telstra and Vodafone recently told the committee that historically held data such as user location or SMS data was no longer kept or deleted within two weeks due to the vast volumes created by an influx of users and increase in data.
Greens Senator Scott Ludlam told iTnews the assessment showed a "basic contradiction" to the current proposal before a Parliamentary inquiry.
"I think [the Attorney-General's Department has] jettisoned all attempts at being interested in privacy protections and they've made a gigantic ambit claim to see how much sticks," he said.
"They've made an enormous wishlist of everything that's not necessarily justified. It's a profoundly cynical exercise we're being subjected to."
He said the assessment was a "good starting point for how you could potentially put some boundaries around a data retention model that would still give law enforcement agencies what they needed".
The Australian Privacy Commissioner Timothy Pilgrim had urged the Government on multiple occasions (pdf) to undertake a privacy impact assessment of the data retention proposal to "help identify and address potential privacy issues associated with the mandatory retention of stored computer data and traffic data".
In his most recent submission (pdf) to the parliamentary committee currently inquiring into the proposed reforms, Pilgrim argued that, "in the event that such analysis has already been undertaken, the [Office of the Australian Information Commissioner] suggests that it should be made public".
iTnews asked Pilgrim's office if it had helped or consulted in the impact assessment but did not receive a reply at the time of writing.
Senator Ludlam said that although the assessment was a good first step, it did not take into account any changes to the warrant regime.
In one element of the assessment, the authors do state that proposals from the Attorney-General's Department could see the regime "permit law enforcement agencies to use accessed non-content data more freely including for intelligence and with less restrictions on data retention".
"I think it gives us a good starting position on half the debate," Senator Ludlam said.
"What the Government is seeking to do is take the existing flaw and entrench it for all data for all people. The fact is what they're building on is structurally flawed already."