After months of uncertainty and two days of absurd debate conducted without even the legislative basics being known, the major parties have hammered out a deal on data retention that will see a public interest advocate arguing against police being able to access a journalist’s metadata as part of a warrant process.

After increasing media agitation about the chilling effect of data retention on journalism, this week the government agreed to a Labor proposal that warrants be required to access the communications data of journalists in order to identify a source for a report or story. However, the media industry wasn’t satisfied, believing a warrant process should be contestable, with media companies and journalists being able to argue against a warrant being granted. Labor and the government, in a deal struck without consultation with the media, have agreed on a compromise measure: under a new “journalist information warrant”, agencies other than ASIO would not be able to obtain retained data for the purpose of identifying a journalist’s source without a journalist information warrant from an independent issuing body such as a judge, which could only be issued if:

“… the public interest in issuing the warrant outweighs the public interest in protecting the confidentiality of the identity of the source in connection with whom authorisations would be made under the authority of the warrant”.

As part of that process, the submissions of a public interest advocate would need to be assessed. PIAs would be appointed by the Prime Minister.

ASIO would not be required to go through such a process — it can simply apply to the Attorney-General for a warrant. In an “emergency”, not even the Attorney-General’s approval would be required —  the head of ASIO could simply issue one off his or her own bat.

The “Public Interest Advocate” process would still mean journalists would remain ignorant of when their data was being targeted. In fact, anyone who reveals the existence of a journalist information warrant, or even a request for such a warrant, or whether such a request has been granted, can be jailed for two years. A journalist who learnt that their data had been targeted would thus be gagged from revealing it.

There is some good news in the rest of the amendments, which implement the recommendations of the Joint Committee on Intelligence and Security report. As Crikey revealed on Tuesday, the Attorney-General George Brandis had sought to back away from his commitment to implement all of them in relation to the reporting of efforts to target journalists’ data. However, Brandis has been forced to stick to his commitment, and the amendment requires that both ASIO and other agencies will be required to report journalism information warrants to the Inspector-General of Intelligence and Security and the Ombudsman, and the Attorney-General must advise JCIS “as soon as practicable”; moreover, the committee can demand a briefing on the warrants from IGIS and the Ombudsman.

A new section 187AA of the Telecommunications (Interception and Access) Act also establishes in legislation, rather than in regulation, what data will be retained under the data retention scheme — finally. However, there is little specific detail in the list, which is similar to that circulated last year, meaning communications companies will be able to interpret the list as they wish. One significant omission, however, is download volumes, which were originally part of the draft dataset but have now disappeared. That appears to be a blow for the copyright industry, which wanted download volumes so they or authorities could pursue file-sharers. Ever since the Australian Federal Police Commissioner Andrew Colvin foolishly revealed his desire to use data retention to pursue file-sharing last year, the government and the AFP itself have tried to row back as far as possible from any suggestion data retention would be used to go after file-sharing.

The many faults of mass surveillance aside, on its own, the journalism warrant amendment (despite the two-year jail sentence) will improve on the existing situation, in which agencies like the AFP can try to track down the sources for journalists’ work using communications data basically on their own recognisance, with no independent approval or oversight. Now agencies like the AFP will have to satisfy a judge that it is in the public interest that they go after a journalist’s source and the benefit to the public interest outweighs the public interest in protecting confidentiality — and be opposed by an advocate for the contrary position.

Still, the legislation even in its amended form opens the way to mass surveillance on a scale never before seen in Australia.