The Attorney-General’s Department has revealed it is working with foreign governments on the possibility of forcing social media and encryption service providers to comply with Australian laws on internet surveillance, and established a taskforce to investigate data retention in 2009.

In a tense appearance before the Joint Committee on Intelligence and Security this morning that focused almost entirely on data retention, the Attorney-General’s officials came under pressure on a number of fronts from committee members Anthony Byrne, Andrew Wilkie, George Brandis and particularly John Faulkner. The Labor veteran zeroed in on why the controversial issue of data retention received minimal coverage in the department’s discussion paper and what work AGD had undertaken previously in pursuing the proposal.

The discussion paper has been widely criticised for failing to address data retention at all, and Senator Faulkner has repeatedly complained about the lack of detail about the proposal to keep telecommunications data for two years. “These issues are best undertaken with as much transparency as possible. And from day one it has been this committee’s experience that there’s no flesh on the bones,” Faulkner told officials.

Under questioning from Faulkner, AGD officials revealed a taskforce had been established within the department in 2009 that had developed a “working paper” with a data retention proposal for discussion with industry and that draft legislation had been drawn up within the department, begging the question of why there was such limited discussion of the proposal in the paper prepared by the department.

It has been widely-known for some time that AGD had consulted with industry about the proposal, but this is the first time we have been provided with specific detail about the highly secretive process, which apparently involved ASIO, the AFP, the Australian Crime Commission, the Department of Broadband and ACMA.

Shadow Attorney-General George Brandis also complained about the lack of clarity around the data retention proposal and probed the department for a clear assurance that content would not be caught up in the data retention scheme. AGD Secretary Roger Wilkins declined to say whether he thought the paper should have included more detail.

Andrew Wilkie also explored the issue of whether the reforms might be a dead letter given many of the targets of the proposals such as major social media companies are based offshore.

Officials admitted compliance by offshore based companies such as Facebook were “on a whim” but that AGD had discussed compliance with both Google and Facebook and had held discussions with US, UK, Canadian and New Zealand officials on the issue of ensuring compliance with each other’s laws. Wilkins offered to brief the committee on those discussions in camera.

The comments raise the prospect of US-based companies like Twitter being dragooned into Australian surveillance laws through bilateral agreements between governments or international treaties. AGD officials also suggested encryption providers could be treated the same way. Wilkie referred at several points to high-profile encryption service Tor; AGD officials appeared to suggest they would seek to obtain encryptions keys for such services.

Tor provides freely-available encryption for internet users under some of the world’s most barbaric régimes and is extremely unlikely to cooperate with any governmental efforts to surrender information about its systems.

The specific issue of the definition of telecommunications data occupied most of the hearing, with Byrne and Brandis both dwelling on the extent to which internet browsing data such as URLs would be caught in the definition. AGD was insistent that it should not include URLs but struggled with the issue of whether telecommunications data could be retained without accidentally retaining content data, as suggested by some industry stakeholders.

AGD also revealed that it had arbitrarily decided on a two-year period for data retention, while its portfolio agencies — ASIO and the AFP — had wanted longer. Wilkins cited the two-year EU data retention directive as a model, but was immediately pulled up by Byrne, who pointed out the EU directive proposed between six months and two years.

Officials also admitted, remarkably, they had not done any work on what the cost impact of data retention on carriers would be.

The tone of questioning bodes poorly for the success of the data retention proposal before the committee. There is plainly widespread concern on the committee about the lack of clarity around the proposal; the information Faulkner prised from the department on how extensive the work has been on it within AGD (but not shared with the Committee) will only deepen those concerns. And both Faulkner and Brandis noted how problematic any data retention proposal would be politically if it included web browsing.

The path to a surveillance state may yet prove a rocky one for the Attorney-General’s Department.