More evidence has emerged that the Attorney-General’s Department has significantly downplayed the extent of its work in developing a data retention regime to record data about Australians’ telephone and internet usage — and may have misled a Senate committee about it.

At an estimates hearing in May, departmental officials appeared before the Legal and Constitutional Affairs Committee and answered a series of questions about the department’s work on data retention from Greens Senator Scott Ludlam. Ludlam asked about the extent to which the department had developed legislation to establish a requirement that telecommunications and internet carriers retain data, before the issue was referred for a public inquiry by the Joint Committee on Intelligence and Security by then attorney-general Nicola Roxon.

Last year, during hearings of that inquiry, the department had told Liberal Senator George Brandis that draft legislation on data retention existed, along with “lots and lots of bits and pieces” but it was “subject to cabinet confidentiality”.

But in response to Ludlam’s questions at estimates, the department backed away from the suggestion that draft legislation existed. The relevant manager, Band 1 SES officer Catherine Smith, when asked by Ludlam when the draft began, replied “we are not drafting data retention legislation. We had in the past consulted, and now we are awaiting an outcome of the Parliamentary Joint Committee on Intelligence and Security.”

Ludlam: “So at no stage were parliamentary counsel engaged to draft a bill?”

Smith: “Not on data retention, no.”

Secretary Roger Wilkins then said there had been “discussions with parliamentary counsel”.

Parliamentary counsel is the Office of Parliamentary Counsel, which drafts legislation (though not regulations) for government departments; typically, line area officials provide drafting instructions to OPC explaining what they want a bill to do, then meet to discuss it with OPC staff if there’s any complexity, then discuss drafts of bills once they’re prepared. A complex draft bill or one that is the subject of extensive discussion between departments and OPC will generate considerable email traffic.

Smith explained that discussions between AGD and OPC were about the “broader need to reform telecommunications interception legislation”. The Telecommunications (Interception and Access) Act 1979 is “a Gothic piece of legislation with gargoyles and flying buttresses”, Wilkins said, a colourful analogy but not incorrect in terms of the state of an Act that has been constantly amended in recent decades. Wilkins went on to volunteer that the discussions with OPC were actually at secretary level — a very rare occurrence.

Ludlam: “But there was no such thing as a draft of a bill that included data retention provisions?”

Wilkins: “I do not know — nothing that I have got.”

Smith: “No, there were some very vague draft provisions, not to do with data retention though.”

Ludlam: “OK. That sounds fairly unequivocal.”

It’s not quite as unequivocal as Ludlam thought, however. Not merely does the estimates hearing evidence sit poorly with what Wilkins told Brandis last year, it sits even worse with what the department told Crikey this week. As part of our investigation of AGD’s data retention efforts since 2009, Crikey recently FOI’d a large amount of material relating to the issue. This week, the department told Crikey the request had been rejected because “due to the size of your request, the work involved in processing your request would substantially and unreasonably divert the resources of this agency” (nothing malign should be read into that — agencies can reasonably reject large applications, and Crikey will narrow its request accordingly). In particular, the department advised Crikey that:

“… the number of documents falling within the scope of this request would be extremely large, particularly the correspondence and briefings to the Secretary and Ministers and the drafting instructions to the Office of Parliamentary Counsel.”

So, which was it — was there just a friendly yarn between Secretary Wilkins and First Parliamentary Counsel Peter Quiggin over flying buttresses in the T(IA) Act, or data retention drafting instructions so voluminous it would be an unreasonable diversion of resources to provide? Does draft legislation exist or not?

There’s a pattern here. As Crikey explained recently, there’s a big contrast between the department’s insistence at the same estimates hearings that its consultations with industry in data retention were a self-initiated set of casual chats by a middle manager, and documents obtained under FOI by the Pirate Party, which show 12 formal consultation meetings in various capitals as well as interdepartmental meetings.

“It shows how determined the department is to run its own agenda through successive attorneys-general,” Ludlam told Crikey. “Knowing how wretchedly unpopular this scheme is, the officers at the table did everything short of bald-faced lying to downplay how far advanced this proposal had come. I can’t immediately think of a better example of how the department reserves total secrecy for itself in pursuit of stark transparency for the rest of the population.”

The Department, however, disputes that there is any discrepancy. “The Attorney General’s Department does not believe there are any inconsistencies” between the department’s statements at different times, a spokesperson told Crikey.

The issue is likely to further concern the Joint Committee on Intelligence and Security, members of which have already expressed concern about the extent of information AGD has provided in its inquiry into national security reforms. That inquiry is likely to report in coming days.

It also goes to the heart of the debate over Edward Snowden’s revelations about the extent of US internet and telecommunications spying. When governments breach privacy in the name of national security and do so entirely in secret, the only way to have the debate that a democratic society should have about balancing security and privacy is via whistleblowers and leakers. And it appears AGD wants to operate in secrecy as much as possible.

*Additional research from Crikey intern Farz Edraki