Two years ago Andrew Forrest and Kevin Rudd launched the Australian Employment Covenant (AEC) with the bold goal of creating 50,000 jobs for indigenous Australians by October 30, 2010. Yet this milestone has now passed without any fanfare: last week media reports focused instead on Forrest’s new project,  GenerationOne.

Forrest has been smart enough to back away from his initial two-year timeframe, but his original goal means it is now timely to review the AEC’s progress. Many will want to know whether Forrest should be castigated for a failure, or congratulated for his achievement. But the more important question is whether the taxpayer funds invested in the scheme have been money well spent.

So what, exactly, did the AEC set out to achieve? Some recent reports suggest that the aim of the AEC has only been to secure 50,000 job pledges from employers. But the formal agreement between the AEC and the Commonwealth government says that the original aim included an aspiration to “secure 50,000 sustainable jobs filled by indigenous Australians”, and the AEC website lists the scheme’s first goal as the “placement of 50,000 indigenous people into work”. If the goal of the AEC related only to job pledges, then these statements are confusing at best.

On job pledges, the AEC has been very effective in securing employer commitments. To date, more than 170 employers have promised a combined total of almost 26,000 “Covenant Jobs” under the scheme. Employers are asked to informally guarantee these jobs to indigenous applicants who complete appropriate training. This could be an important contribution: it not only creates an incentive for indigenous job seekers to undertake training but also seeks to challenge the well-established pitfall in which training is offered simply for training’s sake.

However, to accurately assess the outcomes of the AEC much more transparency is required. In its agreement with employers, the AEC states that it will “use its best endeavours” to:

record and publish the outcomes achieved. This will be measured by the number of indigenous participants who complete job-specific training, commence employment with an AEC employer and subsequently remain in full or part-time employment for a minimum of 26 weeks.

Such public reporting has not occurred. Indeed, two years into the scheme, the AEC does not yet have accurate data with which to assess outcomes on these measures. This is something they are seeking to redress, but even when such data are made available key aspects of the scheme will remain unclear.

For example, the AEC currently estimates that about 2800 indigenous people have been placed into Covenant Jobs. Even this might be a significant achievement. But for proper evaluation more information is needed about retention rates as some employment programs achieve high placement rates but lose many of their participants early on.

And any potential for misinterpreting the data needs to be minimised. This means addressing issues such as “substitution effects” whereby a successful indigenous job placement comes at the expense of another.

Significantly, several AEC employers also have Reconciliation Action Plans, under which they might have already committed to indigenous employment targets. That means it is impossible to know whether a position offered under the AEC would have been filled by another indigenous job seeker anyway. The problem here is not the outcome as increased indigenous employment is a positive result. Rather, it is the technical question of isolating the effects of the AEC in order to accurately determine its contribution.

Understanding value for money also requires transparency about the public investment. The contract between the Commonwealth government and the AEC was signed before the introduction of new rules that require government agencies to publicly list all grants they have made. That means the amount of taxpayer money committed to the AEC has not been listed. The government’s last Closing the Gap report identified that at least $4 million had been earmarked for the AEC including start-up funding and a “long-term funding model based on the achievement of outcomes”, but it is not clear whether this is the total funding committed or just a proportion of it.

What we do know, however, is that this allocation is not to fund training, which is provided through existing government schemes such as the Indigenous Employment Program.

A key feature of the funding agreement is the provision of outcome payments to the AEC on the basis of job pledges, job placements and employee retention to 26 weeks.

To date, because of problems in collecting and publishing accurate data, the AEC has only claimed outcome payments for job pledges. But not all of the jobs promised under the scheme have resulted in jobs for indigenous people, and it is far from clear that all will.

Until more detailed information is publicly released about the AEC’s outcomes and its agreements with government, it will be impossible to assess if taxpayer investments are creating sustainable indigenous jobs. After two years, it is in the public interest that there be far greater public disclosure and discussion about this unusual initiative launched with great fanfare and endorsed by some very influential Australians.