CCTV-camera
(Image: Adobe)

As the profound flaws and poor track record of facial recognition become apparent, Australian lawmakers, especially at the state level, look set to be left behind in an international move to block a technology that is already wrecking lives.

Overnight, Boston banned the technology, following Oakland and San Francisco which banned it last year. Boston’s ban had the support of the Boston police, who declared the technology unreliable. States such as California, New Hampshire, and Oregon have more limited bans and a number of other states have restrictions on its use.

A late 2019 paper by the US National Institute of Standards and Technology examined both one-to-one matching (matching a facial image with another facial image) and one-to-many (matching a facial image to a database of images). One-to-one facial recognition had a vastly higher rate of false positives for Asian, African-American and Native American faces, while one-to-many systems had a significantly high rate of false positives for African-American women.

Funnily enough, systems designed outside the US delivered fewer false positives for non-white faces.

The study confirmed earlier studies showing facial recognition systems, relying heavily on white, and male, facial features, often fail with gender and racial diversity. Unsurprisingly, a Black man was at the centre of the first case of US police arresting an individual after a failure by facial recognition systems.

Earlier this month, IBM said it would no longer sell facial recognition tech, saying “vendors and users of Al systems have a shared responsibility to ensure that Al is tested for bias, particularity when used in law enforcement, and that such bias testing is audited and reported.” In January, Google supported EU consideration of a temporary ban, and later launched legal action against the company behind Clearview, which scrapes social media and the internet for facial data and provides it to law enforcement.

Facial recognition is a large and rapidly growing industry. One estimate suggested it was worth US$3.4 billion globally already and is expected to grow at over 14% a year in coming years.

In Australia, police have been given a free hand to use systems demonstrably biased against non-white individuals, despite the Human Rights Commission calling for moratorium on the technology. Scandal-plagued NSW minister David Elliott even boasted about the use by NSW police of it. Queensland has begun using facial recognition systems at sports stadiums while uploading all Queensland drivers licence photos to a national facial recognition database.

The only political pushback came last year from federal parliament’s joint intelligence and security committee, which unusually outright rejected an attempt by Peter Dutton and Mike Pezzullo to establish the basis for a national facial recognition system without privacy safeguards, oversight, accountability or recognition of the deep biases of facial recognition systems.

However, this only addresses the wild overreach by national security bureaucrats at the federal level; the primary problem is at the state and territory level, where police forces have been able to exploit surveillance technologies without check.

The federal Privacy Commissioner earlier this year launched an inquiry into the possible use of Clearview by Australian police forces. But with persistent evidence of racial bias in Australian police forces and the vast overrepresentation of Indigenous Australians in the criminal justice system, the potential for errors and outright misuse in Australia is rapidly growing without check.

Correction: This article originally referred to the NIST paper as “peer-reviewed”; the NIST has contacted Crikey to advise it was not peer-reviewed as per an academic paper. Crikey has removed the phrase.