One of the more pointed observations made by Kevin Rudd at his Wednesday night press conference announcing a leadership ballot the following morning was that he would not be “lurching to the right on the question of asylum seekers, as some have counselled us to do.”

Rudd was right. Since becoming Prime Minister, Julia Gillard has made a point of hardening the government’s rhetoric on that issue, emphasising that she understood voters’ concerns about people arriving by boats.

On the weekend, she went further and adopted the opposition’s rhetoric about population issues. “I don’t believe in a big Australia,” she told Laurie Oakes. “I don’t believe in simply hurtling down a track to a 36 million or 40 million population, and I think if you talk to the people of western Sydney or western Melbourne or the Gold Coast growth corridor in Queensland people would look at you and say ‘where will these people go’?”

The usual anti-immigration suspects were quick to emerge. On cue, out came Dick Smith and “leading demographer” Bob Birrell to celebrate.

Gillard’s remarks adopted almost word-for-word the line the opposition has been using on the issue — cloaking an attempt to exploit animosity towards immigration by linking it to sustainability and planning issues. The only difference is that the opposition has also attempted to confect a link between immigration and asylum seekers as well, in order to tap into hostility toward Muslims.

Senior Labor figures had for some time been concerned that the opposition would try to use high immigration — which it oversaw while in office — as a way to exploit xenophobia.  Under the apparently innumerate Scott Morrison, they’ve done just that, while more junior players have got the dog-whistle going — Abbott’s parliamentary secretary Cory Bernardi called for the banning of burqas and Kevin Andrews — who presided over record immigration levels — declared he wanted immigration slashed by 80%.

Ironically, they were doing it at a time when Chris Evans as immigration minister was patching some of the large holes left by the Howard government in our immigration program — and particularly the rort that saw student visas being used as a backdoor means to obtain permanent residency in Australia.

Rudd wasn’t averse to a bit of rhetorical realignment himself in the face of the emerging anti-immigration campaign, dropping his liking for a big Australia and summoning the capable Tony Burke from the backwater of agriculture to take on the role of population minister.  Rudd’s strategy, typically, was for a review that would take the issue beyond the next election.

Nevertheless, Gillard’s rhetoric is a substantial further shift toward a hostile view of high immigration. Particularly when there’s any hurtling involved.

As the IPA’s Chris Berg nicely pointed out on Twitter on the weekend, it’s funny how we like policies that increase population levels through a higher birth rate, but not ones that achieve the same goal through immigration.

“Sustainability” is a fine word. Our major parties have little interest in sustainability when it comes to resources exploitation or stopping our carbon addiction from cooking the planet, but it’s a wonderful term to  wave around as a distraction. The opposition is presently proposing the hare-brained scheme of replacing the Productivity Commission — the single most important governmental reform of the past two decades — with a Productivity and Sustainability Commission. And Burke now finds himself Minister for sustainable population.

“Sustainable population” offers a false dichotomy between population growth and sustainability — the same sort of contrast as, indeed, it’s closely related to, the false dichotomy between economic growth and environmental values.

In fact one of the more amusing aspects of the population debate is watching environmental groups, who are normally the first to insist that there need be no tension between strong economic growth and environmental and climate action, insisting that we have too many people already and can’t have any more if we want to preserve our environment.

The “sustainable population” rhetoric is aimed at conning voters into assuming that in policy terms we can’t walk and chew gum, that we can’t get our planning priorities right, we can’t put in place policies that facilitate population growth while addressing environmental, infrastructure and quality-of-life issues.

Charging for externalities will get us a long way down the track to resolving this alleged tension — whether they’re carbon emissions, congestion or infrastructure costs. And letting markets work properly — in areas such as housing supply, or properly charging for carbon — rather than imposing ever greater layers of regulation can’t do any harm either.

The political retreat from high immigration might make voters happy now but voters in future decades will pay the price in lower economic growth, a smaller workforce and higher prices. It’s the ultimate in vision-less leadership.