Not all progress is for the best. When I was a young stock exchange reporter, the exchange was a broker co-operative where bids and offers were displayed on a blackboard and sales reported. If a deal was so big it would distort the market, it could be done off-market, in a broker’s office, as a crossing.

Only the highest bid and lowest offer barked out by the brokers in the pit was put on the board. If someone bid more or offered less, the chalkie rubbed off the old one and replaced it with the new one. It was a crazy, noisy old analogue system that worked tolerably well.

Now there are silent, efficient digital screens showing complete “market depth” — all bids and all offers, not just the front ones. Within a couple of years of that innovation in 1997, high-frequency computer traders started exploiting tiny changes in the market depth and front-running using the speed of light. It’s robbery a fraction of a cent at a time.

The high-frequency traders also use the speed of light to trade the arbitrage between the various competing prices for each stock, exploiting the tiny delay before they catch up with each other.

Computer algorithms also act as automatic market makers, making bids and offers at the speed of light to capitalise on a stock’s momentum, either up or down.

Now these three types of computerised trading, performed by banks of computers renting space in the stock exchange’s own data room, apparently represents up to 70% of all trades.

Authorities also decided to break the monopoly of the big exchanges and their (American) market makers by allowing by allowing investment banks to cross any size trades in their offices off-market. These are not called crossings any more — they’re so big they’re called “dark pools”. Now 50% of stocks are traded in dark pools.

These dark pools are growing because brokers don’t trust the market any more because of all the high-frequency computerised trading that’s going on. The exchange can’t stop HFT, and can’t stop renting out space next to their computers to facilitate it, because there are competitors who would do it instead.

Stock exchanges are innovating themselves to death. The business of matching genuine buyers and sellers of company securities in a transparent, regulated market is becoming a smaller and smaller part of what’s going on.

It’s true that not all of this is bad. Dark pools are what I used to call crossings, but without size limits. The ASX yesterday released a submission to ASIC calling for a $25,000 minimum size to be put on dark pool transactions, which seems reasonable.

Even the respectable HFT such as arbitrage and market making is done so fast that it might as well be dark. It simply involves what has always been done, but doing it at the speed of light so no one can see it.

To cut out the bad sort of HFT — front-running — ASX could go back to only displaying the best bid and offer and removing market depth from the screens, which would throw a spanner in the works of the algorithms.

But we now have competition, so they can’t because others would.

It’s time for the regulators to regulate.

*This article was first published at Business Spectator