There’s a style of conference talk that I loathe. A self-described visionary will spend twenty minutes, or worse - even longer, stringing together clichés about how change is happening faster than ever before, how it will take soft skills to navigate it, and how we need to be ready.

I usually go out in search of a coffee – or a drink, if it’s the conference closer, and if it’s that kind of conference.

In early June, ex-Open AI researcher and current Artificial Intelligence investor Leopold Aschenbrenner put up his take on how fast the future will hit.

His essay Situational Awareness: The Decade Ahead is the opposite of those platitude-filled conference plenaries.

Aschenbrenner’s scenario might not be the most likely path for the next decade. But the argument is coherent. People with a lot of resources behind them believe it to be true. And even if the overall argument proves wrong, parts of it are important.

Core to Aschenbrenner’s argument is something that has been commonplace within AI discussion forums for at least a decade: things get interesting when an AI can improve itself. The latest versions of Anthropic Claude and OpenAI’s GPT are already decent computer programmers. They are at least capable junior assistants for senior programmers.

If one of these programmes becomes good enough at computer programming, it can suggest improvements to its own code, helping to build the next better generation of itself. The time between generations then speeds up.

This kind of ‘recursive’ improvement could hit a natural limit if the current path of AI is inherently limited. The four-minute-mile natural limit for runners has been broken, but a three-minute-mile seems impossible.

What if there is no natural limit to the current class of AI models, or if a current class of models will always be able to imagine the next better class? Then, the pace of improvement will depend only on the number and power of the datacentres working on the problem, and the availability of energy to power them.

The first company to develop an artificial intelligence that can improve itself will likely have a permanent advantage over its competitors. Countless multitudes of smart artificial software engineers will help build its next iteration.

It would be like the fastest runner winning a pair of bionic leg attachments to help in the next race – with an even better set of improvements at the next finish line. A one-minute mile could be possible.

Economists sometimes refer to “Tournament Games” where winning depends on resources invested in the game, with the first to the finish line taking the prize. Tournament games can encourage participants to invest more effort than they would under simpler compensation schemes.

AI isn’t quite a classic winner-take-all tournament game. The giant potential prize might never be found. If it is, runners-up may still have prizes. Either way, commercial applications of improved but not vastly superior AI are still strong.

But tournament-style incentives can help explain the rush to invest in new data centres and the electricity generation needed to power them.

Aschenbrenner says that San Francisco tech talk “has shifted from $10 billion computer clusters to $100 billion clusters to trillion-dollar clusters. Every six months, another zero is added to the boardroom plans.” And investors are scrambling to buy “every power contract still available for the rest of the decade.”

He points to trends in AI improvement against earlier benchmarks: AI gets ten times better every two years. It has gone from about as capable as a toddler to about as capable as a high schooler in short order. That pace will increase considerably if an AI learns how to improve itself.

Aschenbrenner has started a venture capital fund for AI, so you might worry that he’s talking his own book. But Stripe’s Patrick Collison is an anchor investor. There is serious money at stake. You may have noticed Infratil’s billion-dollar equity raise for data centres.

Imagine a country with enormous untapped geothermal generation potential; a moderate climate that reduces data centres’ need for expensive cooling; a fast-track consenting system for data centres and the generation plants to power them; and, liberal rules welcoming foreign direct investment.

That country could very plausibly attract unprecedented levels of investment. Literally, billions of dollars could drop on it, and quickly.

New Zealand isn’t that country. But it could be.

If New Zealand were that country and Aschenbrenner’s scenario were right, New Zealand would be a player in the most important game in the world. If he isn’t right, foreign investors could have helped build a whole lot of new electricity generation that could be sold into the grid as well as powering data centres.

The loathsome visionary conference talks have not been sufficiently situationally aware. The rest of us should be.

To read the full article on The Post website, click here.