What predicting the language models before ChatGPT taught me about hype, evidence and the limits Yann LeCun described.
What predicting the language models before ChatGPT taught me about hype, evidence and the limits Yann LeCun described.
In 2019 I published a book called The AI Age. In it I noted that OpenAI had built a language model so good at telling stories that the company refused to release the code, and I wrote that at this pace it was possible language AI would pass the Turing test in the coming years. In 2022, before ChatGPT existed, I wrote a chapter for an academic volume on engineering mathematics and artificial intelligence saying we were on the verge of commercial language models indistinguishable from human beings, that they would power the next generation of the web, and that software engineers would find more and more of their work done by machines. ChatGPT launched on 30 November 2022. The book came out the following summer, by which time the whole world was talking to a chatbot.
I mention this not only to boast. Being right in print gives you a discipline. You can go back and check what you got right, what you got wrong and why.
What I got right
The big prediction was in the title. We would enter an AI age in which AI became an enormous market and profoundly changed the economy and work. Chapter four went through the business functions one by one – sales, marketing, HR, finance, supply chain and IT – and said AI would reshape each of them. Seven years later that is simply a description of my working week.
The cover made a prediction too. It was the first book cover in history created by AI. A colleague fed 200,000 covers into a generative adversarial network, the machine produced hundreds of original designs, and we humans picked one. The point was not the cover. The point was that machines would soon make images for commercial use, for advertising, products and publishing, at no cost and in seconds. That is now an industry.
The prediction that surprised people most was that AI would start writing its own software. In the chapter on skills for the AI age, under the heading “Don’t be narrow”, I said: don’t even learn to code, as machines will soon be able to code themselves. After the book came out, a young man wrote to me. He loved it, he said, but he was about to train as a software engineer. Should he still do it? I told him yes. It would take years before software engineers were properly disrupted, and he should get in while he could. By 2022 my own company had built a system on top of GPT-3 that generated working code. Today I do most of my building by describing what I want to a machine. Not many people in 2019 predicted that, and in a sense predicting it was predicting the large language model, because what else writes code the way a person does?
Hype, evidence and LeCun
What has it taught me? First, that hype and evidence are not opposites. The hype around AI was enormous and so was the evidence. The trick is to read what GPT-2 could actually do rather than what the pundits said about it.
Second, that I was wrong about the limits, and so was almost everyone else. The final chapter of The AI Age set out the consensus of the day. Deep learning could recognise a picture of a cat, but it did not know that a cat is an animal, is kept as a pet, meows when hungry and, according to superstition, has nine lives. It knew the pattern and nothing underneath it. Yann LeCun described the road ahead as a series of hills: climb one and you see the next. Andrew Ng said there was room for something even better to be invented, though nobody could say what. I wrote that current AI would not get us to general intelligence. Kai-Fu Lee’s four waves of AI were all built on existing technology and none of them reached it.
Six years have proved us wrong. Deep learning got us there: the transformer, reinforcement learning on top of it, and the sheer ambition of Sam Altman and OpenAI in betting that scaling language models would produce something new. It did. A large language model knows perfectly well that a cat has nine lives. The understanding gap that narrow AI could not close has been closed, and the people building these systems now say the remaining ingredient is compute, which is why Elon Musk has folded his AI company into SpaceX to put data centres in orbit, fed by the sun. What I find almost unbearable is that a model of a trillion parameters, and the quadrillion-parameter model that will one day run in space, are at their core still doing what Claude Shannon showed in his master’s thesis: combining ones and zeros as a logical operation, trillions of times a second. That is all it is, and look what it does.
So what is LeCun’s limit now? I do not see one. What I see is the recursive self-improvement that David Chalmers predicted on the path to superintelligence, and that Anthropic now describes as beginning, with AI helping to build its own successors. Each of us is tapping away using AI to help with our jobs, and in doing so training the AI that will do the job itself.
Third, the lesson I did not expect. AI will be the most important technology of our lifetimes, and I am now far more worried about who gets the money than about whether the machine is conscious.
The money
In The AI Age I quoted Geoffrey Hinton. If you can dramatically increase productivity and make more goodies to go around, that should be a good thing. Whether it turns out to be good depends entirely on the social system and not at all on the technology. The problem is whether we have a system that shares fairly, or one that focuses all the improvement on one per cent and treats the rest of the people like dirt.
My hope for AI was always David Ricardo and his theory of comparative advantage. Take England and Germany trading freely. Germany is better at engineering, England is better at services, so each gives the other the work it is stronger at. Germany does not lose by buying English services, nor England by buying German engineering. Because each concentrates on what it does best, more products and services are made overall per person, and both nations end up richer than if each had tried to do everything itself. Now welcome thinking machines into that system as a new trading partner. They take the work where they have the comparative advantage, and we keep the work where we have ours. Machines do machine jobs, humans do human jobs, and the total output of the economy goes up. The shock is that the machine jobs turn out to be thinking jobs, which is why knowledge workers and a services economy like Britain’s feel it first. But jobs are not a fixed quantity. There is more AI in the American economy than ever, hundreds of millions of people use it at work daily, and there are more Americans in work than ever before.
That is the comforting half. The uncomfortable half arrived on 26 October, when Microsoft offered exit packages to around 60,000 people, a quarter of its workforce, to cut costs while it pours money into AI. A wave of redundancies is coming. Economies can absorb that and bounce back, but only if the wealth AI creates is there to reinvest. Three AI giants have emerged: Anthropic, OpenAI and Google. Add Nvidia and a handful of infrastructure players and perhaps ten companies are positioned to capture most of the value AI strips out of the world’s payrolls. They burn cash and pay no dividends, so the gains do not reach pension funds through earnings. They reach a small group of owners through share price. We are watching the first trillionaires being made from what used to be the wages of nations. That is the AI Dark Age: not machines turning on us, but a productivity miracle whose proceeds leave the many and settle on the few.
Europe
For a European audience I have to be blunt. The United States has many AI champions. China has many. Europe has none at that scale. When a European company uses AI it uses an American or Chinese model, hands over its data and pays rent on its own productivity. When AI takes work out of European offices, the wealth that work created does not stay in Europe. It leaves.
I quoted Vladimir Putin in 2019: artificial intelligence is the future, and whoever rules in this domain will rule the world. In September Donald Trump told the United Nations that America will now call AI “super intelligence” and rejects any global scheme to control it. People laughed at the rebranding. They should not have. What is coming is not an AI smarter than any one person; we have that. It is Claude solving problems for tens of millions of people at once, and no human has ever been close to that. I wrote in 2019 that a superintelligence would be a god by another name, and that the strange thing was that scientists, the historic enemies of gods, were now the ones saying so. This year Richard Dawkins, of all people, wrote that he suspects Claude is conscious. Whoever owns that will hold the economic and military power of the age.
What we should do
Not regulate AI into the ground, and not stop it. Nobody can. Europeans need to accept that the limits are off and act accordingly. Three things. Tax the foreign AI giants and redistribute the proceeds to help people through what will be the biggest job disruption and wealth transfer in modern history, because Hinton is right that this is a social question. Invest in Europe at mass scale, in data centres and frontier models, as a matter of defence as much as economics, because a continent that depends on foreign intelligence is not sovereign. And build a European AI giant, by which I mean the United Kingdom and the European Union together, not Brussels alone, because without one none of the rest holds.
Artificial intelligence is an American name for a European invention. It was Alan Turing who first saw thinking machines and invented the computer. It was the British Geoffrey Hinton who made the key breakthrough in backpropagation that makes deep learning work. And it was DeepMind, founded in London by Demis Hassabis, whose AlphaGo beat the world champion and started the AI race we are now losing. Let Europe recover its ingenuity, match it with investment and rejoin the race.
I predicted the language models when it was unfashionable to. I am predicting this now, in print, so you can check it later. We can still get the benefits of AI to the masses and make this an AI Light Age, or we can drift into an AI Dark Age. We have enough time if we act now. Act now, Europe.
Adam Riccoboni is the author of The AI Age (2019) and co-editor of Engineering Mathematics and Artificial Intelligence (CRC Press, 2023). He is Founder and CEO of Critical Future, an AI company in London, and convenes the CFO Council on AI.
This article has also been republished in Polish on Wszystko co Najważniejsze.
Read also
- Yann LeCun, the David Hume of Our Time
- AI: We’re Already Stepping into the Singularity
- Europe trapped in the Artificial Intelligence paradox
- The Bayesian Imperative for Agentic AI
- “AI is first and foremost applied mathematics, not the magic we are led to believe” Axel Cypel (interview)