Is the technological singularity being canceled? Has the data for training AI run out?

“We have almost exhausted the cumulative volume of human knowledge <…> in the field of AI training. And this happened last year,” — stated Musk in a conversation with Stagwell CEO Mark Penn on the social network X.

These remarks came just days after Sutskever, who contributed to the creation of ChatGPT, mentioned at the annual NeurIPS event that companies have reached a data peak — and there will be no more.

If this is true, compensating for the shortfall can be achieved through synthetic data, meaning content generated by generative AI models themselves. However, this approach is far from ideal.

Researchers from Stanford University and Rice University previously discovered that models trained on AI-generated data, whether text or images, tend to “go haywire” after five training cycles.

In November of last year, it was revealed that OpenAI was facing challenges with its new model Orion: it turned out to be less advanced than the company had hoped. Similarly, the latest version of Gemini from Google did not significantly surpass its predecessor. Meanwhile, Anthropic has postponed the release of its model Claude altogether.

It’s worth noting that Musk and OpenAI CEO Sam Altman had previously claimed that AI is close to surpassing the intellectual capabilities of individual humans, and eventually the intelligence of all humanity combined.