In my previous article, I explored the idea of an AI researcher that reads what matters — not everything. Today, let's zoom out and address a deeper issue.
We get the impression that more data = better AI, and that it will become more performant. Each new generation of models is bigger, hungrier, more expensive than the last.
But here's the question that seems most relevant to me: what if we've been solving the wrong problem?
The buffet problem
Imagine you're studying for a medical exam. You have two options:
Option A: You're given access to every medical textbook available, every blog post, every Reddit thread, every Facebook comment about health. Millions of pages.
Option B: A senior doctor hands you 200 carefully selected pages — the essential knowledge, structured, verified, in the right order.
Which student passes the exam?
Current AI takes Option A. It swallows the entire internet. And here's the dirty secret: it works just well enough to be impressive, but not well enough to be reliable.
What actually happens inside
Let me clarify something. When you ask an AI a question, here's what does NOT happen:
•It does not "think" about your question
•It does not consult a database of verified facts
•It does not reason through the logic
Here's what DOES happen:
•Your text is chopped into tokens — pieces of words
•Each token becomes a vector, a list of numbers
•The model calculates statistical relationships between these vectors
•It estimates the next most probable token
•Then the next. Then the next.
The entire response is generated one token at a time, by probability. Not by understanding. Not by reasoning. By math that says "after these words, this word is the most likely to follow."
That's why AI "hallucinates." It's not making mistakes in the human sense — it's doing exactly what it was designed to do: produce the most statistically probable sequence of words. Whether that sequence is true or false is irrelevant to the model.
The real cost of eating everything
This "eat everything" approach has consequences that go beyond accuracy.
Energy. Training these models consumes staggering amounts of electricity. We're talking about the kind of energy bills that could power small towns — not for months, but during a single training run. And every time a new version comes out, the bill gets bigger. We're burning resources at an industrial scale to generate plausible-sounding text.
Noise. When you train on the entire internet, you train on garbage too. Misinformation, outdated content, contradictions, spam. The model has no mechanism to distinguish a peer-reviewed paper from a blog. It's all just tokens with statistical weight.
Diminishing returns: Making models bigger produces smaller and smaller improvements. We're pushing harder for less. The answer to "how do we make AI better" can no longer simply be "make it bigger."
The human is different
Current AI works like a sponge that absorbs data, whereas humans process information selectively. Yet, I get the impression that we are still trying to improve AI by simply injecting more and more data.
And then there's quantum
Quantum computing aims to accelerate the training of models. What takes weeks today could, in the future, take only a few hours.
But this raises a fundamental question: if we give more and more power to systems that are still imperfect in how they filter, reason, and prioritize information, are we solving the problem… or simply amplifying their limitations? Giving a faster engine to a car with no steering wheel doesn't make it a better car. It makes it a disaster.
The real question isn't "how do we train faster." It's "what are we training on, and how does the model process it." Until we address that, more power — even quantum power — is just a bigger buffet for a system that's already obese.
The problem is clear. The solution is yet to be found. You might think I'm highlighting the limits of AI without offering solutions — I have leads, but they're secret.
