Sam Altman recently stated that OpenAI is no longer constrained by compute but now faces a much steeper challenge: improving data efficiency by a factor of 100,000. This marks a quiet admission that simply scaling up compute is no longer the path to AGI. Despite massive investments in data centers, more hardware won’t solve the core problem — today’s models are remarkably inefficient learners.
We've essentially run out of high-quality, human-generated data, and attempts to substitute it with synthetic data have hit diminishing returns. These models can’t meaningfully improve by training on reflections of themselves. The brute-force era of AI may be drawing to a close, not because we lack power, but because we lack truly novel and effective ways to teach machines to think. This shift in understanding is already having ripple effects — it’s reportedly one of the reasons Microsoft has begun canceling or scaling back plans for new data centers.
[link] [comments]