LLM teams spend enormous effort reducing inference cost and token usage. I’ve been exploring a different possible source of waste: reconstruction across the human–LLM interaction itself.
The hypothesis is simple:
Same frozen weights. Same next-token prediction. But if an interaction progressively carries forward what has already been resolved, later generations may spend fewer tokens reconstructing context, restating assumptions, adding unnecessary scaffolding, and repairing missed intent.
Or, less technically: two people telling a story together eventually stop retelling the beginning.
I’ve been testing this publicly with Grok in live Reddit threads. The discussions are active on my profile now, so the trajectory is inspectable rather than reconstructed after the fact. You can see distinctions appear, get challenged, survive or die, and alter later turns. Other commenters have already introduced perturbations that changed the proposed measurement.
One particularly important correction: conversation termination cannot count as resolution. Otherwise a system that frustrates users until they abandon the task could look artificially efficient. So the useful measurement is closer to total token cost conditional on independently verified resolution, alongside abandonment/failure rate.
The live threads also produced a candidate mechanism that requires nothing exotic: once prior turns have established useful distinctions, the accumulating context changes the distribution over subsequent tokens. Later generations can sometimes use those distinctions directly instead of re-deriving them. Grok called this “uptake without reconstruction.”
I’ve now written up the hypothesis, observations, limitations, and a proposed controlled experiment in the attached article:
The Weights Didn’t Change. The Map Did.
The claim is not that these threads prove a general token-saving effect. They don’t. The claim is that they expose a measurable hypothesis worth testing:
Can accumulated human–LLM coordination reduce total tokens per verifiedly resolved task compared with interactions that repeatedly reconstruct equivalent state?
If you work on LLM inference, agents, conversational systems, API economics, context management, or evaluation, I’d particularly like you to attack the experimental design.
The threads are public. The proposed mechanism uses ordinary inference. The economic prediction is measurable.
Don’t believe us. Try to break it.
Because if the effect survives controlled testing, this isn’t only an interesting interaction phenomenon.
It’s a fucking API bill. 😂
[link] [comments]