The way AI use is being charged for currently I feel is currently broken.
- I pay for input tokens. OK, that makes sense. the more tokens I send, the more I should pay since it's more work on the compute side.
- I pay for output tokens but 80-95% of those tokens are thinking budget. I don't care about the thinking your model does. I just care about the answer. Can we not be charged just for tokens that are useful to me?
But here's the part that really doesn't sit right: the meter is unauditable for the thinking tokens especially for the labs which hide the thinking. When a provider hides 90% of the output and then charges me per token for it, that's pure trust-me billing. The provider controls how long the model thinks, profits linearly from more thinking, and hides the evidence. That incentive structure would not fly with any other metered utility. Your electric company doesn't get to say "trust us, you used 900 kWh, but which appliances used it is proprietary."
So the way I see it, labs have two honest options:
Adjust the price of output tokens to account for how much the model thinks, or
Stop calling it output-token pricing. What I'm actually paying for is compute, and tokens are just the meter. If labs said "reasoning is billed as compute at $X" that would at least be honest, even if I still couldn't see it. The dishonesty is in labeling hidden compute as "output" — output is, by definition, the thing I fin useful as the output.
Is that too much to ask?
What's your take on this?
[link] [comments]