at what point do logs and dashboards stop being enough for llm costs?

Hello everyone, currently digging into workflow-layer economics and trying to figure out how people track unexpected runtime spikes at scale.

At an early stage simple margin buffers are fine because volume is bounded. But once you move past basic apps, factors like failed loops, retries, and context window inflation create a ton of cost variance that is hard to forecast or map to clean client billing.

For those running agent or voice workflows in production, or working on complex ai products what do you currently use to understand costs and failures at the individual workflow level?

More importantly, what's something you still can't easily answer with your current setup? Like why did a specific workflow suddenly cost 2x more, or which exact customer trigger is driving the increase? Are you guys just manually digging through raw api logs to catch leakage like infinite loops, or has it not become a big enough issue for your teams yet?

Curious to hear how other teams handle the infrastructure discipline here.