Wasting hundreds on API credits with runaway agents is basically a rite of passage at this point. Here’s mine.

I'm starting to think this is a shared experience now. Everyone I know building with agentic AI has the same quiet confession tucked somewhere in their git history. The weekend they left an agent running unsupervised. The invoice that arrived on Monday. The forensic work trying to figure out what it actually did.

Mine was over 400 dollars across two days. My agent rephrased the same research task to itself for forty eight hours and produced nothing. Felt like I'd been mugged by a very polite philosopher.

After the third time this happened I stopped being annoyed and started being curious. What is the agent actually thinking during one of these loops. Can I see it happen. Can I catch it before the Monday invoice.

So I built a dashboard. It turned into a 3D visualisation of the agent's working memory in real time, with deliberate colour coding because I wanted to understand what was going on at a glance.

Here's what the colours mean, because this is the part that took me longest to get right and I haven't seen anyone else frame it this way.

Nodes are beliefs the agent is holding. The colour of a node is its health. Bright green means the belief is fresh and actively being used in reasoning. Soft blue means it's older but still relevant. Grey means it's fading and likely to be forgotten on the next cleanup.

Edges are connections the agent has drawn between facts. Edges pulse softly when the agent cross references two beliefs to make a decision. A tight cluster pulsing the same edges over and over is the visual signature of a loop, and you can see it long before the invoice notices.

The whole graph also carries an overlay tint. Green is healthy. Yellow is "the agent is starting to overthink, keep an eye on this". Orange is repeated self referencing, probably looping. Red is stop the agent now, it has burned through its reasoning budget and is no longer making progress. Red is what would have saved me the forty seven dollar weekend if I'd had this running at the time.

Here's the thing I didn't expect. A looping agent doesn't look chaotic. It looks calm. A small cluster of three or four nodes with the same two edges pulsing in rotation, like a tiny orbit. The first time I watched a real loop play back with colour, I understood why I hadn't caught it by reading logs. The logs looked busy. The graph looked bored.

I've been sitting with this a few weeks now and I'm increasingly convinced agent observability is about to become its own category. We spent the last decade figuring out how to watch microservices. We're about to spend the next decade figuring out how to watch agents, and I don't think it's going to look anything like the first one.

Anyway, enough from me. Genuinely want to hear the rite of passage stories. What's the dumbest way an autonomous agent has eaten your API budget. Mutually assured commiseration in the comments.

www.octopodas.com

I would love peoples feedback!