I used to think “better prompt” would fix everything. Then I watched my system break because the agent returned: Sure! { "route": "PLAN", } So now I treat agent outputs like API responses: Strict JSON only (no “helpful” prose) Exac…
Since it’s Christmas, I ended up thinking through a Christmas-themed use case, mostly as a way to explain how I approach agent design beyond the foundational layer. The theme itself doesn’t matter much. It just gives you a nice mix of: vague, emotiona…
Something I don’t see talked about enough: How much time AI builders spend repeating setup work. Every project: – Pull data – Clean it – Structure it – Validate outputs – Fix edge cases – Re-run when something changes None of this is the interesting pa…
I used to think building an AI feature was mostly model choice + prompts. Then we shipped one. What went wrong: The assistant started giving different answers to the same questions. We didn’t change the model. We didn’t change the UI. It looked like th…
After building RAG + multi-step agent systems, three lessons stood out: Good ingestion determines everything downstream. If extraction isn’t deterministic, nothing else is. Verification is non-negotiable. Without schema/citation checking, errors sprea…
In our multi-step workflow where each step depended on the previous one’s output, problems we observed were silent errors: malformed JSON, missing fields, incorrect assumptions, etc. We added verification nodes between steps: check structure check sch…
We were debugging a workflow where several steps were orchestrated by an AI agent. At first glance, the failures looked like reasoning errors. But the more we investigated, the clearer the pattern became: The tools themselves were unreliable. Examples:…
We were debugging a pretty complex automation pipeline and kept blaming the model for inconsistent behavior. Turns out… the model wasn’t the problem. The actual failure points were architectural: Tasks weren’t specific enough -> different agents in…
When evaluating an agent system that changes its behavior as tools and planning steps evolve, it can be hard to choose metrics that actually explain what went wrong. We tried several complex scoring schemes before realizing that a simple grouping works…
Agent systems change shape as you adjust tools, add reasoning steps, or rewrite planners. One challenge I ran into is that the JSON output shifts while the evaluation script expects a fixed structure. A small structural drift in the output can make an …
