I spent the first month of running a small food company on AI agents (small programs that each handle one piece of the work) assuming the risk was the software getting things wrong. It wasn't. The risk was a program that could read everything and write anything.
Early on I had agents with broad database access, because it was faster to wire up that way. Nothing catastrophic happened, but I noticed I got nervous every time I handed off a new task, because I could not say for certain what the program could touch versus what it could only see.
The fix was boring. Give every new system its own database, its own tables, and put anything that leaves that sandbox behind a queue a human has to approve. Read access to the real data. Write access only to its own isolated store. Nothing goes out the door without someone flipping a switch.
Software quality was not the lesson. The programs are decent at the actual work now, better than I expected going in. What burned time was not thinking about the boundary first. Every hour I spent building read and write walls after the fact was an hour I did not spend improving the work itself.
The hype I would push back on: the idea that a smarter model fixes the risk on its own. That is not what fixed anything for me. A hard wall around anything outward facing did.
Curious if anyone else running operations on AI landed on the same read everything, write nothing outside your own sandbox rule, or found a different boundary that mattered more.
[link] [comments]