Most agentic AI projects never make it past the pilot stage. But the ones that do usually get a security review that amounts to "the model provider handles that."
They don't, once an agent has permission to actually do things. OWASP's Agentic Security Initiative put out a Top 10 for this last year and it's basically a list of failure modes nobody's treating as failure modes yet; goal hijacking, tool misuse, an agent's own identity and privileges getting abused.
I've watched a team hand an agent write access to a IT ticketing system, a CRM, and internal docs, call the security review done because the model card mentioned jailbreak resistance, and move on.
The stuff that actually bites people: nobody's logging agent decisions in a way that would hold up if something went wrong later. Nobody tests what happens when a tool call comes back malicious, only what happens when the user's prompt does. And the permissions are usually looser than what you'd give a new hire with the same job.
There's a newer standard trying to formalize this, AIUC-1, it's mapped against ISO 42001 and NIST's AI RMF instead of starting from scratch, which is a decent sign the space is maturing. Still moving slower than the agents that are actually shipping.
I don't think this is unsolvable. Least privilege, real logging, segmentation, none of it is new. It's just not getting applied here yet.
If your org's gotten an agent into production with an actual security review behind it, I'd like to hear what that looked like.
[link] [comments]