Building security testing tools for AI agents for the past few months and realised teams build the agent then test it for accuracy and test it for hallucinations.
Do you test for prompt injection, system prompt extraction, data exfiltration until it breaks in production.
I used to think the LLM's model is smart enough to handle it and that was my initial security plan.
What are your experiences and
- Do you test for malicious inputs before shipping?
- If yes whats does that process?
- If no what would make you start?
[link] [comments]