Hiding In Plain Sight: Abusing Model Assumptions To Deceive & Attack Offensive AI Agents

AI Agents are increasingly being used offensively in an autonomous fashion, with chain of thought and tool chaining available to them. An operator (or ‘human in the loop’) is hopefully still involved to review output at set stages/decisions based on how things have been setup but the agents make and take decisions on their own. With the autonomous move, we see traditional defensive techniques from the ‘90s, like general deception with honeypots and honey tokens being deployed to deceive, deter, or slow down these automated attacks.