The Shift to Taxonomy-Driven Risk Discovery

Traditional red teaming for LLMs often relies on manual prompt engineering or simple adversarial attacks. However, agentic AI—systems that execute multi-step plans, interact with tools, and maintain persistent state—introduces a broader attack surface. This framework proposes a taxonomy-driven approach to black-box red teaming, which categorizes potential failure modes into structured domains. By mapping these risks, developers can automate the generation of adversarial test cases that specifically target the agent's decision-making logic, tool-use capabilities, and long-term planning rather than just its linguistic output.

Automating Black-Box Adversarial Testing

The core of this methodology is the decoupling of the red teaming agent from the target system. By treating the target agent as a black box, the framework generates adversarial inputs designed to trigger specific failure states identified in the taxonomy. This includes testing for:

• Tool Misuse: Forcing the agent to execute unauthorized or malicious tool calls by manipulating the environment or the agent's perceived context.
• Goal Hijacking: Inducing the agent to deviate from its primary objective through prompt injection or state corruption.
• Planning Failures: Exploiting the agent's reasoning chain to force inefficient, dangerous, or infinite loops in task execution.

By automating these tests, the framework allows for continuous security evaluation during the development lifecycle, ensuring that as agent capabilities expand, the safety guardrails scale accordingly. This approach emphasizes that securing agents requires testing the entire execution loop, not just the underlying model's safety alignment.