AI agents can search systems, call APIs, change files, and trigger business actions. That power makes them useful, but it also creates a large security boundary. A safer design treats an agent as a decision-maker that must work through narrowly scoped tools, independent authorization checks, and clear approval rules.
Start With the Agent’s Action Space
Begin by listing everything the agent can read, change, send, approve, or delete. Include databases, cloud accounts, source repositories, email systems, file stores, and external APIs. OWASP recommends documenting every tool, permission, and external integration because an agent that only returns text has a different risk profile from one that can execute code or modify production systems. (OWASP AI threat modeling guidance)
Use that inventory to define a small set of explicit capabilities. For example, an incident assistant might read monitoring data and create a draft ticket, but it should not disable an alert, change a firewall rule, or restart a production service. A coding assistant might read one repository and open a pull request, but it should not receive unrestricted shell access, production credentials, or permission to merge its own changes.
Enforce Least Privilege Per Tool
Least privilege means granting only the access required for the current task. Apply it at several layers: the agent’s tool list, each tool’s arguments, the data returned by the tool, and the external identity used to perform the action. OWASP advises using per-tool permission scopes such as read-only access, resource restrictions, and separate tool sets for different trust levels. (OWASP AI Agent Security Cheat Sheet)
These limits should not depend on the model. The execution service should validate each requested operation against a policy before calling the underlying system. OWASP says authorization must be enforced in the execution component and that approval must be checked against the current actor and exact tool call. The service should therefore check the user, agent identity, target resource, action type, data scope, and current approval state for every sensitive request.
Credentials also need narrow boundaries. Use short-lived credentials with limited scopes whenever possible. OWASP’s Secure Coding with AI guidance recommends auditing connected MCP servers, maintaining an allowlist of approved servers and tools, reviewing tool descriptions for hidden instructions, validating tool arguments, and restricting coding agents to the tools they actually need. (OWASP Secure Coding with AI Cheat Sheet)
Separate Decisions From Execution
Once tools and permissions are defined, separate the agent’s proposed action from the decision to execute it. A policy layer should decide whether the action can run. This separation limits the damage from prompt injection, malicious documents, poisoned tool results, and ordinary model mistakes. Treat external content as data rather than trusted instructions, and validate structured outputs against a schema before they reach an execution system.
Classify actions by impact. Reading a public status page may be low risk. Updating an internal ticket may require an authenticated session. Sending an external email, changing infrastructure, approving a payment, deleting data, or modifying permissions should require explicit human confirmation and additional controls. OWASP recommends separating decision-making from execution, binding approval to the exact action, and requiring step-up authentication for critical operations such as privilege changes, payment initiation, bulk deletion, and production deployment. (OWASP high-impact action guidance)
Keep each approval tied to one exact action. Include the actor, tool name, target resource, normalized parameters, timestamp, and expiry in the approval record. Use short-lived authorization artifacts and replay protection for irreversible operations, as OWASP recommends.
Make Agent Activity Auditable
With the authorization boundary in place, log each request, decision, tool call, authorization result, approval, and returned result. The record should identify the human or service that initiated the work, the agent version, the model provider, the tool version, the target resource, and the time of execution. Store enough context to reconstruct what happened without placing passwords, API keys, or unnecessary personal data into the log.
Monitoring should look for unusual behavior, not just failed requests. Useful signals include sudden increases in tool calls, access to a new data class, repeated authorization failures, unexpected network destinations, unusually large responses, and attempts to expand the agent’s permissions. OWASP recommends structured logging for high-risk actions, anomaly detection, cost and retry limits, and adversarial testing after changes to prompts, tools, memory, retrieval, or model providers. (OWASP AI Agent Security Cheat Sheet)
NIST’s Generative AI Profile places these practices within a broader risk-management process. The profile is a voluntary companion to the AI Risk Management Framework. It helps organizations address trustworthiness across the design, development, use, and evaluation of generative AI systems. (NIST AI RMF Generative AI Profile)
Test the Boundary Before Launch
Before launch, security testing should attempt to make the agent exceed its authority. Use malicious instructions in documents, emails, web pages, and tool responses. Test whether the agent can access another user’s data, call an unapproved tool, alter a parameter after approval, reuse an expired authorization, or continue operating after a denial.
Repeat those tests after changing the model, prompt, tool schema, memory store, retrieval source, policy engine, or identity provider. OWASP recommends structured adversarial testing before production and after material changes to prompts, tools, memory, retrieval, or model providers. A permission boundary is part of the application’s security design, so it belongs in code review, automated testing, deployment controls, and incident response. The goal is to keep useful work inside a narrow, visible, and revocable authority.
References
- OWASP Cheat Sheet Series. (n.d.). AI agent security. OWASP Foundation. https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html
- OWASP GenAI Security Project. (n.d.). AI threat modeling. OWASP Foundation. https://genai.owasp.org/glossary/ai-threat-modeling/
- OWASP Cheat Sheet Series. (n.d.). Secure coding with AI. OWASP Foundation. https://cheatsheetseries.owasp.org/cheatsheets/Secure_Coding_with_AI_Cheat_Sheet.html
- OWASP Cornucopia. (n.d.). Agentic AI (AAI9). OWASP Foundation. https://cornucopia.owasp.org/edition/companion/AAI9
- Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., & Roberts, K. (2024, July 26). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.600-1
