Why “Good Intentions” Should Not Be Relevant When Discussing AI Security Policy
Intent-based security for agents has arrived heavily recently to the conversation, and I think we are giving it far too much credit.
An agent’s intent can help explain what it is trying to do. It is not authorization. Security policy needs to answer a more concrete question: is this specific action allowed, under the rules that apply, before it executes? If the answer depends on whether a model believes it means well, you do not have a security control. You have an opinion.
We do not give employees a pass for good intentions
Imagine an accounts payable employee sends a payment to the wrong bank account because they were trying to resolve an urgent supplier issue. Or an employee emails a sensitive spreadsheet to a personal account because a colleague needs it quickly.
The employee may have meant well. The payment is still wrong. The data still left the company. The policy violation is still a violation.
We do not say, “The employee’s explanation sounded reasonable, so the transaction must have been authorized.” We check the relevant rules, permissions, approvals, and records. If the employee was not authorized to make the payment, good intentions do not change that fact.
So why would we hold agents to a lower standard than people?
An agent that moves money to the wrong account causes the same operational problem whether it meant harm or not. Intent might matter later when you investigate what happened. It does not make the action safe before it happens.
Intent is not a control
Intent-based approaches infer what an agent is trying to accomplish, then assess whether a command seems consistent with that purpose. Useful context. Not a dependable authorization decision.
Intent is inferred, subjective, and can be shaped by a misleading request, incomplete context, or an attacker. After an incident, “the agent thought this matched the user’s intent” is hard to verify independently.
A deterministic control asks about the action itself. Is this agent allowed to modify this record? Does the transaction exceed a value threshold? Does it cross a segregation-of-duties boundary or touch a regulated system? If the policy requires approval, did the named approver approve this exact action?
Those questions have checkable answers. “Did the agent mean well?” does not.
The prompt may come from another agent
Here is where intent checking gets especially slippery: agents are increasingly generating prompts for other agents.
Suppose one agent breaks a business request into tasks and tells another agent to issue a refund or change a customer record. Whose intent are we checking now? The agent that wrote the instruction? The human’s original request? The interpretation two steps upstream?
If the action is judged against a prompt written by another model, you are checking one model’s output against another model’s interpretation. Both can be wrong. That is not authorization. It is a vibe check between two systems.
A prompt describes what someone asked for. It does not establish that the action is permitted.
Compliance needs evidence, not a story
Legal and governance frameworks differ, but they share one operational demand: define controls, assign responsibility, and demonstrate how those controls work.
The Sarbanes-Oxley Act requires internal control over financial reporting, and segregation of duties exists so one person cannot both initiate and approve an improper transaction. An agent that can both create and approve a payment raises the same control question. A good explanation of intent does not resolve it.
The GDPR demands purpose limitation and accountability for personal data. The EU AI Act expects risk management, record-keeping, and human oversight for covered systems. The NIST AI RMF and ISO/IEC 42001 all push the same way: documented, demonstrable control.
“The model believed this was appropriate” is not something you can show an auditor. A policy decision tied to the action, policy version, identity, timestamp, and any required human approval is.
And monitoring does not fix this. Detecting a violation after the payment goes out is not the same as stopping it. The control has to sit before execution.
Good intentions belong in the investigation, not the policy
Intent still has a place: explaining behavior, triage, investigation. It should not decide whether a business action is authorized.
The control has to be explicit: evaluate the action against policy, decide before execution, record the result. Allow what meets the rules. Block what violates them. Escalate what needs a person.
The question is never, “Did the agent mean well?” The question is, “Was this action authorized before it executed?”
Guardrails on a highway do not care about the driver’s intentions. We do not need agents to have better intentions. We need controls that work even when those intentions are wrong.
Related reading: What is an AI control plane?