Every Sunday I ride around the Surrey Hills on my mountain bike. It is one of the few moments in the week when I get a bit of uninterrupted time to think about the messy complexity of the world we live in.
This Sunday I could not stop thinking about a dinner-table conversation with my 16-year-old and my wife.
We had somehow ended up debating whether morality was invented by humanity.
Perfectly normal Saturday-night conversation.
The discussion was roughly this: did humans invent morality as a way of making societies work, or are some of its foundations much deeper than that? Are things like empathy, fairness and cooperation partly biological and evolutionary, with societies then building rules, customs and laws around them?
The more you read about it, the less binary the answer becomes.
There is a strong argument that some of the foundations of morality are very old. Humans are social animals and, at some point in our evolutionary history, cooperation mattered to survival. Groups that could trust each other, share, reciprocate and restrain certain behaviours probably had an advantage over groups built purely around individual self-interest. At the same time, the specific moral rules we live by clearly vary across cultures and history.
So perhaps morality is a little like language. We may be born with some of the underlying capacity, but the particular grammar, vocabulary and rules are shaped by the society around us.
And somewhere halfway up one of the Surrey Hills, I realised I had stopped thinking about morality.
I was thinking about authorization.
We have a language for permissions. We do not really have one for intent.#
For most of the history of computing, authorization has been a reasonably tidy problem.
We identify who you are, understand what role or attributes you have, look at the resource you are trying to access, and decide whether the action should be allowed.
We built RBAC, ABAC, scopes, entitlements, ACLs and policy engines around that model, and for the most part it has served us very well.
But autonomous agents introduce something subtly different.
An agent does not simply execute a predetermined function. We give it a goal, it interprets that goal, develops a plan, chooses actions, calls tools, perhaps delegates work to another agent, and may change its approach as the environment changes.
The awkward bit is that the permission model can remain completely valid while the behaviour becomes completely wrong.
An agent may be perfectly authorized to execute every individual action it performs and still end up doing something that makes you say:
“That isn’t what I meant.”
I think that sentence may become one of the defining authorization problems of the agentic era.
Because humans do not usually communicate by enumerating permissions. We communicate through intent.
We say things like “sort out my travel”, “move some money into savings”, “find me the best supplier” or “fix the customer issue”, and in doing so we assume a huge amount of shared context.
We assume the other person understands the boundaries. We assume they understand the purpose. We assume they know what we would consider reasonable. We assume they know when to come back and ask.
With people, that normally works because there is a great deal of shared social context sitting behind the instruction.
With agents, we cannot rely on that.
Intent is more complicated than a goal#
Sometimes the problem with intent is that we assume the other side will understand what we meant, rather than simply do exactly what we said.
When the instruction is followed perfectly and the intent is missed completely. Image source: GRC.com.
Someone put up a sign saying “POST NO SIGNS.”
And, quite literally, people responded by posting signs saying “NO.”
Technically, they followed the instruction.
They just completely missed the intent.
AI agents can do the same thing, only at rather more consequential scale.
One of the mistakes I think we are in danger of making is treating intent as if it were simply the goal.
Imagine I ask an agent:
Move £5,000 into the investment that gives me the best return.
The obvious goal is to maximise return on £5,000.
But that is probably not the full intent.
I may implicitly mean that it should only use my existing regulated accounts. I may not want to increase my risk profile. I may not want to invest in cryptocurrency. I may not want to trigger a tax event. I may want it to ask me before locking the money away for more than twelve months.
None of those things are necessarily in the original sentence, but they are part of what I meant.
So intent starts to look much richer than a goal. It contains purpose, but also constraints, context, values, assumptions, risk tolerance and sometimes a point at which the system should stop and ask for approval.
That is exactly where the morality conversation became interesting to me.
Morality is not simply a list of good actions and bad actions. We judge behaviour in context. We take into account values, relationships, consequences and circumstances.
Intent is similar.
In my notes I described the problem as needing to understand not only the goal, but also the values within the team, the organisation and potentially society more broadly.
That is where authorization begins to get much more interesting.
If we cannot describe intent, we cannot detect intent drift#
We talk a lot about guardrails for AI agents.
Kill switches are a good example.
It is easy to say that every agent should have one, but the harder question is much more basic: when exactly do we activate it?
That is where things get complicated.
Before I can decide whether an agent needs to be degraded, constrained or shut down completely, I first need to know what acceptable behaviour looks like. In other words, I need some view of the agent’s intended goal, the boundaries around that goal, and the conditions that would tell me it is starting to move away from them.
Only then can I measure what is actually happening against what was intended.
And only after that can I make a sensible decision about what response is appropriate.
Maybe the first response is not a kill switch at all. Perhaps the agent loses access to a particular tool, perhaps its scope is reduced, perhaps it falls back to a more constrained mode, or perhaps a human has to approve the next step. The kill switch is really the end of that chain, not the beginning of it.
So the interesting question is not whether we need a kill switch.
It is how we know when the system has crossed the line that should trigger one.
And that takes us straight back to intent.
Today, in many cases, intent exists mainly as natural language: buried in a requirement, captured in a prompt, written into a product document, or perhaps simply agreed in a meeting.
The problem is that the system does not stand still after that. The model changes, prompts get rewritten, tools are added or removed, orchestration evolves and the agent itself gets refactored again and again.
By the time you are on version 9, version 20 or version 100, a fairly obvious question appears: is this still doing what we originally intended it to do?
And that is surprisingly difficult to answer if the original intent was never captured in a form that could survive all of those changes.
Unless we have first determined what acceptable behaviour looks like, it becomes very hard to know when an agent has moved outside its intended boundary.
And even “good” and “bad” are slightly misleading terms.
An attacker’s agent may be doing an extremely good job of achieving its objective.
So the real question is not whether the behaviour is good in some abstract sense.
It is whether the behaviour remains consistent with the authority, purpose, constraints and values under which the agent is operating.
That sounds much more like authorization.
Authorization needs to move from permission to purpose#
Traditional authorization might evaluate something like:
actor = agent-123
action = transfer
resource = account-456
permission = allowedThat works when the question is simply whether the actor has permission to perform the operation.
But an agentic authorization request may need to carry much richer meaning.
Perhaps the principal is the customer, the actor is an investment agent, the authority is delegated on behalf of that customer, the purpose is to improve return, the amount is capped at £5,000, destinations are limited to approved accounts, risk must remain within an agreed threshold, and human approval is required if any of those boundaries are crossed.
At that point, authorization is not just answering whether access is allowed.
It is preserving the meaning under which the authority was granted.
And those constraints need to survive the execution chain.
If Agent A delegates to Agent B, and Agent B calls Tool C, the original purpose cannot simply disappear along the way.
We need to know what authority was granted, why it was granted, what constraints came with it, how much of that authority was delegated, whether it was attenuated or expanded, and whether the action currently being attempted still serves the original purpose.
That starts to look much more like an Authorization Management System than a traditional IAM product.
Centralise the meaning, not necessarily every enforcement point#
I do not think the answer is one enormous central authorization engine.
We are going to have a heterogeneous environment, and that is probably a good thing.
A firewall rule is a policy. A Cedar rule is a policy. A Rego rule is a policy. An API gateway rule is a policy. A banking transaction limit is a policy. A model guardrail is arguably a policy as well.
The issue is not that we lack policies.
The issue is that many of them lose their meaning over time.
Why was the policy created? Who owns it? What obligation or intent did it support? What happens when the system changes? What happens when the policy gets refactored three times and nobody remembers why the original rule existed?
What I find fascinating is that Netflix was already wrestling with this problem nearly a decade ago. In 2017, long before we were talking about autonomous agents planning, delegating and acting across systems, Manish Mehra and Torin Sandall were explicitly talking about “capturing intent” in authorization.
Their problem was simpler than the one we have today, but the underlying concern feels remarkably familiar. An engineer could believe they had expressed one thing while the policy they actually wrote did something slightly different. Netflix tried to close that gap by making policy easier to express and, importantly, by preserving tests alongside the policy so that future changes could still be checked against what the original author intended.
I love that because it is a useful reminder that we do not have to invent all of this from scratch. We can stand on the shoulders of people who were already thinking deeply about these problems before the technology made them urgent.
Nearly a decade later, agentic systems have made the problem much harder. We are no longer just asking whether a deterministic policy still does what its author intended. We now have to ask whether an autonomous system, capable of interpreting goals and changing its plan, is still acting within the intent under which its authority was granted.
The technology has changed dramatically. The problem of preserving meaning has not.
Watch the Netflix session on YouTube
We are quite good at creating policies, but much less good at thinking about the lifecycle around those policies and preserving the original context behind them.
That, to me, is where a policy or authorization control plane starts to become valuable.
Its job should not necessarily be to execute every decision centrally.
Its job may be to maintain the meaning around those decisions.
That means preserving who owns the policy, what intent it supports, where it is enforced, what obligation created it, which version is active, what changed, why it changed, and whether the behaviour we see at runtime is still consistent with the outcome we originally intended.
That is a very different proposition from simply centralising enforcement.
Intent probably needs to become a first-class object#
This is the part I find most interesting.
Perhaps intent needs to become a proper architectural object. Today it is often just something written down in a requirement, a prompt or a design document, and over time that meaning gets diluted or lost.
If intent is going to matter to authorization, we probably need a way of capturing it that is structured enough for machines to reason about, versioned as things change, and linked back to the authority that was originally delegated. Then you have something a policy engine can evaluate, something an eval can test against, and something runtime evidence can be compared with later.
Otherwise, we are effectively trying to detect intent drift without ever having properly captured the intent in the first place.
And importantly, that intent should not be something the agent can quietly redefine for itself.
You could imagine that intent sitting at the start of a chain that then follows the delegated authority through the system:
Human → Intent → Delegation → Agent → Plan → Action → Policy Decision → Evidence
Once you have that, some interesting possibilities start to open up. You can test whether a new version of an agent still behaves within its original boundaries, ask for human approval when an action stretches the original intent, reason about delegation between agents, and compare what was supposed to happen with what actually happened.
At that point, authorization becomes much more than permission checking.
It becomes a way of preserving meaning as authority moves through the system.
We probably will not solve morality. But there is something useful in the analogy.#
I am not suggesting we encode moral philosophy in Cedar.
Although I suspect somebody eventually will.
The point is much narrower.
Human societies discovered a long time ago that governing behaviour requires more than simply saying yes or no. We rely on norms, values, context, authority and some mechanism for deciding when a situation is ambiguous enough that it needs to be escalated.
We rarely judge an action completely in isolation. We judge it in relation to what someone was trying to do and the circumstances around it.
Autonomous systems are pushing computing toward a similar problem.
Authentication has traditionally answered who you are.
Authorization answers what you are allowed to do.
Intent-aware authorization needs to add another layer and ask what you are trying to achieve, under whose authority, with what constraints, and whether the actions you are taking are still consistent with that original purpose.
That is a much harder problem.
But I increasingly think it is one of the foundational architecture problems we are going to have to solve if we want to govern autonomous agents properly.
And perhaps the interesting part is that the answer may not come from traditional IAM at all.
It may emerge from the combination of authorization, policy management, delegation, provenance, evals and runtime evidence.
The future Intent-Based Authorization Governance System will not just manage deterministic permissions and policy decisions. It will need to preserve the meaning behind those decisions: the original intent, the authority that was delegated, the constraints that came with it, and the context in which an action remains legitimate.
That is the shift I think matters most.
From managing permission to preserving meaning.