Intent as permissions

[Alan Karp]

I'm participating in an agent use cases effort.  The question of expressing intent came up today.  One person posited that any intent could be expressed as permissions.  I thought it would be easy to come up with a counterexample, but I couldn't during the meeting.  What about you?

For those of you not up on the latest agent stuff, intent is supposed to express why you're asking.  For example, booking a flight.  You're probably safe booking a flight to your friend's wedding but not for a business meeting with an unreliable customer.

--------------
Alan Karp

[Raoul Duke]

Seems to me that if permissions were sufficiently narrow, and perhaps
with more than one working together, they could emulate intent - but
they never really are. Just look at smartphones asking for permission
to use resources. If it gets too fine-grained it is unusable. If it is
too loose, it is possibly security theatre. Not sure how you would
even define "intent" more technically. You'd have to come up with some
well defined taxonomy and types I guess.

[Matt Rice]

On Thu, Sep 17, 2026 at 10:02 AM Alan Karp <alan...@gmail.com> wrote:
>
> I'm participating in an agent use cases effort. The question of expressing intent came up today. One person posited that any intent could be expressed as permissions. I thought it would be easy to come up with a counterexample, but I couldn't during the meeting. What about you?
>
> For those of you not up on the latest agent stuff, intent is supposed to express why you're asking. For example, booking a flight. You're probably safe booking a flight to your friend's wedding but not for a business meeting with an unreliable customer.
>

Yeah, I think like Raoul says, "sufficiently narrow".

As an example, if you have an agent where the intent is you want it to
summarize the contents of a read channel, and write the summary to a
write channel.
With the basic read/write permission it could ignore the read channel
entirely and write garbage to the write channel.
Narrowing it, you could give it a "read summary", and "write summary",
permission, it could still ignore both.
Narrowing it further, the "write summary", could produce a report or
token with the required type for the agent to finish, assuming the
agent is well-founded (is not capable of just looping infinitely, and
is guaranteed to return) it must obtain the token from the write
channel, and can only do that by producing the summary from the read
channel.
So sufficiently narrow seems to me that it probably implies some kind
of dependent type system...

[Mark S. Miller]

I arrive in a new city, ask my ai car “I’d like to eat at the best Thai restaurant near me. Take me there”. I could only express this as a narrow permission if I already know which restaurant I want to go to. The essence of principal/agent problems is mixing the principal’s intent with the agent’s specialized knowledge, knowledge the principal does not have.

--
You received this message because you are subscribed to the Google Groups "friam" group.
To unsubscribe from this group and stop receiving emails from it, send an email to friam+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/friam/CACTLOFriN9PKEZci%3DP%2BLT7Qv5H8qxbD%3D_RL9cR1sG4R%2B0Qg_xg%40mail.gmail.com.

[Matt Rice]

I don't see why you couldn't just read the list of nearby Thai
restaurants, sort it by quality, and return the first entry in the
sorted list.
I think that would be sufficiently narrow while also being dependent
on the input ("Thai restaurant", coordinates)

> To view this discussion visit https://groups.google.com/d/msgid/friam/CAK-_AD5Yc0a9y%3DeAMLgEvSVa9z2wEdwE%2Bupw7%3DzxbgAU4aLcwQ%40mail.gmail.com.

[Alan Karp]

Good example.  I'm looking for something along the lines of "I'm giving you (the agent) this permission with that intent."

One example that we discussed used intent to fill in a gap in the application API.  For example, you tell your agent to summarize the first paragraph of a file, delegating read permission to the agent, but there's no way to express "the first paragraph" in the API.  You could (vibe) code a proxy to read the file and only forward the first paragraph to your agent.  However, knowing the intent, a well behaved agent could create that proxy instead.

--------------
Alan Karp

On Thu, Sep 17, 2026 at 11:36 AM 'Mark S. Miller' via friam <fr...@googlegroups.com> wrote:

To view this discussion visit https://groups.google.com/d/msgid/friam/CAK-_AD5Yc0a9y%3DeAMLgEvSVa9z2wEdwE%2Bupw7%3DzxbgAU4aLcwQ%40mail.gmail.com.

[Niki Aimable Niyikiza]

I think we may be conflating two layers: the UX or policy language used to express and decompose intent, and the mechanism that enforces the resulting authority.

Some parts of intent can be compiled into permissions when they are observable at the enforcement point. "Read only the first paragraph" becomes enforceable if a component outside the agent's authority exposes only that paragraph. Having the broadly authorized agent create the proxy itself is voluntary restraint. Other parts, such as purpose and preference, are not permissions at all.

So permissions can represent an enforceable projection of intent, but not necessarily the whole of it.

Niki

To view this discussion visit https://groups.google.com/d/msgid/friam/CANpA1Z3uD_gj3FpG%2Bw8Mo3w5wcBPzsdLk4136djv6%3DwO4cscxg%40mail.gmail.com.

--

Niki Aimable Niyikiza
Tenuo

[Mark S. Miller]

https://www.youtube.com/watch?v=upsBL46a2yQ is quite relevant. It brings together the economist's notion of principal-agent problems with our sense of POLA and much more. It explains why we don't actually mean "least", but rather an approximation to "least" that is consistent with effective delegation. Again, the reasons for delegating include the agent possessing knowledge the principal might lack. Also, the agent might need a somewhat larger range of possibilities to keep its costs low. Enjoy!

On Thu, Sep 17, 2026 at 1:40 PM Alan Karp <alan...@gmail.com> wrote:

To view this discussion visit https://groups.google.com/d/msgid/friam/CANpA1Z3uD_gj3FpG%2Bw8Mo3w5wcBPzsdLk4136djv6%3DwO4cscxg%40mail.gmail.com.

[Mark S. Miller]

On Thu, Sep 17, 2026 at 2:37 PM 'Niki Aimable Niyikiza' via friam <fr...@googlegroups.com> wrote:

I think we may be conflating two layers: the UX or policy language used to express and decompose intent, and the mechanism that enforces the resulting authority.

Some parts of intent can be compiled into permissions when they are observable at the enforcement point. "Read only the first paragraph" becomes enforceable if a component outside the agent's authority exposes only that paragraph. Having the broadly authorized agent create the proxy itself is voluntary restraint. Other parts, such as purpose and preference, are not permissions at all.

So permissions can represent an enforceable projection of intent, but not necessarily the whole of it.

Yes! Well put

 

To view this discussion visit https://groups.google.com/d/msgid/friam/CALGH9Z_a-m02T390t3wdee4AFGjo5WQ2BhfVc%2BwHvwXebUHzNA%40mail.gmail.com.

[Mark S. Miller]

Key graphics from our talk

[Matt Rice]

On Thu, Sep 17, 2026 at 2:43 PM 'Mark S. Miller' via friam
<fr...@googlegroups.com> wrote:
>
>
>
> On Thu, Sep 17, 2026 at 2:37 PM 'Niki Aimable Niyikiza' via friam <fr...@googlegroups.com> wrote:
>>
>> I think we may be conflating two layers: the UX or policy language used to express and decompose intent, and the mechanism that enforces the resulting authority.
>>
>> Some parts of intent can be compiled into permissions when they are observable at the enforcement point. "Read only the first paragraph" becomes enforceable if a component outside the agent's authority exposes only that paragraph. Having the broadly authorized agent create the proxy itself is voluntary restraint. Other parts, such as purpose and preference, are not permissions at all.
>>
>> So permissions can represent an enforceable projection of intent, but not necessarily the whole of it.
>
>
> Yes! Well put
>

One of the things I was (probably badly) trying to convey, is that as
permission more precisely conveys intent,
agents lose agency. We can have a very precise enforcement, but as the
projection approaches intent,
the agent can just follow along step by step, with the permissions as given.

That is to say, in the thai food example, the principal can convey the
steps the agent must take exactly to deduce the destination
even though the agent has (or is given permission to the data). At
some point the only purpose for the agent becomes that
they won't let you access/resolve your intent you gave them directly
on the data they have permission to. Similar to Norm's old examples
with the chemical database.

> To view this discussion visit https://groups.google.com/d/msgid/friam/CAK-_AD5_rjm3SMuaZG1dd8PVFB9-aFbn-xwnSopcmr-Pd6MeEw%40mail.gmail.com.

[Raoul Duke]

Probably need actual agreed upon definition of "intent". It is hardly
even well communicated among humans. I doubt there's a way to really
encode it... without turning it into a full formal specification style
language, combined with a pretty amazing natural language system.

the examples in the real world of humans not-quite-communicating are
of course endless.
https://thelanguagenerds.com/2024/25-times-people-messed-up-so-bad-with-given-instructions-it-resulted-in-hilarious-mishaps/
http://static.zerorobotics.mit.edu/docs/team-activities/ProgrammingPeanutButterAndJelly.pdf

[Alan Karp]

Exactly right.  "Intent" covers what you can't express as permissions, either because the intent is amorphous, e.g., for my wife's birthday, or because the API doesn't support it, e.g., the first paragraph.

--------------
Alan Karp

To view this discussion visit https://groups.google.com/d/msgid/friam/CALGH9Z_a-m02T390t3wdee4AFGjo5WQ2BhfVc%2BwHvwXebUHzNA%40mail.gmail.com.

[Niki Aimable Niyikiza]

Hi Matt,

> As permission more precisely conveys intent, agents lose agency.

Perhaps we are using “permission” in two different ways. I don’t expect permissions to convey intent to the agent; instructions do that. If permissions progressively prescribe the agent’s next steps, then sufficiently precise permissions become a workflow, leaving the agent with little agency.

I mean permissions as an independently enforced boundary on effects. They do not tell the agent what to do; they determine which attempted actions the enforcement point will accept. In the Thai example, the boundary might constrain distance, price, or permissible destinations while leaving the agent to decide which is best and how to get there.

With complete enforcement, every executed effect must fall within that boundary and attempts outside it are denied. Whether the agent understood or fulfilled the original intent is a separate concern.

The same distinction applies when an agent delegates. It can give a sub-agent instructions describing the work while separately constraining what the sub-agent is allowed to execute.

Raoul, > Probably need actual agreed upon definition of "intent".

Intent can remain an ambiguous upstream input. The security boundary can still be drawn where some portion of the intent is translated into explicitly enforceable constraints, with the limits of that projection made clear. A complete formal definition is not a prerequisite for useful enforcement.

The resulting guarantee is not ideal but still useful: the agent will stay within the authority of the intended task that was defined roughly based on the translated human's intent.

Niki

To view this discussion visit https://groups.google.com/d/msgid/friam/CANpA1Z3fZ93%2Brjc7MUXhU-6ybzzC98XS7onLAVX0tpMt4OCypg%40mail.gmail.com.

[Matt Rice]

On Thu, Sep 17, 2026 at 5:13 PM 'Niki Aimable Niyikiza' via friam
<fr...@googlegroups.com> wrote:
>
> Hi Matt,
>

Hi Niki!

> > As permission more precisely conveys intent, agents lose agency.
>
>
> Perhaps we are using “permission” in two different ways. I don’t expect permissions to convey intent to the agent; instructions do that. If permissions progressively prescribe the agent’s next steps, then sufficiently precise permissions become a workflow, leaving the agent with little agency.
>
> I mean permissions as an independently enforced boundary on effects. They do not tell the agent what to do; they determine which attempted actions the enforcement point will accept. In the Thai example, the boundary might constrain distance, price, or permissible destinations while leaving the agent to decide which is best and how to get there.
>
> With complete enforcement, every executed effect must fall within that boundary and attempts outside it are denied. Whether the agent understood or fulfilled the original intent is a separate concern.
>
> The same distinction applies when an agent delegates. It can give a sub-agent instructions describing the work while separately constraining what the sub-agent is allowed to execute.
>

Essentially all I am saying is that in a capability system where
authority is represented by a program or function producing a result,
there is both a large amount of flexibility to encode intent in
different programs, and different languages and type systems can
further refine the number of programs. For instance in a language like
granule it is pretty common for a function's type to have only a
single function capable of conforming to that type (because the type
system is able to eliminate most instances of programs since it is
encoding the number of times a value can be used etc).

By "as precision conveys intent", I mean if we define intent as
producing an "expected result", there is some number that counts the
amount of divergence from the intended result as the number of
possible choices an agent can make that diverges from that. If the
number of choices == 1 the agent is only capable of producing the
intended/expected result.

Systems where you can provide your own objects, For instance by
wrapping an object which strips off all but the first paragraph of
text, for building a summary of the first paragraph. Have relatively
flexible ability to encode intent in objects, while system where the
types of permissions are largely provided by the system itself and
users aren't able to introduce their own object wrappers may provide
little ability to encode intent.

So I'm essentially just saying that there is *some* system out there
which is probably capable of conveying intent extremely precisely,
because it is a spectrum of how precisely a program can be specified,
but that kind of system reduces the number of choices the agent may
make by definition.

But that is to say the principal may provide a program that compares
quality, a program that defines `nearby` via a distance calculation.
If we do this such that the agent is just running a user provided
filter on a user defined query, over the agen'ts restaurant database.

I'm just trying to convey the idea that a permissions system fully
capable of expressing intent as permission leaves agents with no
choice or agency in the result
they produce. Then to say that this is kind counter to the purposes
for which one would typically employ some agent. Part of the value (or
even intent) is that they typically fill in the blanks with some
heuristic like quality rather than forcing the user to provide a
function that gives weight to reported quality of ingredients or
service or whatever is important to them.

Anyhow I don't really consider this that important, but i'm just
saying taken to the extreme permission == intent can leave very little
room for choice.

> To view this discussion visit https://groups.google.com/d/msgid/friam/CALGH9Z81n1guxdVow%2BrFSaGw2GwZCJvf1WK%2B8_Zix20wwQyErA%40mail.gmail.com.

[Raoul Duke]

> The resulting guarantee is not ideal but still useful: the agent will stay within the authority of the intended task that was defined roughly based on the translated human's intent.

i feel like i am missing some optimism gene. all i can think is, good
luck with that? i mean sure it is nice to try to get better at
handling intent, but it must come with a galaxy sized pinch of salt,
it seems to me. from both a UX and actual security of final behavior
perspective.