Confused deputies as "failures of judgement"
19 views
Skip to first unread message
Christopher Lemmer Webber
Apr 2, 2021, 2:36:24 PM4/2/21
to cap-...@googlegroups.com
Crock challenged us to (again) try to come up with another alternate
term for confused deputy, that the term "confused deputy" maybe made
sense in Norm's original example but not others. I threw one framing
out there: "failure of judgement". I'm not saying this is the term we
should use, but I want to use it as a starting point for thinking of
other terms.
By saying "failure of judgement" we're already doing something
interesting: explaining that we've introduced "judgement calls" into our
system. Judgement calls happen typically by thinking within some sort
of ambient context. This is hard enough for humans to get right; it's a
deadly level of difficulty to get programs we fire off and run to try to
do.
Most security attacks, when seen, are often responded to with a "Captain
Hindsight" type comment. Of course, looking back, you can see there was
a judgement error... so you figure out how to code in that new judgement
case, and try to pretend there aren't many more hidden judgement call
errors there.
The solution ocaps provide is that certain kinds of attacks are
*inexpressable*... there's no judgement call when "only connectivity
begets connectivity". (Plus, ocaps as programs bake "code paths" where
authority is used in specific ways, so the usage within a particular
agent is usually even more intentional.)
Thus, avoid programs which make judgement calls based on an ambient
context as much as possible.
- Chris
PS: Note that we've discussed recently how "unsealers in a bucket"
reintroduce confused deputy attacks. This is unsurprising; we've
flattened the information where we reintroduce something along the lines
of a judgement call and having to consider a context. Code which pulls
an unsealer out in advance because it expects to use an unsealer in a
particular context and tries it is thus likely to fail only in the safe
way.
(Contributors to this conversation: Douglas Crockford, Alan Karp, Mike
Samuel, Terry Hayes, Bill Frantz, Kevin Reid, Baldur Jóhannsson)
term for confused deputy, that the term "confused deputy" maybe made
sense in Norm's original example but not others. I threw one framing
out there: "failure of judgement". I'm not saying this is the term we
should use, but I want to use it as a starting point for thinking of
other terms.
By saying "failure of judgement" we're already doing something
interesting: explaining that we've introduced "judgement calls" into our
system. Judgement calls happen typically by thinking within some sort
of ambient context. This is hard enough for humans to get right; it's a
deadly level of difficulty to get programs we fire off and run to try to
do.
Most security attacks, when seen, are often responded to with a "Captain
Hindsight" type comment. Of course, looking back, you can see there was
a judgement error... so you figure out how to code in that new judgement
case, and try to pretend there aren't many more hidden judgement call
errors there.
The solution ocaps provide is that certain kinds of attacks are
*inexpressable*... there's no judgement call when "only connectivity
begets connectivity". (Plus, ocaps as programs bake "code paths" where
authority is used in specific ways, so the usage within a particular
agent is usually even more intentional.)
Thus, avoid programs which make judgement calls based on an ambient
context as much as possible.
- Chris
PS: Note that we've discussed recently how "unsealers in a bucket"
reintroduce confused deputy attacks. This is unsurprising; we've
flattened the information where we reintroduce something along the lines
of a judgement call and having to consider a context. Code which pulls
an unsealer out in advance because it expects to use an unsealer in a
particular context and tries it is thus likely to fail only in the safe
way.
(Contributors to this conversation: Douglas Crockford, Alan Karp, Mike
Samuel, Terry Hayes, Bill Frantz, Kevin Reid, Baldur Jóhannsson)
Mark S. Miller
Apr 2, 2021, 2:48:28 PM4/2/21
to cap-...@googlegroups.com
I like "co-mingled authority". I bounced this off Norm while I still could, and he was fine with it.
The main problem with the term "confused deputy" is implying the bug is in the deputy. That if the deputy were coded better, it wouldn't be confused. I think "failure of judgement" has the same problem. What I like about "co-mingled authority" is a) it does not suggest that misunderstanding. b) sets up for explaining that the authority is already co-mingled before the deputy gets it. The deputy *cannot* know which authority to use for what purpose, since the required distinctions are already gone.
Also "co-mingled authority" fits nicely with our explanation of the other major problems:
"excess authority"
"ambient authority"
"undeniable authority"
--
You received this message because you are subscribed to the Google Groups "cap-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/871rbsileg.fsf%40dustycloud.org.
Bill Frantz
Apr 2, 2021, 3:00:41 PM4/2/21
to cap-...@googlegroups.com
On 4/2/21 at 2:48 PM, ma...@agoric.com (Mark S. Miller) wrote:
> I like "co-mingled authority". I bounced this off Norm while I still could,
> and he was fine with it.
>
> The main problem with the term "confused deputy" is implying the bug is in
> the deputy. That if the deputy were coded better, it wouldn't be confused.
> I think "failure of judgement" has the same problem. What I like about
> "co-mingled authority" is a) it does not suggest that misunderstanding. b)
> sets up for explaining that the authority is already co-mingled before the
> deputy gets it. The deputy *cannot* know which authority to use for what
> purpose, since the required distinctions are already gone.
>
> Also "co-mingled authority" fits nicely with our explanation of the other
> major problems:
>
> "excess authority"
> "ambient authority"
> "undeniable authority"
I like the idea.
> I like "co-mingled authority". I bounced this off Norm while I still could,
> and he was fine with it.
>
> The main problem with the term "confused deputy" is implying the bug is in
> the deputy. That if the deputy were coded better, it wouldn't be confused.
> I think "failure of judgement" has the same problem. What I like about
> "co-mingled authority" is a) it does not suggest that misunderstanding. b)
> sets up for explaining that the authority is already co-mingled before the
> deputy gets it. The deputy *cannot* know which authority to use for what
> purpose, since the required distinctions are already gone.
>
> Also "co-mingled authority" fits nicely with our explanation of the other
> major problems:
>
> "excess authority"
> "ambient authority"
> "undeniable authority"
Perhaps "Pre-mingled authority"??
Cheers - Bill
Mark S. Miller
Apr 2, 2021, 3:04:30 PM4/2/21
to cap-...@googlegroups.com
Even better! Thanks!
Cheers - Bill
--
You received this message because you are subscribed to the Google Groups "cap-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/r480Ps-10146i-9EAF163BCA1041A3A95063A9BFF25777%40Williams-MacBook-Pro.local.
Carl Hewitt
Apr 2, 2021, 3:09:46 PM4/2/21
to cap-talk
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CAK-_AD6zbPiTwvenbihD8r9_8RomRxHXsS5zu2xw8bjq%3De4eQg%40mail.gmail.com.
Chip Morningstar
Apr 2, 2021, 3:10:45 PM4/2/21
to cap-...@googlegroups.com
“Co-mingled authority” nicely parallels the accounting idea of “co-mingled funds”, wherein once you’ve put money from different sources that were supposed to be managed separately into a single account, you can no longer keep track of which is which to maintain proper accounting controls.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CAK-_AD5P3BA%3Df4iZwnqZ5uzB-NzyVMH8Y-0n6yERMS6vw8%2B0dQ%40mail.gmail.com.
John Kemp
Apr 2, 2021, 3:15:18 PM4/2/21
to cap-...@googlegroups.com
I guess I see the “confused deputy” as being two things:
1. The reference being used does not adequately express authority to use the reference.
2. The request also carries ambient authority (cookies in the HTTP case).
So I would argue that the confusion is between “no/inadequate authority” and additional, “ambient authority”.
The thing about the term “deputy” that makes a lot of sense to me is that there is (to me) this idea of an agent, under the control of a human. In the confused deputy attacks I’m most familiar with, the confusion occurs because the deputy doesn’t know whether it is acting on behalf of the user, or some other entity. "Deputy" adds what I feel is an important element of “identity” or “user”.
Dunno if that helps with an actual new term though, and perhaps I’m just saying that “confused deputy” is evocative for me?
- johnk
1. The reference being used does not adequately express authority to use the reference.
2. The request also carries ambient authority (cookies in the HTTP case).
So I would argue that the confusion is between “no/inadequate authority” and additional, “ambient authority”.
The thing about the term “deputy” that makes a lot of sense to me is that there is (to me) this idea of an agent, under the control of a human. In the confused deputy attacks I’m most familiar with, the confusion occurs because the deputy doesn’t know whether it is acting on behalf of the user, or some other entity. "Deputy" adds what I feel is an important element of “identity” or “user”.
Dunno if that helps with an actual new term though, and perhaps I’m just saying that “confused deputy” is evocative for me?
- johnk
Mark S. Miller
Apr 2, 2021, 3:15:55 PM4/2/21
to cap-...@googlegroups.com
It's not unauthorized.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CA%2BymXc18cH5RiEEWbRZLdce0jEy_0o0gKGKq_f3fGczzvzW8tg%40mail.gmail.com.
Carl Hewitt
Apr 2, 2021, 3:23:03 PM4/2/21
to cap-talk
Chip Morningstar
Apr 2, 2021, 3:49:13 PM4/2/21
to cap-...@googlegroups.com
It’s a classic case of the computer doing what you said instead of what you meant.
You may have meant not to authorize, but what you *said* did authorize.
In any case, the issue with labeling the Phenomenon Formerly Called The Confused Deputy is that while the consequence is that access you didn’t intend to authorize got authorized, the underlying problem that we’re trying to describe in the label is a problem of information loss, namely the decoupling of intent and action.
--
You received this message because you are subscribed to the Google Groups "cap-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CA%2BymXc23yKogqsXL-MTiyNrZfvMciy%3DD9WALKc6NJZRhGVHUiA%40mail.gmail.com.
Carl Hewitt
Apr 2, 2021, 6:30:32 PM4/2/21
to cap-talk
Thanks Chip!
With the rise of Intelligent Systems, it is more complex to
determine
exactly
what the computer was told to do.
The action taken could have been mediated by
computer inferences using information in disagreement.
Also, determining intent is more complex.
Some preliminary work has been done here:
"Epistemology Cyberattacks"
https://papers.ssrn.com/abstract=3603021But we have a long way to go!
Regards,
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/E6353D23-C9ED-4602-A1CA-A3A3EF441226%40fudco.com.
Christopher Lemmer Webber
Apr 5, 2021, 12:19:28 PM4/5/21
to cap-...@googlegroups.com, Chip Morningstar
This observation of Chip's makes "co-mingled authority" make more sense
to me.
Still, the phrase feels like it's missing something though; one could
say an object which composes multiple capabilities is co-mingling
authority, but doing so safely.
Chip Morningstar writes:
> “Co-mingled authority” nicely parallels the accounting idea of
> “co-mingled funds”, wherein once you’ve put money from different
> sources that were supposed to be managed separately into a single
> account, you can no longer keep track of which is which to maintain
> proper accounting controls.
>
>> On Apr 2, 2021, at 11:48 AM, Mark S. Miller <ma...@agoric.com> wrote:
>>
>> I like "co-mingled authority". I bounced this off Norm while I still
>> could, and he was fine with it.
>>
>> The main problem with the term "confused deputy" is implying the bug
>> is in the deputy.
I both agree and also think that's partly the point. The confused
deputy *is* mis-programmed, in the sense that it's programmed within the
wrong *paradigm*. The deputy might be fairly well programmed within
that construction, but sometimes when leaks keep appearing everywhere,
it's time to change your construction materials.
>> That if the deputy were coded better, it wouldn't be confused.
And in general, these criticisms are *correct*! One can almost always
plug a hole in the confused deputy authority leak. Meanwhile tech
journalists will show up and wag their fingers and say "they should have
seen this leak was here and patched it."
The problem is that the surface is so porous, there are always more
leaks about to burst through. This *retrospective criticism* is the
real problem. It is akin to "memory safety in C": in theory, it is
possible to program the perfect C programs which have no memory safety
issues. In practice, it is not possible. But a mistake can always be
pointed out in retrospect. My heart bleeds for the openssl developers
and how it must have felt to receive all that "obvious in retrospect"
criticism. But the right solution would have been to not put developers
in this position to make those kinds of mistakes over and over.
You are correct then that "failure of judgement" is thus insufficient,
because people wag their fingers saying "use better judgement next
time". But the right answer is "don't rely on judgement at all".
>> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com <mailto:cap-talk%2Bunsu...@googlegroups.com>.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/871rbsileg.fsf%40dustycloud.org <https://groups.google.com/d/msgid/cap-talk/871rbsileg.fsf%40dustycloud.org>.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CAK-_AD5P3BA%3Df4iZwnqZ5uzB-NzyVMH8Y-0n6yERMS6vw8%2B0dQ%40mail.gmail.com <https://groups.google.com/d/msgid/cap-talk/CAK-_AD5P3BA%3Df4iZwnqZ5uzB-NzyVMH8Y-0n6yERMS6vw8%2B0dQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
to me.
Still, the phrase feels like it's missing something though; one could
say an object which composes multiple capabilities is co-mingling
authority, but doing so safely.
Chip Morningstar writes:
> “Co-mingled authority” nicely parallels the accounting idea of
> “co-mingled funds”, wherein once you’ve put money from different
> sources that were supposed to be managed separately into a single
> account, you can no longer keep track of which is which to maintain
> proper accounting controls.
>
>> On Apr 2, 2021, at 11:48 AM, Mark S. Miller <ma...@agoric.com> wrote:
>>
>> I like "co-mingled authority". I bounced this off Norm while I still
>> could, and he was fine with it.
>>
>> The main problem with the term "confused deputy" is implying the bug
>> is in the deputy.
deputy *is* mis-programmed, in the sense that it's programmed within the
wrong *paradigm*. The deputy might be fairly well programmed within
that construction, but sometimes when leaks keep appearing everywhere,
it's time to change your construction materials.
>> That if the deputy were coded better, it wouldn't be confused.
plug a hole in the confused deputy authority leak. Meanwhile tech
journalists will show up and wag their fingers and say "they should have
seen this leak was here and patched it."
The problem is that the surface is so porous, there are always more
leaks about to burst through. This *retrospective criticism* is the
real problem. It is akin to "memory safety in C": in theory, it is
possible to program the perfect C programs which have no memory safety
issues. In practice, it is not possible. But a mistake can always be
pointed out in retrospect. My heart bleeds for the openssl developers
and how it must have felt to receive all that "obvious in retrospect"
criticism. But the right solution would have been to not put developers
in this position to make those kinds of mistakes over and over.
You are correct then that "failure of judgement" is thus insufficient,
because people wag their fingers saying "use better judgement next
time". But the right answer is "don't rely on judgement at all".
>> To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/871rbsileg.fsf%40dustycloud.org <https://groups.google.com/d/msgid/cap-talk/871rbsileg.fsf%40dustycloud.org>.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "cap-talk" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com <mailto:cap-talk+u...@googlegroups.com>.
>> --
>> You received this message because you are subscribed to the Google Groups "cap-talk" group.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CAK-_AD5P3BA%3Df4iZwnqZ5uzB-NzyVMH8Y-0n6yERMS6vw8%2B0dQ%40mail.gmail.com <https://groups.google.com/d/msgid/cap-talk/CAK-_AD5P3BA%3Df4iZwnqZ5uzB-NzyVMH8Y-0n6yERMS6vw8%2B0dQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Mark S. Miller
Apr 5, 2021, 1:19:38 PM4/5/21
to cap-...@googlegroups.com, Chip Morningstar
You underestimate the depth of the confused deputy problem. Everyone, let's reread Tyler's "ACLs don't" and let's discuss.
To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/875z10901d.fsf%40dustycloud.org.
Bill Frantz
Apr 5, 2021, 5:41:44 PM4/5/21
to cap-...@googlegroups.com
On 4/5/21 at 12:19 PM, cwe...@dustycloud.org (Christopher Lemmer Webber) wrote:
> This observation of Chip's makes "co-mingled authority" make more sense
> to me.
More accurately, co-mingled authorizations.
> This observation of Chip's makes "co-mingled authority" make more sense
> to me.
Cheers - Bill
---------------------------------------------------------------------------
Bill Frantz |"We used to quip that "password" is the most common
408-348-7900 | password. Now it's 'password1.' Who said users haven't
www.pwpconsult.com | learned anything about security?" -- Bruce Schneier
Alan Karp
Apr 5, 2021, 5:51:25 PM4/5/21
to cap-...@googlegroups.com
Frantz <fra...@pwpconsult.com> wrote:
On 4/5/21 at 12:19 PM, cwe...@dustycloud.org (Christopher Lemmer Webber) wrote:
> This observation of Chip's makes "co-mingled authority" make more sense
> to me.
More accurately, co-mingled authorizations.
I would say the problem is more one of co-mingled designations. The deputy is designating one object, and the attacker another one. The deputy then necessarily co-mingles those designations when making an invocation.
--------------
Alan Karp
Mark S. Miller
Apr 5, 2021, 6:13:49 PM4/5/21
to cap-...@googlegroups.com
No. In the compiler example, the various parties designate exactly what they intend to designate. This is not the problem.
--
You received this message because you are subscribed to the Google Groups "cap-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cap-talk+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CANpA1Z08FFmVmBn1Ks2cANhn_SMGbx44RWiq6gzs%3DVxd-pW3mg%40mail.gmail.com.
Alan Karp
Apr 5, 2021, 6:22:14 PM4/5/21
to cap-...@googlegroups.com
The attacker intends to designate the log file for the compiler output. The compiler intends to designate the file specified by the attacker for the output. The compiler does not intend to designate the log file for the compiler output.
--------------
Alan Karp
--------------
Alan Karp
To view this discussion on the web visit https://groups.google.com/d/msgid/cap-talk/CAK-_AD7VTxBzo03gB4Ts8SC5GM4hLta0mkizzSem6z2_LaqtJg%40mail.gmail.com.
John Kemp
Apr 5, 2021, 7:26:26 PM4/5/21
to cap-...@googlegroups.com
On Apr 5, 2021, at 12:19 PM, Christopher Lemmer Webber <cwe...@dustycloud.org> wrote:That if the deputy were coded better, it wouldn't be confused.
And in general, these criticisms are *correct*! One can almost always
plug a hole in the confused deputy authority leak. Meanwhile tech
journalists will show up and wag their fingers and say "they should have
seen this leak was here and patched it."
The problem is that the surface is so porous, there are always more
leaks about to burst through. This *retrospective criticism* is the
real problem.
When I think specifically about the Web case, a lot of the blame falls on the very specific issue of cookies, and the use of URLs to designate resources. With Netscape's need to invent the “shopping cart” and subsequent introduction of Web cookies to allow HTTP state, we created the opportunity for the “confused deputy” (URL + cookies).
Ever since, we’ve been trying various different patches for the leaks, to ensure that large amounts of code didn’t require rewriting.
So, perhaps the “real problem” is that someone with influence proposed something that became quickly popular and thus “too big to fail”? That sounds like a mistake it would be pretty easy to make again.
- johnk
btu...@gmail.com
Apr 15, 2021, 10:06:19 PM4/15/21
to cap-talk
I believe I coined the phrase commingled (two m's btw) authority in discussion with MarkM a while back, precisely based on the analogy Chip mentions. This was while we were working on our "The Elements of Decision Alignment" paper https://isr.uci.edu/content/mark-s-miller.
In that work, we borrow the notion of principal and agent from economics (and law), where the agent performs work on behalf of the principal. Norm called the agent more colorfully a deputy, but I think of it as a confused agent problem. The setup is that the agent is doing work on behalf of two or more principals and has been (or should have been) granted authority to perform just the work required within the granted scope of authority separately for each principal. The law of agency talks specifically of agent's acting within or outside of their scope of authority.
Like MarkM said, the problem arises not so much because the agent acts incorrectly, but because they have no reliable means to keep the authority grants separate (not commingled). Confused agent makes you want to fix the agent, rather than the commingling.
- Bill T.
Reply all
Reply to author
Forward
0 new messages