Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Kerberos protocol transition with unconstrained delegation (i.e. TGT impersonation)
37 views
Skip to first unread message
Jonathan Calmels
Oct 27, 2022, 10:34:28 AM10/27/22
to kerb...@mit.edu
Hi,
We have a Linux cluster fully kerberized including its own MIT Kerberos KDC which we control.
Users authenticate to it through a one-way trust with an Active Directory. After being authenticated, users submit their workload with their TGT and the scheduler will forward it to the nodes it allocated (i.e. unconstrained delegation).
So far everything is working as expected.
Now the problem is that we need to support the same workflow from a CI/CD webservice.
Users authenticate to the CI/CD webservice through SAML and will trigger some kind of job to be scheduled. The scheduler knows the user's principal but doesn't have a TGT associated with it.
Basically, the scheduler needs a way to impersonate users' TGTs to start their workload.
How does one go about that? given that:
- We can't use SPNEGO on the CI/CD webservice or request anything from the user there. It has to be regular SAML and we don't control user interactions.
- We can't use constrained delegation (aka. S4U) because the scheduler needs the user's TGT not a service ticket. Users are free to use their TGT however they want from the allocated nodes.
So far, the only hack we can think of is replicating the AD users into the MIT KDC and writing some kind of GSS service that would issue TGTs for those (given the proper service ticket).
Something like:
1. The scheduler does protocol transition with the AD UPN it got from the CI/CD
2. The scheduler contacts this GSS service with the resulting service ticket
3. The GSS service converts the UPN from the AD realm to its MIT realm counterpart
4. If everything checks out, it sends back a TGT for the user (this might involve some unconventional calls to libkadm5)
5. The scheduler forwards this TGT as usual
Is there a cleaner alternative? Ideally, one that doesn't involve replicating users.
If not, is libgssapi and likadm5 the best way to implement it or would something like a plugin module be better suited?
Thanks for any insight
We have a Linux cluster fully kerberized including its own MIT Kerberos KDC which we control.
Users authenticate to it through a one-way trust with an Active Directory. After being authenticated, users submit their workload with their TGT and the scheduler will forward it to the nodes it allocated (i.e. unconstrained delegation).
So far everything is working as expected.
Now the problem is that we need to support the same workflow from a CI/CD webservice.
Users authenticate to the CI/CD webservice through SAML and will trigger some kind of job to be scheduled. The scheduler knows the user's principal but doesn't have a TGT associated with it.
Basically, the scheduler needs a way to impersonate users' TGTs to start their workload.
How does one go about that? given that:
- We can't use SPNEGO on the CI/CD webservice or request anything from the user there. It has to be regular SAML and we don't control user interactions.
- We can't use constrained delegation (aka. S4U) because the scheduler needs the user's TGT not a service ticket. Users are free to use their TGT however they want from the allocated nodes.
So far, the only hack we can think of is replicating the AD users into the MIT KDC and writing some kind of GSS service that would issue TGTs for those (given the proper service ticket).
Something like:
1. The scheduler does protocol transition with the AD UPN it got from the CI/CD
2. The scheduler contacts this GSS service with the resulting service ticket
3. The GSS service converts the UPN from the AD realm to its MIT realm counterpart
4. If everything checks out, it sends back a TGT for the user (this might involve some unconventional calls to libkadm5)
5. The scheduler forwards this TGT as usual
Is there a cleaner alternative? Ideally, one that doesn't involve replicating users.
If not, is libgssapi and likadm5 the best way to implement it or would something like a plugin module be better suited?
Thanks for any insight
Russ Allbery
Oct 27, 2022, 11:57:46 AM10/27/22
to Jonathan Calmels via Kerberos, Jonathan Calmels
Jonathan Calmels via Kerberos <kerb...@mit.edu> writes:
> So far, the only hack we can think of is replicating the AD users into
> the MIT KDC and writing some kind of GSS service that would issue TGTs
> for those (given the proper service ticket). Something like:
> 1. The scheduler does protocol transition with the AD UPN it got from
> the CI/CD
> 2. The scheduler contacts this GSS service with the resulting service
> ticket
> 3. The GSS service converts the UPN from the AD realm to its MIT
> realm counterpart
> 4. If everything checks out, it sends back a TGT for the user (this
> might involve some unconventional calls to libkadm5)
> 5. The scheduler forwards this TGT as usual
Yup, this is also what I would do given your constraints. (You have a
> So far, the only hack we can think of is replicating the AD users into
> the MIT KDC and writing some kind of GSS service that would issue TGTs
> for those (given the proper service ticket). Something like:
> 1. The scheduler does protocol transition with the AD UPN it got from
> the CI/CD
> 2. The scheduler contacts this GSS service with the resulting service
> ticket
> 3. The GSS service converts the UPN from the AD realm to its MIT
> realm counterpart
> 4. If everything checks out, it sends back a TGT for the user (this
> might involve some unconventional calls to libkadm5)
> 5. The scheduler forwards this TGT as usual
fairly weird security corner case that requires arbitrary user
impersonation with no chain of authentication back to the user being
impersonated, which Kerberos doesn't really natively support for somewhat
obvious reasons.)
> Is there a cleaner alternative? Ideally, one that doesn't involve
> replicating users.
cross-realm key so that it can forge TGTs that look to the MIT Kerberos
KDC as if they were issued by AD. I think this would be roughly
equivalent from a security standpoint (either way, the GSS service is
essentially a KDC and has full access to the entirety of your MIT realm),
but would avoid the need to create principals in your KDC database for
every user.
The drawback of this approach is that you're likely to need to write some
low-level Kerberos code to forge the tickets, whereas in your plan above
you can just generate keytab files for every user and store them on disk
(again, the GSS service is functionally equivalent to a KDC, so this is
just the KDC database in another format), and then your GSS service can
generate TGTs through normal libkrb5 calls using the keytab and doesn't
have to do anything special.
--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>
Carson Gaspar
Oct 27, 2022, 12:13:42 PM10/27/22
to kerb...@mit.edu
Another option is to use PKINIT to get TGTs, with whatever your desired
AuthZ is on the CA to limit which users can be impersonated. I've
implemented this for other reasons, but it probably won't get open
sourced until mid-late next year (and does a bunch of stuff you probably
won't want/need).
On 10/26/2022 11:34 PM, Jonathan Calmels via Kerberos wrote:
> Hi,
>
> We have a Linux cluster fully kerberized including its own MIT Kerberos KDC which we control.
> Users authenticate to it through a one-way trust with an Active Directory. After being authenticated, users submit their workload with their TGT and the scheduler will forward it to the nodes it allocated (i.e. unconstrained delegation).
> So far everything is working as expected.
>
>
> Now the problem is that we need to support the same workflow from a CI/CD webservice.
> Users authenticate to the CI/CD webservice through SAML and will trigger some kind of job to be scheduled. The scheduler knows the user's principal but doesn't have a TGT associated with it.
>
> Basically, the scheduler needs a way to impersonate users' TGTs to start their workload.
> How does one go about that? given that:
>
>
> - We can't use SPNEGO on the CI/CD webservice or request anything from the user there. It has to be regular SAML and we don't control user interactions.
> - We can't use constrained delegation (aka. S4U) because the scheduler needs the user's TGT not a service ticket. Users are free to use their TGT however they want from the allocated nodes.
>
>
> Kerberos mailing list Kerb...@mit.edu
> https://mailman.mit.edu/mailman/listinfo/kerberos
AuthZ is on the CA to limit which users can be impersonated. I've
implemented this for other reasons, but it probably won't get open
sourced until mid-late next year (and does a bunch of stuff you probably
won't want/need).
On 10/26/2022 11:34 PM, Jonathan Calmels via Kerberos wrote:
> Hi,
>
> We have a Linux cluster fully kerberized including its own MIT Kerberos KDC which we control.
> Users authenticate to it through a one-way trust with an Active Directory. After being authenticated, users submit their workload with their TGT and the scheduler will forward it to the nodes it allocated (i.e. unconstrained delegation).
> So far everything is working as expected.
>
>
> Now the problem is that we need to support the same workflow from a CI/CD webservice.
> Users authenticate to the CI/CD webservice through SAML and will trigger some kind of job to be scheduled. The scheduler knows the user's principal but doesn't have a TGT associated with it.
>
> Basically, the scheduler needs a way to impersonate users' TGTs to start their workload.
> How does one go about that? given that:
>
>
> - We can't use SPNEGO on the CI/CD webservice or request anything from the user there. It has to be regular SAML and we don't control user interactions.
> - We can't use constrained delegation (aka. S4U) because the scheduler needs the user's TGT not a service ticket. Users are free to use their TGT however they want from the allocated nodes.
>
>
> So far, the only hack we can think of is replicating the AD users into the MIT KDC and writing some kind of GSS service that would issue TGTs for those (given the proper service ticket).
> Something like:
>
> 1. The scheduler does protocol transition with the AD UPN it got from the CI/CD
> 2. The scheduler contacts this GSS service with the resulting service ticket
> 3. The GSS service converts the UPN from the AD realm to its MIT realm counterpart
> 4. If everything checks out, it sends back a TGT for the user (this might involve some unconventional calls to libkadm5)
> 5. The scheduler forwards this TGT as usual
>
>
> Something like:
>
> 1. The scheduler does protocol transition with the AD UPN it got from the CI/CD
> 2. The scheduler contacts this GSS service with the resulting service ticket
> 3. The GSS service converts the UPN from the AD realm to its MIT realm counterpart
> 4. If everything checks out, it sends back a TGT for the user (this might involve some unconventional calls to libkadm5)
> 5. The scheduler forwards this TGT as usual
>
>
> Is there a cleaner alternative? Ideally, one that doesn't involve replicating users.
>
>
> If not, is libgssapi and likadm5 the best way to implement it or would something like a plugin module be better suited?
>
> Thanks for any insight
>
> ________________________________________________
>
> Thanks for any insight
>
> Kerberos mailing list Kerb...@mit.edu
> https://mailman.mit.edu/mailman/listinfo/kerberos
Jeffrey Hutzelman
Oct 27, 2022, 12:37:17 PM10/27/22
to Russ Allbery, Jonathan Calmels via Kerberos, Jonathan Calmels
You don't need libkadm5 for any of this -- all you need to print a service
ticket (even a TGT) is the service's key. Heimdal comes with a program,
kimpersonate, which does this and could easily be used as a basis for your
impersonation service. Naturally, you should be cautious about giving an
application the key for the TGS (even a cross-realm TGS); as Russ points
out, this makes that application equivalent to a KDC.
-- Jeff
On Thu, Oct 27, 2022 at 11:59 AM Russ Allbery <ea...@eyrie.org> wrote:
> Jonathan Calmels via Kerberos <kerb...@mit.edu> writes:
>
ticket (even a TGT) is the service's key. Heimdal comes with a program,
kimpersonate, which does this and could easily be used as a basis for your
impersonation service. Naturally, you should be cautious about giving an
application the key for the TGS (even a cross-realm TGS); as Russ points
out, this makes that application equivalent to a KDC.
-- Jeff
On Thu, Oct 27, 2022 at 11:59 AM Russ Allbery <ea...@eyrie.org> wrote:
> Jonathan Calmels via Kerberos <kerb...@mit.edu> writes:
>
> > So far, the only hack we can think of is replicating the AD users into
> > the MIT KDC and writing some kind of GSS service that would issue TGTs
> > for those (given the proper service ticket). Something like:
>
> > 1. The scheduler does protocol transition with the AD UPN it got from
> > the CI/CD
> > 2. The scheduler contacts this GSS service with the resulting service
> > ticket
> > 3. The GSS service converts the UPN from the AD realm to its MIT
> > realm counterpart
> > 4. If everything checks out, it sends back a TGT for the user (this
> > might involve some unconventional calls to libkadm5)
> > 5. The scheduler forwards this TGT as usual
>
> > the MIT KDC and writing some kind of GSS service that would issue TGTs
> > for those (given the proper service ticket). Something like:
>
> > 1. The scheduler does protocol transition with the AD UPN it got from
> > the CI/CD
> > 2. The scheduler contacts this GSS service with the resulting service
> > ticket
> > 3. The GSS service converts the UPN from the AD realm to its MIT
> > realm counterpart
> > 4. If everything checks out, it sends back a TGT for the user (this
> > might involve some unconventional calls to libkadm5)
> > 5. The scheduler forwards this TGT as usual
>
> Yup, this is also what I would do given your constraints. (You have a
> fairly weird security corner case that requires arbitrary user
> impersonation with no chain of authentication back to the user being
> impersonated, which Kerberos doesn't really natively support for somewhat
> obvious reasons.)
>
> fairly weird security corner case that requires arbitrary user
> impersonation with no chain of authentication back to the user being
> impersonated, which Kerberos doesn't really natively support for somewhat
> obvious reasons.)
>
> > Is there a cleaner alternative? Ideally, one that doesn't involve
> > replicating users.
>
> > replicating users.
>
> You possibly could cheat by giving the GSS service access to the
> cross-realm key so that it can forge TGTs that look to the MIT Kerberos
> KDC as if they were issued by AD. I think this would be roughly
> equivalent from a security standpoint (either way, the GSS service is
> essentially a KDC and has full access to the entirety of your MIT realm),
> but would avoid the need to create principals in your KDC database for
> every user.
>
> The drawback of this approach is that you're likely to need to write some
> low-level Kerberos code to forge the tickets, whereas in your plan above
> you can just generate keytab files for every user and store them on disk
> (again, the GSS service is functionally equivalent to a KDC, so this is
> just the KDC database in another format), and then your GSS service can
> generate TGTs through normal libkrb5 calls using the keytab and doesn't
> have to do anything special.
>
> --
> Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>
> cross-realm key so that it can forge TGTs that look to the MIT Kerberos
> KDC as if they were issued by AD. I think this would be roughly
> equivalent from a security standpoint (either way, the GSS service is
> essentially a KDC and has full access to the entirety of your MIT realm),
> but would avoid the need to create principals in your KDC database for
> every user.
>
> The drawback of this approach is that you're likely to need to write some
> low-level Kerberos code to forge the tickets, whereas in your plan above
> you can just generate keytab files for every user and store them on disk
> (again, the GSS service is functionally equivalent to a KDC, so this is
> just the KDC database in another format), and then your GSS service can
> generate TGTs through normal libkrb5 calls using the keytab and doesn't
> have to do anything special.
>
> --
> Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>
Russ Allbery
Oct 27, 2022, 9:40:09 PM10/27/22
to Jonathan Calmels, Jeffrey Hutzelman, Jonathan Calmels via Kerberos
Jonathan Calmels <jcal...@nvidia.com> writes:
> Thank you for the suggestions, I didn't know about kimpersonate and this
> would indeed solve part of the problem. The reason why I mentioned
> libkadm5 is because we were thinking of relying on
> "kadm5_get_principal_keys" instead of keytabs. This way we could also
> reuse the kadm ACLs and have a rule like "gsshostservice@REALM e
> */ci@REALM".
You can certainly do that, but just be aware that there's not much in the
way of a security difference between that and keytabs. In theory it means
the GSS service doesn't have access to all the user's keys at once but has
to ask for them on demand, but if I were you I'd just colocate the GSS
service with the KDC since the security model is essentially the same, at
which point the KDC database is right there and there's really no
particularly meaningful distinction.
(I have implemented something akin to this before, and that's how I did
it.)
> Forging a cross-realm TGT would definitely be preferable, although I'm
> not sure if it's doable with libkrb5.
It should be doable, but you may have to use really low-level functions to
duplicate what kimpersonate is doing.
> Thank you for the suggestions, I didn't know about kimpersonate and this
> would indeed solve part of the problem. The reason why I mentioned
> libkadm5 is because we were thinking of relying on
> "kadm5_get_principal_keys" instead of keytabs. This way we could also
> reuse the kadm ACLs and have a rule like "gsshostservice@REALM e
> */ci@REALM".
You can certainly do that, but just be aware that there's not much in the
way of a security difference between that and keytabs. In theory it means
the GSS service doesn't have access to all the user's keys at once but has
to ask for them on demand, but if I were you I'd just colocate the GSS
service with the KDC since the security model is essentially the same, at
which point the KDC database is right there and there's really no
particularly meaningful distinction.
(I have implemented something akin to this before, and that's how I did
it.)
> Forging a cross-realm TGT would definitely be preferable, although I'm
> not sure if it's doable with libkrb5.
It should be doable, but you may have to use really low-level functions to
duplicate what kimpersonate is doing.
Greg Hudson
Oct 28, 2022, 12:07:06 AM10/28/22
to Jeffrey Hutzelman, Russ Allbery, Jonathan Calmels via Kerberos, Jonathan Calmels
On 10/27/22 12:36, Jeffrey Hutzelman wrote:
> You don't need libkadm5 for any of this -- all you need to print a service
> ticket (even a TGT) is the service's key. Heimdal comes with a program,
> kimpersonate, which does this and could easily be used as a basis for your
> impersonation service.
MIT krb5 has a sort-of equivalent: "kinit -k -t KDB: username". The KDC
> You don't need libkadm5 for any of this -- all you need to print a service
> ticket (even a TGT) is the service's key. Heimdal comes with a program,
> kimpersonate, which does this and could easily be used as a basis for your
> impersonation service.
is still in the loop, but no password or keytab for the user is
required. (Add "-S krbtgt/OTHERREALM" for a cross-realm TGT.)
Jeffrey Hutzelman
Oct 28, 2022, 8:30:53 AM10/28/22
to Greg Hudson, Russ Allbery, Jonathan Calmels via Kerberos, Jonathan Calmels
Ah, I didn't realize MIT Kerberos had grown the "KDB" keytab method.
That's similar to Jonathan's idea of using the kadmin libraries to extract
the client's key from the kdb, but didn't require wiring custom code. It
does require colocating with a KDC, but I agree with Russ; it's probably
best to do that anyway.
-- Jeff
That's similar to Jonathan's idea of using the kadmin libraries to extract
the client's key from the kdb, but didn't require wiring custom code. It
does require colocating with a KDC, but I agree with Russ; it's probably
best to do that anyway.
-- Jeff
Jonathan Calmels
Nov 9, 2023, 5:53:42 AM11/9/23
to Jeffrey Hutzelman, Greg Hudson, Russ Allbery, Jonathan Calmels via Kerberos
I finally had some time to implement this so here is the link if someone's interested: https://github.com/NVIDIA/sybil
This is a PoC which essentially does what was suggested in this thread. The service can forge TGTs or cross-realm TGTs, although I found the latter less useful since most tool can't deal with those on their own.
I'm sure this can be improved further, but it seems to do the job for the scenario I described initially.
Hopefully, somebody finds it useful. Also, contributions are welcomed if somebody has a slightly different use case in mind.
________________________________
From: Jeffrey Hutzelman <jh...@cmu.edu>
Sent: Friday, October 28, 2022 5:30:41 AM
To: Greg Hudson <ghu...@mit.edu>
Cc: Russ Allbery <ea...@eyrie.org>; Jonathan Calmels via Kerberos <kerb...@mit.edu>; Jonathan Calmels <jcal...@nvidia.com>
Subject: Re: Kerberos protocol transition with unconstrained delegation (i.e. TGT impersonation)
External email: Use caution opening links or attachments
Ah, I didn't realize MIT Kerberos had grown the "KDB" keytab method. That's similar to Jonathan's idea of using the kadmin libraries to extract the client's key from the kdb, but didn't require wiring custom code. It does require colocating with a KDC, but I agree with Russ; it's probably best to do that anyway.
-- Jeff
This is a PoC which essentially does what was suggested in this thread. The service can forge TGTs or cross-realm TGTs, although I found the latter less useful since most tool can't deal with those on their own.
I'm sure this can be improved further, but it seems to do the job for the scenario I described initially.
Hopefully, somebody finds it useful. Also, contributions are welcomed if somebody has a slightly different use case in mind.
________________________________
From: Jeffrey Hutzelman <jh...@cmu.edu>
Sent: Friday, October 28, 2022 5:30:41 AM
To: Greg Hudson <ghu...@mit.edu>
Cc: Russ Allbery <ea...@eyrie.org>; Jonathan Calmels via Kerberos <kerb...@mit.edu>; Jonathan Calmels <jcal...@nvidia.com>
Subject: Re: Kerberos protocol transition with unconstrained delegation (i.e. TGT impersonation)
External email: Use caution opening links or attachments
Ah, I didn't realize MIT Kerberos had grown the "KDB" keytab method. That's similar to Jonathan's idea of using the kadmin libraries to extract the client's key from the kdb, but didn't require wiring custom code. It does require colocating with a KDC, but I agree with Russ; it's probably best to do that anyway.
-- Jeff
0 new messages