Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
nsswitch
2 views
Skip to first unread message
David Holland
Jul 4, 2015, 3:04:37 AM7/4/15
to
so I've been meaning for some time (usual peanut gallery comments
here) to look at the nsswitch and getpw* (and similar) code, for the
following reasons:
- we want to be able to kick db 1.85 out of libc when the time for
the mythical libc bump comes;
- this should not involve losing the ability to use db 1.85 (or
db4/db5/whatever, too) to hash the password file or other system
databases;
- this needs to work even with static linking;
- therefore the nsswitch logic needs to support forking
subprocesses as an alternative to loading .so plugin files;
- and it can't currently do this.
and,
- the "compat" mode should be generalized so you can use it with
arbitrary backends, which currently AFAIK doesn't quite work;
- the password file code is a big mess.
Also, cursory examination in the past has suggested that the nsswitch
code is also itself kind of a mess.
I have now done some preliminary investigation, and have the following
preliminary conclusions:
- The architecture of the nsswitch code may have had noble
intentions, but it's far more general than really needed; and its data
structures are more general still. This has several consequences; one
is that it's messy to interface to (as readily observed in the
password code); one is that it's possible for an external nss module
to provide meaningless combinations of methods; another is that it
uses a lot of arbitrary strings internally and there's nothing to
crosscheck for typos; and finally, while the dispatching code does
seem to more or less be legal C it isn't remotely typesafe.
- The interfacing, at least for the builtin data sources, is the
transpose of what it ought to be; for abstraction reasons there ought
to be first sources, and then methods; but it's set up with methods
first and then sources. This is a direct contributor to (though not
the only cause of) the level of mess in getpwent.c.
- If we had cvs rename the first step I would take is to move both
the nsswitch code and its various clients to their own subdir of libc,
something like "sysdb" or "systemdb" (name to be bikeshedded later...)
Right now the code's spread across net/ and gen/ with no structure.
However, we don't have cvs rename so I guess this won't be happening
anytime soon.
- Since we can't have code that's polymorphic over function types in
C, and cutpasting the logic for iterating over sources into every
method of every data feed is not a winner, we should have something
like the vnode dispatching code, which came about the way it is (with
argument structures) largely in order to be able to handle lots of
assorted methods uniformly. This is not as safe as one might like, but
it's a lot safer than what the nsswitch dispatching is currently
doing. And it gets rid of a lot of the use of arbitrary strings too.
- Rather than dispatching by passing some default function pointers
to nsdispatch() and having it do magic, things should work by fetching
a source object from the nsswitch code and then calling methods on it
explicitly. (And, the source objects for the different data feeds
should be different types -- I don't think there's going to be much to
gain from having them all be the same and it offers the opportunity
for confusion. But this isn't entirely clear.)
- As far as I can tell nsdispatch() is not actually an outside-facing
function; it may have been conceived that way originally (e.g. to be
used by amd to get amd maps) but currently there is no data feed it
supports that makes sense for anything outside libc to be calling.
Therefore, we shouldn't have to provide compat for it, which is good
because rearchitecting things and providing compat for it would be a
big headache.
- Providing binary compat for existing 3rd-party nss modules looks
quite doable, although after rearchitecting the source interface
should probably be different. This will require patching the ones that
exist; but there aren't very many and this shouldn't be a problem. (I
only see two in pkgsrc.)
- It ought to be possible to use db 1.85, or sqlite, or db[456] from
pkgsrc, or whatever else to hash the local password file. (And other
"files" data feeds, too.) At the moment I'm not sure how choosing one
of these should relate to nsswitch modules, although obviously there
has to be something external that's linked to whatever database
library. (I'm sort of wondering if there should be an independent
switch in libc for key/value databases.) Right now there's no
abstraction for this at all in the password code.
- After the rearchitecting (and maybe after the mythical libc bump
too) I'd expect the NIS and Hesiod password lookup code to live in
external nss modules and not in libc itself. With luck this and other
cleanup will allow kicking a bunch of sunrpc stuff out of libc too
when the bump comes.
Comments?
I realize this is all very broad generalizations, but it's too early
to be very specific.
--
David A. Holland
dhol...@netbsd.org
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
here) to look at the nsswitch and getpw* (and similar) code, for the
following reasons:
- we want to be able to kick db 1.85 out of libc when the time for
the mythical libc bump comes;
- this should not involve losing the ability to use db 1.85 (or
db4/db5/whatever, too) to hash the password file or other system
databases;
- this needs to work even with static linking;
- therefore the nsswitch logic needs to support forking
subprocesses as an alternative to loading .so plugin files;
- and it can't currently do this.
and,
- the "compat" mode should be generalized so you can use it with
arbitrary backends, which currently AFAIK doesn't quite work;
- the password file code is a big mess.
Also, cursory examination in the past has suggested that the nsswitch
code is also itself kind of a mess.
I have now done some preliminary investigation, and have the following
preliminary conclusions:
- The architecture of the nsswitch code may have had noble
intentions, but it's far more general than really needed; and its data
structures are more general still. This has several consequences; one
is that it's messy to interface to (as readily observed in the
password code); one is that it's possible for an external nss module
to provide meaningless combinations of methods; another is that it
uses a lot of arbitrary strings internally and there's nothing to
crosscheck for typos; and finally, while the dispatching code does
seem to more or less be legal C it isn't remotely typesafe.
- The interfacing, at least for the builtin data sources, is the
transpose of what it ought to be; for abstraction reasons there ought
to be first sources, and then methods; but it's set up with methods
first and then sources. This is a direct contributor to (though not
the only cause of) the level of mess in getpwent.c.
- If we had cvs rename the first step I would take is to move both
the nsswitch code and its various clients to their own subdir of libc,
something like "sysdb" or "systemdb" (name to be bikeshedded later...)
Right now the code's spread across net/ and gen/ with no structure.
However, we don't have cvs rename so I guess this won't be happening
anytime soon.
- Since we can't have code that's polymorphic over function types in
C, and cutpasting the logic for iterating over sources into every
method of every data feed is not a winner, we should have something
like the vnode dispatching code, which came about the way it is (with
argument structures) largely in order to be able to handle lots of
assorted methods uniformly. This is not as safe as one might like, but
it's a lot safer than what the nsswitch dispatching is currently
doing. And it gets rid of a lot of the use of arbitrary strings too.
- Rather than dispatching by passing some default function pointers
to nsdispatch() and having it do magic, things should work by fetching
a source object from the nsswitch code and then calling methods on it
explicitly. (And, the source objects for the different data feeds
should be different types -- I don't think there's going to be much to
gain from having them all be the same and it offers the opportunity
for confusion. But this isn't entirely clear.)
- As far as I can tell nsdispatch() is not actually an outside-facing
function; it may have been conceived that way originally (e.g. to be
used by amd to get amd maps) but currently there is no data feed it
supports that makes sense for anything outside libc to be calling.
Therefore, we shouldn't have to provide compat for it, which is good
because rearchitecting things and providing compat for it would be a
big headache.
- Providing binary compat for existing 3rd-party nss modules looks
quite doable, although after rearchitecting the source interface
should probably be different. This will require patching the ones that
exist; but there aren't very many and this shouldn't be a problem. (I
only see two in pkgsrc.)
- It ought to be possible to use db 1.85, or sqlite, or db[456] from
pkgsrc, or whatever else to hash the local password file. (And other
"files" data feeds, too.) At the moment I'm not sure how choosing one
of these should relate to nsswitch modules, although obviously there
has to be something external that's linked to whatever database
library. (I'm sort of wondering if there should be an independent
switch in libc for key/value databases.) Right now there's no
abstraction for this at all in the password code.
- After the rearchitecting (and maybe after the mythical libc bump
too) I'd expect the NIS and Hesiod password lookup code to live in
external nss modules and not in libc itself. With luck this and other
cleanup will allow kicking a bunch of sunrpc stuff out of libc too
when the bump comes.
Comments?
I realize this is all very broad generalizations, but it's too early
to be very specific.
--
David A. Holland
dhol...@netbsd.org
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
Edgar Fuß
Jul 5, 2015, 6:49:54 AM7/5/15
to
> Comments?
It's reassuring to see that someone seems to care about nsswitch.
Seing no comment on PR#49804 made me believe otherwise.
> - therefore the nsswitch logic needs to support forking
> subprocesses as an alternative to loading .so plugin files;
As this would make the libraries pulled in by the NSS method not being
pulled in to the libc consumer, I would regard this as a big win. I
regularily ran into problems (before switching to nslcd) with nss_ldap
pulling in a different (e.g. pkgsrc vs. base, _r vs. non-_r) version of
libldap, libcrypto or whatever.
But wouldn't it be a more general approach to rip most of the code out of
libc and switch to a client/server model? I.e. a nsswitchd listening on a
socket?
I think to remember IRIX using an elegant solution with a pseudo-filesystem,
but the drawback would be how to add custom sources to it.
> I only see two in pkgsrc.
There would be another one if somebody cared to import my nslcd port.
> I realize this is all very broad generalizations, but it's too early
> to be very specific.
I'm afraid I can't sensibly comment on your other points, sorry.
It's reassuring to see that someone seems to care about nsswitch.
Seing no comment on PR#49804 made me believe otherwise.
> - therefore the nsswitch logic needs to support forking
> subprocesses as an alternative to loading .so plugin files;
pulled in to the libc consumer, I would regard this as a big win. I
regularily ran into problems (before switching to nslcd) with nss_ldap
pulling in a different (e.g. pkgsrc vs. base, _r vs. non-_r) version of
libldap, libcrypto or whatever.
But wouldn't it be a more general approach to rip most of the code out of
libc and switch to a client/server model? I.e. a nsswitchd listening on a
socket?
I think to remember IRIX using an elegant solution with a pseudo-filesystem,
but the drawback would be how to add custom sources to it.
> I only see two in pkgsrc.
There would be another one if somebody cared to import my nslcd port.
> I realize this is all very broad generalizations, but it's too early
> to be very specific.
David Holland
Jul 5, 2015, 1:05:36 PM7/5/15
to
On Sun, Jul 05, 2015 at 12:49:37PM +0200, Edgar Fu? wrote:
> > Comments?
> It's reassuring to see that someone seems to care about nsswitch.
> Seing no comment on PR#49804 made me believe otherwise.
There's an unfortunate tendency for new package submissions to sit
> > Comments?
> It's reassuring to see that someone seems to care about nsswitch.
> Seing no comment on PR#49804 made me believe otherwise.
around, regardless of what they're about...
> > - therefore the nsswitch logic needs to support forking
> > subprocesses as an alternative to loading .so plugin files;
>
> As this would make the libraries pulled in by the NSS method not being
> pulled in to the libc consumer, I would regard this as a big win. I
> regularily ran into problems (before switching to nslcd) with nss_ldap
> pulling in a different (e.g. pkgsrc vs. base, _r vs. non-_r) version of
> libldap, libcrypto or whatever.
it uses pkgsrc openssl and base openssl is a different version.
(The dlfcn(3) man page remains unclear on whether a dlopen'd image can
have an isolated symbol name scope, but my understanding has always
been that this sort of thing doesn't work.)
> But wouldn't it be a more general approach to rip most of the code out of
> libc and switch to a client/server model? I.e. a nsswitchd listening on a
> socket?
anytime you have a daemon, the daemon can die... and with nsswitch
that becomes critical: if you can't look up passwords you can't su to
restart the daemon. Plus since the daemon itself is necessarily linked
to libc, you get into an interesting situation if it itself
accidentally tries to use anything nsswitched.
So maybe it should be supported as a supplementary method, and
connecting to an already-running global daemon is only slightly
different from forking a private one, but I don't think it should be
the general method.
> I think to remember IRIX using an elegant solution with a
> pseudo-filesystem, but the drawback would be how to add custom
> sources to it.
/nsswitch/passwd, and /nsswitch is a virtual fs whose usermode driver
knows how to e.g. do NIS?
> > I only see two in pkgsrc.
> There would be another one if somebody cared to import my nslcd port.
Edgar Fuß
Jul 5, 2015, 2:41:39 PM7/5/15
to
> There's an unfortunate tendency for new package submissions to sit
> around, regardless of what they're about...
Yes, but I thought that if anyone would seriously be using nss_ldap, he
> around, regardless of what they're about...
would run into FreeRADIUS crashing as soon as it uses LDAP, sshd crashing
as soon as it uses PAM etc. He would then probably look for nslcd in pkgsrc,
see it's missing, think about how hard it would be to port and then realize
someone else has already done that two years earlier.
> I imagine there's no chance of a nsswitch plugin .so file working if
> it uses pkgsrc openssl and base openssl is a different version.
your code runs into that function without hitting the breakpoint. Well,
you have two copies of that function, one from libldap and another from
libldap_r...
> if you can't look up passwords you can't su to restart the daemon.
EF> I think to remember IRIX using an elegant solution with a
EF> pseudo-filesystem, but the drawback would be how to add custom
EF> sources to it.
DH> Not sure I follow how this would be set up: libc reads from (e.g.)
DH> /nsswitch/passwd, and /nsswitch is a virtual fs whose usermode driver
DH> knows how to e.g. do NIS?
I don't remember. I dimly recall you may have registered yourself at
/nss/passwd/nis and /nss/passwd would merge all the /nss/passwd/* sources.
0 new messages