PATCH: (updated) %U, %S and %D substitutions in filters AND search base
Derrick 'dman' Hudson
--1yeeQ81UyVL57Vl7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Oct 21, 2003 at 01:51:18PM -0400, Victor....@morganstanley.com=
wrote:
| Finally a question for Wietse: How strongly do you feel about using
| ${name} vs. using %X? If we change the syntax
Either way, make the comprehensive list of available substitutions
clearly marked and easy to find :-). (I belive the relevant parameter
in sample-ldap.cf has the complete list, but it took me a while to
find that when I wanted to change the substitution provided by some
other howto or example)
| we should probably rename
| "result_filter" to "result_format". Many new users are confused by the
| name thinking it is a constraint on the result attributes, rather than a
| format string.
This sounds fine (good, even) to me (FWIW).
-D
--=20
Q: What's the difference between a computer salesman and a used
car salesman?
A: A used car salesman knows when he's lying.
=20
http://dman13.dyndns.org/~dman/
--1yeeQ81UyVL57Vl7
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)
iD8DBQE/lY7JiB6vp1xAVUARAsxZAJ9qtYpkKvuHVNQA1jxumfec+N0+nwCcD6Ta
WvM5gKeUAf8fyQ1nA9GVyME=
=jHZ8
-----END PGP SIGNATURE-----
--1yeeQ81UyVL57Vl7--
Liviu Daia
[...]
> Finally a question for Wietse: How strongly do you feel about using
> rename "result_filter" to "result_format". Many new users are confused
> by the name thinking it is a constraint on the result attributes,
> rather than a format string.
FWIW, there may be a better way to handle all this, inspired by the
SASL functions in OpenLDAP. As I see it:
(1) A regexp should map the lookup keys (that is, the e-mail addresses)
to LDAP URIs;
(2) The LDAP dictionary should be able to handle full LDAP URIs on
input.
Among other things, special_result_attribute could then actually
provide chaining of LDAP maps.
On an unrelated topic, I suspect special_result_attribute is
slightly broken (it doesn't work as expected when the number of result
attributes is not the same as the number of special attributes), but I
didn't have time to look at it in detail so far. I also suspect very
few people actually understand what special_result_attribute is supposed
to do. :-)
Regards,
Liviu Daia
--
Dr. Liviu Daia e-mail: Liviu...@imar.ro
Institute of Mathematics web page: http://www.imar.ro/~daia
of the Romanian Academy PGP key: http://www.imar.ro/~daia/daia.asc
Liviu Daia
>
> > FWIW, there may be a better way to handle all this, inspired by
> > the SASL functions in OpenLDAP. As I see it:
> >
> > (1) A regexp should map the lookup keys (that is, the e-mail
> > addresses) to LDAP URIs;
>
A special version of ${N} could take care of that (a non-escaping
version is needed too).
[...]
> > (2) The LDAP dictionary should be able to handle full LDAP URIs on
> > input.
>
> These are already supported in special_result_attribute, just not for
> top level queries. What is the advantage of LDAP URIs for queries
Consistency with special_result_attribute.
> (example please)?
Like I said, this could be used for LDAP map chaining. My personal
motivation for this is automatic generation of group aliases, when the
addresses are not readily available in LDAP in the required form, and
need some (simple) munging. Instead of writing a LDAP map for each
group, a single map could take care of that. This is essentially an
extension of special_result_attribute.
> > Among other things, special_result_attribute could then actually
> > provide chaining of LDAP maps.
>
> How is this different from the "URL" support in
> special_result_attribute today.
It makes possible processing of the intermediate results between
recursion stages.
> > On an unrelated topic, I suspect special_result_attribute
> > is slightly broken (it doesn't work as expected when the number
> > of result attributes is not the same as the number of special
> > attributes), but I didn't have time to look at it in detail so far.
>
> It works well for me, but I only use DN valued attributes, not URL
> valued attributes.
Like I said, look at the way special attributes are counted.
> > I also suspect very few people actually understand what
> > special_result_attribute is supposed to do. :-)
>
> I think the list of people just two: LaMont Jones and I :-) Search the
> archives, I have tried to explain it a few times.
Oh, I believe I do understand how it works. My point is, we are
members of a pretty exclusive minority. :-)
Liviu Daia
>
> > > How will escaping of the input address be handled?
> >
> > A special version of ${N} could take care of that (a
> > non-escaping version is needed too).
>
> (a concrete example please)?
This is a common way of mapping e-mail addresses to DNs. An
examples have been given by the author of the patch.
> > > These are already supported in special_result_attribute, just not
> > > for top level queries. What is the advantage of LDAP URIs for
> > > queries
> >
> > Consistency with special_result_attribute.
> >
> > > (example please)?
> >
> > Like I said, this could be used for LDAP map chaining. My
> > personal motivation for this is automatic generation of group
> > aliases, when the addresses are not readily available in LDAP in the
> > required form, and need some (simple) munging. Instead of writing a
> > LDAP map for each group, a single map could take care of that. This
> > is essentially an extension of special_result_attribute.
>
> This is not an example... Is the existence of the group dynamic, or is
> the membership dynamic?
Both.
> If the latter what is wrong with groups that include members via the
> specicial_result_attribute...
Ok, here's an example:
dn: cn=group,ou=Groups,dc=example,dc=com
objectClass: posixGroup
cn: group
gidNumber: 1001
memberUid: user1
memberUid: user2
dn: uid=user1,ou=People,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: CourierMailAccount
cn: First1 Last1
sn: Last1
uid=user1
uidNumber: 2001
gidNumber: 1001
mail: First1...@example.com
homeDirectory: /usr/users/user1
loginShell: /bin/sh
dn: uid=user2,ou=People,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: CourierMailAccount
cn: First2 Last2
sn: Last2
uid=user1
uidNumber: 2002
gidNumber: 1001
mail: First2...@example.com
homeDirectory: /usr/users/user2
loginShell: /bin/sh
special_result_attribute doesn't work here. Granted, this is a
design problem with the DIT. However, it's not an uncommon one, and
it's not necessarily easy to fix.
> > > > Among other things, special_result_attribute could then
> > > > actually provide chaining of LDAP maps.
> > >
> > > How is this different from the "URL" support in
> > > special_result_attribute today.
> >
> > It makes possible processing of the intermediate results between
> > recursion stages.
>
> What does this mean? Example?
LDAP map chaining would allow memberUid above to be rewritten to a
LDAP URI mapping uid to mail.
> > > > On an unrelated topic, I suspect special_result_attribute
> > > > is slightly broken (it doesn't work as expected when the number
> > > > of result attributes is not the same as the number of special
> > > > attributes), but I didn't have time to look at it in detail so
> > > > far.
> > >
> > > It works well for me, but I only use DN valued attributes, not URL
> > > valued attributes.
> >
> > Like I said, look at the way special attributes are counted.
> >
>
> I looked at the code again (20030921) and it is correct. Please
> justify this claim.
From src/util/dict_ldap.c:
897 if (ldap_is_ldap_url(vals[i])) {
898 if (msg_verbose)
899 msg_info("%s[%d]: looking up URL %s", myname,
900 recursion, vals[i]);
901 rc = ldap_url_parse(vals[i], &url);
902 if (rc == 0) {
903 rc = ldap_search_st(dict_ldap->ld, url->lud_dn,
904 url->lud_scope, url->lud_filter,
905 url->lud_attrs, 0, &tv,
906 &resloop);
907 ldap_free_urldesc(url);
908 }
909 } else {
Now, what happens if url->lud_attrs has a different number of
attributes than dict_ldap->num_attributes? Distinguishing between
ordinary and special attributes by the count is not a very good idea.
> > > I think the list of people just two: LaMont Jones and I :-) Search
> > > the archives, I have tried to explain it a few times.
> >
> > Oh, I believe I do understand how it works. My point is, we are
> > members of a pretty exclusive minority. :-)
>
> Prove you are member of this minority! :-) :-)
I believe at this point I'm supposed to say something like: "all
members recognize me as such". :-) But then again, perhaps this is not
the case.
Liviu Daia
> On Wed, 22 Oct 2003, Liviu Daia wrote:
>
> > > This is not an example... Is the existence of the group dynamic,
> > > or is the membership dynamic?
> >
> > Both.
>
> means that the lookup of the group itself is via or similar.
>
> query_filter = cn=%u
>
> I don't understand what you mean when you say "both"?
In the real-world example I had in mind the group name itself was
defined as a member of another group. Anyway, this is not important to
the discussion.
> > Ok, here's an example:
> >
> > dn: cn=group,ou=Groups,dc=example,dc=com
> > objectClass: posixGroup
> > cn: group
> > gidNumber: 1001
> > memberUid: user1
> > memberUid: user2
>
> Right the schema is buggy, this should be:
>
> memberDN: uid=user1,ou=People,dc=example,dc=com
> memberDN: uid=user2,ou=People,dc=example,dc=com
Then it would no longer be a posixGroup, would it? :-) A lot of
Linux fans seem to be using LDAP for network-based authentication these
days, as a substitute for NIS / NIS+. Should they change their LDAP
scheme, other things would break. Like I said, this is broken, but it's
not necessarily easy to fix.
> In which case special_result_attribute works just fine. So what what
> you like special_result_attribute to do? Do you want a "format"
> template to substitute the result attribute into a DN or URL (default
> %s)? Something along the lines of:
>
> special_result_attribute_format = "uid=%s,ou=People,dc=example,dc=com"
No, because this would solve the Wrong Problem (TM). Among other
things, special_result_attribute makes the optimistic assumption that
it can iterate over and over without ever encountering an object of an
unexpected type. Basically, here again, recursion seems wrong. As a
replacement, I'm suggesting that the special attribute, if present, be
fed to another map (what I call "chaining"). The result at the end of
the "chain" would be the result of the lookup. I was initially thinking
of another LDAP map for that purpose (which would allow the chain to be
extended), but any map type would be fine (in which case the chain would
necessarily end there).
> The problem is perhaps that you want to overload POSIX groups as email
> groups and that POSIX groups are not "normalized" (use member DN
> references) presumably for efficiency.
FWIW, I don't "want" anything of this kind. I was just describing
what seems to be fashionable among some groups of people these days.
[...]
> I still don't see why this translates into a requirement to process
> top level query "URIs" on input.
For consistency reasons, and also because a LDAP URI is easier to be
stored in a LDAP entry than a Christmas tree of parameters defining a
Postfix LDAP map. :-) Basically, that would be a simpler version of map
chaining.
[...]
> I have independently of this in the past contemplated augmenting the
> attribute array with a hash that designates the special attributes,
> it does look like this is necessary for URI valued special attribute
> recursion.
>
> The only question is whether some attribute names are likely to be
> special only for the top level query or only for the URI subqueries,
> do you think that special attributes need contextual qualifiers?
This is precisely the reason why recursion is bad here: you don't
know in advance the type of objects it will run into. Worse, this
depends on input from a database which may not be under your control.
With my map chaining, the chaining depth would be under control, and
each object could be transformed by its own rules.
Liviu Daia
> On Wed, 22 Oct 2003, Liviu Daia wrote:
>
> > > special_result_attribute_format =
> > > "uid=%s,ou=People,dc=example,dc=com"
> >
> > No, because this would solve the Wrong Problem (TM). Among
> > other things, special_result_attribute makes the optimistic
> > assumption that it can iterate over and over without ever
> > encountering an object of an unexpected type. Basically, here
> > again, recursion seems wrong. As a replacement, I'm suggesting
> > that the special attribute, if present, be fed to another map (what
> > I call "chaining"). The result at the end of the "chain" would be
> > the result of the lookup. I was initially thinking of another LDAP
> > map for that purpose (which would allow the chain to be extended),
> > but any map type would be fine (in which case the chain would
> > necessarily end there).
>
> of DN valued members (used in some common schemas). For this special
> result attribute works quite well. Groups defined this way can have
> subgroups, so the expansion is recursive.
>
> In terms of your idea of map chaining, this is a map that chains back
> into itself, and the nesting depth is not fixed (though it is now
> bounded by the recursion limit parameter).
>
> Map chaining sounds interesting. Do you have a proposed syntax that
> does not exclude the current recursive design for groups of groups?
I'd leave special_result_attribute alone (well, after fixing the
special attribute behaviour). I'd also enhance domain, query_filter and
result_filter to support regexp and pcre mapping:
domain = regexp:/^@(.*\.)?my\.domain$/
query_filter = regexp:/^(.*)$/:(mailacceptinggeneralid=${1})
result_filter = regexp:/^(.*)$/:${1}
Are there any security implications of the last two? Apart from that,
the syntax you suggest below looks fine. I'd stay away from LDAP URIs
until the other things settle down.
> > > The only question is whether some attribute names are likely
> > > to be special only for the top level query or only for the URI
> > > subqueries, do you think that special attributes need contextual
> > > qualifiers?
> >
> > This is precisely the reason why recursion is bad here: you
> > don't know in advance the type of objects it will run into. Worse,
> > this depends on input from a database which may not be under your
> > control. With my map chaining, the chaining depth would be under
> > control, and each object could be transformed by its own rules.
> >
>
> Yes, LDAP map chaining can be more general, but then complications
> arise, what if the special result attribute needs to be subjected to
> more than one query (look in base A with query and scope A, else if
> no result in base B with query B)? This can happen when a group has
> members of non-uniform types. Where does this stop?!
We don't really need to go there. That situation could be handled
like this:
chain_attributes = A, B
A_chain_map = ldap:/path/A.cf
B_chain_map = ldap:/path/B.cf
The final result would be the union of results of A and B chains. For
unexpected object types:
(1) If the requested attribute is not found, no lookup is performed;
(2) If the requested attribute is found, but its value is not what the
map expects, the situation is indistinguishable from "normal"
garbage input;
(3) Minimal format validation might be done with the domain parameter;
(4) Chaining depth would still be controlled by recursion_limit.
> At some point it becomes reasonable to ask the user to provide a more
> natural schema (perhaps with partially duplicated data) or use SQL
> which can perform arbitrary projections of restrictions of joins...
>
> Anyway this is not to discourage you, but do you have a design
> proposal for how map chaining would be specified?
>
> chain_attributes = x, y, z
> x_chain_map = ldap:/path/x.cf
> y_chain_map = ldap:/path/y.cf
> z_chain_map = ldap:/path/z.cf
>
> Are the chained maps going to be defined in advance? (I think you are
> saying yes, since you don't fully trust the LDAP data quality, which
> is my situation also).
Yes. This may change if I can figure out a reasonable way to encode
a Postfix LDAP map as a LDAP URI (perhaps the "extension" filed can be
used to specify the non-standard parameters). However, the security
implications of such a change don't look very promising; f.i. various
kinds of mutual map recursions can be imagined.
> How does this interact with the currently supported (in part modulo
> the counting bug) true dynamic groups (URI valued attributes), as
> opposed to map chaining of regular attributes?
Like I said, I would fix it, but leave it alone apart from that.
On a side note, I'm not sure this doesn't have security implications
though.
> These are the issues. It may be best for you come up with a complete
> design rather than respond to this point by point, or we can try to
> hash it out via the list...
Well, I believe that is all there is to it. Did I miss anything?
> (I am still in shock from the useful to understand the issues, but
> frustrating for lack of outcome masquerading thread :-)
I'm still working at fixing the maps exceptions patch. Among other
things, there are minor conflicts with it in the latest snapshot. I'm
also trying to have a life in the real world. :-) Perhaps I'll get to
finish it this week-end. The LDAP thing is lower on the priority list.