Identification with ldap.
Thierry Dumont
As I explained some days ago on sage-support (am I on the good list
here?), I would like to add ldap identification to Sage.
This seems necessary for some projects:
1) building a Sage server for the whole French math community
(we have a large ldap server used for identifications on some services)
2) In my University, I want to build a prototype of Sage server for the
students (there exists an active directory)
and there are other projects.
The idea is to add a Python module (using Python-ldap) at the good place
in Sage. If the identification succeeds, then -at the first login- a
Sage user will be created.
My question is: where to put this module? I can find this, yes: but I
lack time. If somebody can explain me where and how the
identification/login is done in the notebook, I will certainly ave a lot
of time...
What happens when I press "Sign in" ?
yours
t.d.
----
Thierry Dumont
Institut C. Jordan CNRS & Université Lyon 1.
Villeurbanne, France.
mabshoff
> Hello,
Hi Thierry,
> As I explained some days ago on sage-support (am I on the good list
> here?), I would like to add ldap identification to Sage.
would be great.
> This seems necessary for some projects:
>
> 1) building a Sage server for the whole French math community
> (we have a large ldap server used for identifications on some services)
> 2) In my University, I want to build a prototype of Sage server for the
> students (there exists an active directory)
>
> and there are other projects.
> The idea is to add a Python module (using Python-ldap) at the good place
> in Sage. If the identification succeeds, then -at the first login- a
> Sage user will be created.
>
> My question is: where to put this module? I can find this, yes: but I
> lack time. If somebody can explain me where and how the
> identification/login is done in the notebook, I will certainly ave a lot
> of time...
> What happens when I press "Sign in" ?
next three months and he has some long list of ideas. He might already
know what to do where, so hopefully in a couple hours he will answer.
> yours
> t.d.
> ----
> Thierry Dumont
Michael
> Institut C. Jordan CNRS & Université Lyon 1.
> Villeurbanne, France.
>
> 1KDownload
Timothy Clemans
2008/7/6 Thierry Dumont <tdu...@math.univ-lyon1.fr>:
I think login information is first handled by the fuction
getCredentials. The actual username and password checking is done in
the function requestAvatarId. Right above requestAvatarId is code for
creating a new user.
Function getCredentials: guard.py, line 259,
http://www.sagemath.org/hg/sage-main/file/c25e04ebfb67/sage/server/notebook/guard.py
Function requestAvatarId: avatars.py, line 55,
http://www.sagemath.org/hg/sage-main/file/c25e04ebfb67/sage/server/notebook/avatars.py
Philippe Saade
i know it would be specific to Linux boxes but i think there is a
python_PAM module that would help Sage rely on native Linux PAM
mechanism..
Maybe an option..
Philippe
Michael_D_G
although it is not going so smoothly.
I was trying to override the classes in user_db.py with some custom
classes.
Fine. That seems ok. The UserDatabase
is used throughout twist.py which seemed to be the server software.
avatars.py and guard.py
rely heavily on this. Unfortunately there seems to be a somewhat
independed db of users in
notebook. This is really 'twisting' my head around. I have code using
python-ldap for doing
the queries (happy to share though it is not pretty) but the user
management is very hard to follow.
Is there some cruft here? Do notebooks keep their own subsets of users
for the purposes of
limited collaborations?
Any leg up would be nice!
On Jul 7, 12:38 am, "Philippe Saade" <psa...@gmail.com> wrote:
> Hi,
>
> i know it would be specific to Linux boxes but i think there is a
> python_PAM module that would help Sage rely on native Linux PAM
> mechanism..
>
> Maybe an option..
>
> Philippe
>
> On Sun, Jul 6, 2008 at 8:38 PM, Timothy Clemans
>
>
> > Hi,
>
> > 2008/7/6 Thierry Dumont <tdum...@math.univ-lyon1.fr>:
> >> Great!
William Stein
>
> I am in the processes of implementing ldap authentication as well...
> although it is not going so smoothly.
> I was trying to override the classes in user_db.py with some custom
> classes.
> Fine. That seems ok. The UserDatabase
> is used throughout twist.py which seemed to be the server software.
> avatars.py and guard.py
> rely heavily on this. Unfortunately there seems to be a somewhat
> independed db of users in
> notebook. This is really 'twisting' my head around. I have code using
> python-ldap for doing
> the queries (happy to share though it is not pretty) but the user
Why is it not pretty?
> management is very hard to follow.
> Is there some cruft here?
Of course.
> Do notebooks keep their own subsets of users
> for the purposes of limited collaborations?
By "notebooks" do you mean "worksheets"?
If so, then yes each worksheet has a list of users that
are allowed to collaborate on that worksheet. It's
just a plain text list of usernames.
Each complete notebook has a list of users. This is
completely independent of twisted, avatar.py, guard.py,
etc. except that those files will query this list to decide
whether login credentials are valid.
Here is an example of using the users() method to get a list
of them:
EXAMPLES:
sage: n = sage.server.notebook.notebook.Notebook(tmp_dir())
sage: n.create_default_users('password')
Creating default users.
sage: list(sorted(n.users().iteritems()))
[('_sage_', _sage_), ('admin', admin), ('guest', guest),
('pub', pub)]
sage: n.delete()
The list of users for a notebook is a very very simple data
structure. A user is just an instance of the class User that
is defined in user.py. The list of users (the attribute
__users of the notebook instance) is just pickled as part
of the notebook object. That's all there is to
users in the Sage notebook.
-- William
Michael_D_G
No I am not talking about the worksheets but the notebook.
What confused me was that a UserDatabase looked like
the place were all the users were being stored. I saw user.py
but that doesn't have a container class. There is clearly some
redundancy between UserRecord and User.
What you seem to be saying is that rather than
put my effort into user_db.py, I should be reworking users.py.
What would be nice is to have a single containter class that
holds the user information. I sort of started in this direction
by making UserDatabase subclass dict. The password,
rather than actually being a string or hash is its own
class. That subclasses the new-style "object" class.
The thing is we can override the __eq__ method. This
means that when you check if two password objects
are equal you can use what ever authentication
method the system is configured to use. This
pushes all the complication for authentication
away from the notebook (which probably
shouldn't have it) and into classes for
managing user information.
A first draft (messy prototype):
class UserRecord(object):
def __init__(self, username, passwd=None, email=None):
self.username = username
self.passwd = LDAP_Password(username,passwd)
self.email = email
class LDAP_Password(object):
def __init__(self, username,passwd):
self.uid = username
if passwd:
valid =
authenticate_uid(base,server_ip,server_port,self.uid,passwd)
if valid:
#cache password
self.passwd=passwd
else:
self.passwd=None
else:
self.passwd=None
def __eq__(self,passwd):
# Allows checking against either a string
# or another LDAP_Password
# object.
if type(self)==type(passwd):
pw=passwd.passwd
else:
pw = passwd
#Password never checked
if self.passwd == None:
valid =
authenticate_uid(base,server_ip,server_port,self.uid,pw)
if valid:
self.passwd=pw
return str(self.passwd).__eq__(pw)
def __repr__(self):
# Really insecure
return str(self.passwd)
One thing I am not happy about is that I am implicitly using this as a
caching method. If
the plain text password is authenticated then I store it and use it to
check instead
of the ldap. More properly the caching mechanism should be at least
hashing and
probably pushed into authenticate_uid.
-michael
On Jul 18, 8:23 am, "William Stein" <wst...@gmail.com> wrote:
William Stein
Hi,
I don't like the really insecure storing of the plain text password at
all; that's
of course terrible since it will get pickled to disk probably. But I like
the design strategy you suggest above, especially overloading __eq__
and improving user.py. I very very very strongly encourage you to create
one single tiny patch that does something right in this direction and submit
it to be reviewed and included in Sage. It will help you get a sense of
how the Sage workflow goes without biting off more than you can easily
chew.
-- William
Thierry Dumont
> Thanks so much William!
>
>....
Hello,
I have posted some questions about ldap identification in Sage (in the
notebook), and I started to look a bit more in details... But it seems
that you are much more advanced than me (I'm absolutely not a specialist
of ldap programing, and twisted or zope are a strange world for me);
I'm progressing very very slowly... but I'll need a ldap identification
to convince my University to make a Sage server for the students
(actually using an active directory) and to build a server for the
French Math community (there, we will use a true ldap).
If you are looking for a beta-tester, It could be me :-)
t.d.
--
Thierry Dumont. Institut Camille Jordan -- Mathematiques--
Univ. Lyon I,43 Bd du 11 Novembre 1918, 69622
- Villeurbanne Cedex - France.
tdu...@math.univ-lyon1.fr web: http://math.univ-lyon1.fr/~tdumont
Michael_D_G
Dear William or anyone....
I tried to unify the objects in user_db and users and let the
passwords proxy ldap if they are
not the default users (which could use the basic hash+salt method). I
ran into this odd problem
that on startup it was trying to set a non-existent admin with the
password. On some
investigation I ran across the part of run_notebook.py quoted below
which I just don't understand.
How does it ever get to the else branch???
http://www.sagemath.org/hg/sage-main/file/f1f9a0884cdf/sage/server/notebook/run_notebook.py
The tabbing seems to have go lost here.
103 if not nb.user_exists('admin'):
104 reset = True
105
106 if reset:
107 passwd = get_admin_passwd()
108 if reset:
109 nb.user('admin').set_password(passwd)
110 print "Password changed for user 'admin'."
111 else:
112 nb.create_default_users(passwd)
113 print "User admin created with the password you specified."
114 print "\n\n"
115 print "*"*70
116 print "\n"
117 if secure:
118 print "Login to the SAGE notebook as admin with the password you
specified above."
119 #nb.del_user('root')
On Jul 18, 3:43 pm, "William Stein" <wst...@gmail.com> wrote:
William Stein
>
>
> Dear William or anyone....
>
>
> I tried to unify the objects in user_db and users and let the
> passwords proxy ldap if they are
> not the default users (which could use the basic hash+salt method). I
> ran into this odd problem
> that on startup it was trying to set a non-existent admin with the
> password. On some
> investigation I ran across the part of run_notebook.py quoted below
> which I just don't understand.
> How does it ever get to the else branch???
That code looks to me like it is just crap. I'm not insulting
anybody by saying
that since I wrote that code. I would like to strongly encourage you
(Michael D. G.)
to take a clean copy of Sage, change just that code by say getting rid of the
else, test things out, make a patch, and submit it for inclusion in sage. This
is a hopefully simple straightforward task and will give you a sense for the
sage project's workflow (plus you get your name in big red lights as a
contributor).
-- William
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Michael_D_G
come back to it
in a month or so.
I love the sage idea and what has been put together so far is VERY
impressive.
Unfortunately the warts from the developer side,
are fairly daunting to me. I have some experience with Django, and
some
other web frameworks. I really wish the web server portion of sage
could be factored out. LDAP
authentication, as well as web templates, session management, and
forms, are well handled by other software
frameworks. Not that there is anything fundamentally wrong with
twisted, or zope for that matter.
It just from the web application side, twisted is fairly low level and
building ones one
web application framework from the ground up from twisted is highly
non-trivial.
Yes, yes, it is easy to criticize and whats really needed is energy
and effort... well
I hope to try some more... but the structure of the files makes it
very difficult
to follow. When one only has a limited amount of time to contribute it
is really
depressing spending 9/10 ths of it just trying to figure out what
calls
what. Ideally it would be nice to make a small patch as you suggest
but
I don't see how. Every tweak I make breaks something else. Perhaps if
I knew how it was all connected a small patch would be possible. I
don't, and
trial and error is very inefficient.
On Jul 21, 12:30 am, "William Stein" <wst...@gmail.com> wrote:
>
> > Dear William or anyone....
>
> > I tried to unify the objects in user_db and users and let the
> > passwords proxy ldap if they are
> > not the default users (which could use the basic hash+salt method). I
> > ran into this odd problem
> > that on startup it was trying to set a non-existent admin with the
> > password. On some
> > investigation I ran across the part of run_notebook.py quoted below
> > which I just don't understand.
> > How does it ever get to the else branch???
>
> That code looks to me like it is just crap. I'm not insulting
> anybody by saying
> that since I wrote that code. I would like to strongly encourage you
> (Michael D. G.)
> to take a clean copy of Sage, change just that code by say getting rid of the
> else, test things out, make a patch, and submit it for inclusion in sage. This
> is a hopefully simple straightforward task and will give you a sense for the
> sage project's workflow (plus you get your name in big red lights as a
> contributor).
>
> -- William
>
>
>
root
> mich...@gmail.com writes:
>
> Yes, yes, it is easy to criticize and whats really needed is energy
> and effort... well I hope to try some more... but the structure of the
> files makes it very difficult to follow. When one only has a limited
> amount of time to contribute it is really depressing spending 9/10 ths
> of it just trying to figure out what calls what. Ideally it would be
> nice to make a small patch as you suggest but I don't see how. Every
> tweak I make breaks something else. Perhaps if I knew how it was all
> connected a small patch would be possible. I don't, and trial and
> error is very inefficient.
The Mythical Man-Month (Fred Brooks, a MUST read book) has an interesting
example of a dinosaur stuck in a tarpit. The dinosaur can lift any foot
but cannot escape the tarpit. Really large systems, like Axiom and Sage,
have massive, dinosaur-sized tarpits of code.
Developers of these systems have the belief that it is sufficient to write
code that the machine understands. After all, any part of the system can
be understood by anyone with sufficient effort (see... we can lift any
foot you point at...). And every piece of code is perfectly clear
despite having many levels of inherited behavior and unstated background
mathematical theory.
The problem is that tarpit gets larger and deeper with every passing
day and with every additional piece of code added. As the current
developers move on and new people join, they have no way to understand
the prior work except to try to read each file and "integrate".
The best solution I've found it Knuth's Literate Programming (Axiom
uses noweb). The fundamental idea is to write for people while
writing the code so the machine can follow along. This adds about
two to three times as much work for the original developer but
saves the time for everyone else in the future.
And, yes, I know that nobody has the time. But if you don't take
the time now, the next person can only guess, probaby incorrectly,
or give up and rewrite it.
Do you want your code to live?
Tim
root
>
>think I did. But when you realize that an awful lot of code isn't meant
>to "live" -- it's meant to solve an immediate problem, and then be
>discarded -- then you don't care whether it lives or not.
>
>Axiom was clearly designed to live, rather than solve an immediate
>problem. But how much code is out there that was designed with a 30-year
>horizon?
More to the point, how much code is in Sage that is designed for the
30 year horizon? Will a matrix inversion still give the right answer
30 years from now? If so, how many dozens of times will it have to
be reimplemented because nobody can understand the current code?
Which of the 16 kinds of matrix inversions are used and how do they
differ in terms of input domains, performance, robustness, etc?
Should I use the Maxima inversion or the Sympy inversion or the
Lapack inversion or implement a new one?
The front ends will likely change. Axiom's hyperdoc help system is
being reimplemented using a firefox front end. The build system is
about to be moved from using 'make' to using 'asdf'. The file system
layout is disappearing, replaced by latex literate books.
<http://axiom.axiom-developer.org/axiom-website/documentation.html>
Sage's notebook is certain to be replaced.
Even so, if we want the next generation of developers to contribute
effectively it seems important to write down what they need to know so
they don't give up in frustration. That's a serious loss. We need a
computational mathematician to be able to contribute effectively
without becoming frustrated. We also need to capture the special
knowledge that lies behind the contributed code since there will be
very few experts worldwide in a given research area. Would you
recognize and be able to fix a bug in a feynman quantum computation?
William has stated that he doesn't care about this issue since he
wants a system NOW that he can use to do his number theory research.
But it is fundamentally about the quality of the system. The 4Ms
don't care because they pay people to learn the code.
Google and Microsoft, having invested real dollars into the
effort, ought to try for a higher standard and a longer horizon.
I really wish they would hire a team with a mandate to make this
code literate.
Tim
Mike Hansen
> effort, ought to try for a higher standard and a longer horizon.
> I really wish they would hire a team with a mandate to make this
> code literate.
Tim: Please create a new thread if you would like to push literate
programming as it is distracting to the topic at hand.
Thierry and Michael_D_G: I played around with python-ldap for a
little bit, and this is definitely doable. There is already some code
for modularizing the authentication and user code at
http://trac.sagemath.org/sage_trac/ticket/2936 , but it still needs
some work. I has sort of sat there since there hadn't been many
requests for the functionality it provides until now. I suspect that
I could have a working first version of the notebook using LDAP in an
evening, and I will try get to it in the next few days.
--Mike
William Stein
>
>> Google and Microsoft, having invested real dollars into the
>> effort, ought to try for a higher standard and a longer horizon.
>> I really wish they would hire a team with a mandate to make this
>> code literate.
>
> Tim: Please create a new thread if you would like to push literate
> programming as it is distracting to the topic at hand.
+1.
> Thierry and Michael_D_G: I played around with python-ldap for a
> little bit, and this is definitely doable. There is already some code
> for modularizing the authentication and user code at
> http://trac.sagemath.org/sage_trac/ticket/2936 , but it still needs
> some work. I has sort of sat there since there hadn't been many
> requests for the functionality it provides until now. I suspect that
> I could have a working first version of the notebook using LDAP in an
> evening, and I will try get to it in the next few days.
Don't just think one should use the code at #2936 -- it has quality
issues. However, looking at the code is helpful.
Mike -- I can help too on getting ldap support into the notebook, as I've
mentioned earlier in this thread.
-- William
Michael_D_G
working code. If it is worth anything I will let you know what I
believe would be helpful. Ldap authentication itself is actually kind
of trivial. Like almost anything else there is a establish connection
portion, a query portion and a process response portion. The establish
a connection portion is a “bind” and this is often all that is needed
for authentication since ordinary users should be able to establish a
connection with their credentials.
The problem as I see it is NOT getting ldap to work but aggregating
the user databases/information into one place. There is one dictionary
for users in notebook and a seemingly independent one in twist.py. The
twist.py uses UserDatabase which is a container for the UserRecord
class. The notebook.py is based on the User class in user.py.
Notebook.py simply maintains a dictionary of these. My plan was to
merge user.py/user_db.py into a single file. I will attach the code I
started below for reference called glasslab_user.py. Basically the the
idea is to merge the methods of the User class and the UserRecord
class.
Here is my use case. I have a small lab with mostly linux boxes, one
or two window boxes. We run a web server and svn repo for our students
and collaborators. We do image compression, analysis and recognition
research. The authentication we have goes through an LDAP. It is able
to do the authentication for both our web server and all the linux
boxes as well. On odd numbered Thursdays during a full moon we can get
the windows boxes to authenticate using pgina but thats another story.
I want to set up a group called sage_users and another group called
sage_admins on my ldap. On a login attempt I would like sage to see if
the user one of these groups and treat them accordingly. At this point
I have a small group and am not really concerned about speed so every
piece of information, user name, email address, password, may be
queried from the ldap each time. For a larger system it might make
sense to cache this information. I am guessing that sage should
probably keeps a couple of local “accounts” that do not depend on the
LDAP such as admin, guest, and pub. I do not want to allow my users to
create accounts... I have a separate mechanism for this via the ldap.
While this is not critically important, I have an apache web server
and I do a reverse proxy to an internal machine that runs sage.I would
eventually like the sage jobs to run on a couple of very fast multi-
core machines. We also have access to some Beowulf clusters so
distributed jobs would be very nice. All our users have disk space on
NFS shared accounts. It would also be good if the data related to that
user were somehow stored in their directory although thats is
definitely only a “nice to have”.
Here was the outline for my proposed approach:
Make the UserDatabase class subclass dict. I had some difficulty
overriding its constructor but it is reasonably easy to override the
getitem/setitem methods and key method. The key method makes a query
to the ldap to get all the users and takes a union of the ldap users
with the local users which is
made unique to allow for caching, and returns a list of usernames. On
a call to getitem, we can make a call to the superclass dict getitem
to see if we have the user locally. If not we catch the keyerror
exception, and check if the user is in the ldap. If so great, if not
we raise a keyerror exception and let the caller deal with it (not a
current user case). For local users
we can use the current md5 hashed password mechanism. For ldap users
the user object can either become a thin proxy, that is all calls to
get or set methods are passed to and from ldap, or on first call a
local object is created by coping all the fields from the lap EXCEPT
the password. The password should be maintained at the LDAP.
The User object contains a password object which has an overridden
__eq__ class as explained in an earlier post. This password object
also contains the username. Passwords are checked like this
Bob.password == input_password, where Bob is a user, Bob.password is a
password object and input_password is either a string or a password
object. The __eq__ makes the ldap query if the user is an ldap user or
the hash check if the user is local.
One way to stage this is to make the modifications of localizing the
User code first with the help of William and others familiar with the
structure of sage. The password class would be implemented just to
support the hash method... I think I basically have that. The reason
to do it this way is that William is probably more able to mess with
twist.py and notebook.py and run_notebook.py without running into all
the problems I did.
Once all the user stuff is in ONE PLACE ... it is fairly trivial to
drop in the ldap stuff on password as a “small patch”.
I am including all the files I was building.... the modifications
necessary to the rest of the code I am not really sure... there only
seem to be two places where there is a database of users but I am not
really certain how guard.py and avatars.py mix in.
A word of warning on the attached files.
(1) I scrubbed info from glasslab_conf.py obviously...I used some code
that makes a dict look like an object... not sure whether this is that
useful beyond cool.
(2) I am abroad and am including some script to set up an ssh tunnel.
Our ldap is only open in our local network so I wanted to be able to
debug from outside. This is the glasslab_tunnel.py ... it mostly works
although it depends on having ssh set up properly.
(3) the glasslab_authentication.py code has all the functions that use
pyldap. As I said I was trying to set up tunnels possibly so this can
be simplified a bit. Also on our ldap the userid is not the primary
key for authentication. You need the common name or cn. So I do an
anonymous search for userid to common name first. The authentication
then tries to bind using the username/password for authentication.
Password fails or succeeds on that bind. Some folks will have the
searches also password protected. Since our network is internal we do
not care but more general support would require the initial search uid
to password to bind with a system user/password stored by sage and
with limited permissions (to search). Thats about it. Hopefully this
is useful.
Will check back in an pitch in when I can. I love sage and greatly
appreciate the hard work of everyone who is contributing.
http://glasslab.engr.ccny.cuny.edu/glasslab/files/sage_ldap.tgz
Michael_D_G
is unfinished. I ran out of time so please don't
expect too much.