Using Sage with a large number of undergraduate students.
Thierry Dumont
I would like to know if someone as used Sage with undergraduate
students. My question is mainly a question about the material which
would be necessary.
We are currently building a project (this means: asking for money, doing
a lot a bureaucracy and so on) on Sage in my University: the idea is to
start the replacement of the current commercial system (Maple) by Sage.
Nowadays, students use Maple on a network of Windows machines. We would
like to make a large Sage server so that all the students use the web
interface.
I can add to Sage an identification on the ldap server (actually active
directory) of the University, and Sage accounts will be created
automatically.
The more difficult question for us is: which machine? One can expect to
have about 200 students using Sage at the same time, with large
subgroups of students doing the same thing at the same time. Ok, as it
is undergraduate students, they will not make very large computations...
But what about the "scalability" of Sage, of the web server?
We are currently looking at machines with 2x4 core processors, 32 gb of
RAM, 2x450 Gb of disk (raid1). Larger machines exists, but are extremely
expensive. We can ask for two machines like this.
As anyone some experience in this domain?
Yours,
t.d.
Dan Drake
> I would like to know if someone as used Sage with undergraduate
> students. My question is mainly a question about the material which
> would be necessary.
>
> We are currently building a project (this means: asking for money,
> doing a lot a bureaucracy and so on) on Sage in my University: the
> idea is to start the replacement of the current commercial system
> (Maple) by Sage. Nowadays, students use Maple on a network of Windows
> machines. We would like to make a large Sage server so that all the
> students use the web interface.
I don't have any experience with the above, although I'm guessing the
sagenb.org machine deals with similar loads. I definitely want to hear
about your experiences, as I have distant plans for doing the same thing
at my university.
My own plan is to run a server on a powerful personal machine and let
students create their own accounts, and when they start complaining that
it's too slow, pass their complaints on and ask for a better server. :)
Dan
--
--- Dan Drake <dr...@mathsci.kaist.ac.kr>
----- KAIST Department of Mathematical Sciences
------- http://mathsci.kaist.ac.kr/~drake
William Stein
>
> I would like to know if someone as used Sage with undergraduate
> students. My question is mainly a question about the material which
> would be necessary.
>
> We are currently building a project (this means: asking for money, doing
> a lot a bureaucracy and so on) on Sage in my University: the idea is to
> start the replacement of the current commercial system (Maple) by Sage.
> Nowadays, students use Maple on a network of Windows machines. We would
> like to make a large Sage server so that all the students use the web
> interface.
>
> I can add to Sage an identification on the ldap server (actually active
> directory) of the University, and Sage accounts will be created
> automatically.
>
> The more difficult question for us is: which machine? One can expect to
> have about 200 students using Sage at the same time, with large
There is no way you can have 200 students all using exactly one
sage notebook server on the same port *at the same time* and
have it feel snappy still. I've done classes with 30 people at once
and that worked OK -- the server I used was sage.math (16 1.8Ghz
opterons from 2005). If you ran say 6 different servers, even on the
same hardware, at once, that would probably work fairly well.
Probably using two of the 8-core 32GB of RAM machines you list
below, with say three sage notebook servers on each, would work
reasonably well.
If you publish the relevant notebooks the students need, and
just redirect students from some master login page to one of those
six distinct servers, this could work pretty well.
> subgroups of students doing the same thing at the same time. Ok, as it
> is undergraduate students, they will not make very large computations...
> But what about the "scalability" of Sage, of the web server?
To emphasize again, I doubt it scales to more than 30 users all hammering
the server at once.
> We are currently looking at machines with 2x4 core processors, 32 gb of
> RAM, 2x450 Gb of disk (raid1). Larger machines exists, but are extremely
> expensive. We can ask for two machines like this.
>
> As anyone some experience in this domain?
sagenb.org obviously has a lot of usage.
I basically started it as a server from scratch
1 month ago, and over 750 people created
accounts on it during the last month.
-- William
kcrisman
> To emphasize again, I doubt it scales to more than 30 users all hammering
> the server at once.
>
size server even 15-20 at once becomes very problematic.
- kcrisman
William Stein
One thing I should emphasize is that it's not necessarily a hardware
or bandwith limitation, i.e., in my experience running multiple servers
on different ports on the same hardware allows one to go serve
more users at once.
William
Jason Grout
Can anyone comment on the possibility of running 25-30 sage sessions
from the command line on a modest configuration? In other words, is the
problem in running 25-30 sage processes, or is the bottleneck in the
webserver and notebook interface?
Based on your comment about running several servers on the same box, it
seems that the problem is that the web server cannot handle very many
concurrent connections. In that case, the recent query about using sage
with mod_python or some other higher performance solution might be worth
looking at again.
Jason
William Stein
If your hardware is pretty good (which the OP's hardware is), the
problem is definitely the webserver and notebook interface.
Running many sage sessions at once gets around this.
Note -- if the notebook servers all operated on the same
data (via a central database or files on the filesystem or something),
then one could have the best of both worlds... I guess.
But I doubt I'm putting another month of my life into the
Sage notebook anytime in the near future.
> Based on your comment about running several servers on the same box, it
> seems that the problem is that the web server cannot handle very many
> concurrent connections. In that case, the recent query about using sage
> with mod_python or some other higher performance solution might be worth
> looking at again.
Maybe it would be....
William
Robert Bradshaw
Yep, I remember someone a while back was going to convert the
interface to use a database backend to make it scale better, but I
don't think it ever got finished.
As a workaround, you could start up several notebooks at once and
assign 20 or so students per port, which should work fine.
- Robert
Alex Clemesha
Note -- if the notebook servers all operated on the same
data (via a central database or files on the filesystem or something),
then one could have the best of both worlds... I guess.
But I doubt I'm putting another month of my life into the
Sage notebook anytime in the near future.
As we discussed with William, but for others who are interested,
via SQLAlchemy, and it is something that is definitely worth doing.
The two files here describe our entire data scheme:
http://trac.knoboo.com/browser/trunk/knoboo/knoboo/database/
and with SQLAlchemy you can use SQLite, MySQL, PostgreSQL, etc
as the centralized database behind the scenes.
> Based on your comment about running several servers on the same box, it
> seems that the problem is that the web server cannot handle very many
> concurrent connections. In that case, the recent query about using sage
> with mod_python or some other higher performance solution might be worth
> looking at again.
connections, read more about that here: twistedmatrix.com, especially
see all the 'big' companies using Twisted.
In a web application like the notebook, the webserver should be doing
only 2 main things: 1) passing snippets of code to another process (the 'kernel')
that actually evaluates the code, and 2) talking to a database to preserve
the input and output (among other less frequent data access actions).
We have spend a majority of our effort on Knoboo trying to make
it a robust and scalable web application (like, for example, the
'frontend' is totally decoupled from the backend 'kernel').
What's missing from Knoboo, and what is so great about the Sage Notebook,
is all the awesome usability features like @interact, etc.
I'm optimistic that we will be able to merge both our best attributes
in due time.
-Alex
--
Alex Clemesha
clemesha.org
Ondrej Certik
Indeed, that'd be awesome. Knoboo is lightweight. I had to stop
running Sage on my virtual server (with only about 360MB of virtual
ram) because it was eating several hundreds of megabytes of memory.
Knoboo is running just fine.
Ondrej
Timothy Clemans
Did you have to stop supporting Sage altogether or just the Sage Notebook?
Ondrej Certik
<timothy...@gmail.com> wrote:
>
> Ondrej,
>
> Did you have to stop supporting Sage altogether or just the Sage Notebook?
Not sure what you mean by supporting. I was running the sage notebook
from Sage so that anyone can log into it, just like sagenb.org. But my
virtualserver doesn't have enough memory for that. So now I only run
knoboo notebook:
This seems to work well. And I just don't run Sage permanently on that
server. I miss the features from the Sage notebook though.
Ondrej
Thierry Dumont
>
> If your hardware is pretty good (which the OP's hardware is), the
> problem is definitely the webserver and notebook interface.
> Running many sage sessions at once gets around this.
>
ok, if I understand correctly, running "many" servers (listening on
different ports) will make the job.
> Note -- if the notebook servers all operated on the same
> data (via a central database or files on the filesystem or something),
> then one could have the best of both worlds... I guess.
> But I doubt I'm putting another month of my life into the
> Sage notebook anytime in the near future.
>
Can you give me some hints? The notebook servers cannot share the datas?
May be we can have some support to develop something...
Yours
t.d.
>
Jason Grout
> We have spend a majority of our effort on Knoboo trying to make
> it a robust and scalable web application (like, for example, the
> 'frontend' is totally decoupled from the backend 'kernel').
>
> What's missing from Knoboo, and what is so great about the Sage Notebook,
> is all the awesome usability features like @interact, etc.
> I'm optimistic that we will be able to merge both our best attributes
> in due time.
>
Like others have said, the interact and other features from the sage
notebook are important (important enough that I can't run Knoboo for
what I need). However, I'd *love* to run knoboo as a front end. If
@interact and these other features were added to knoboo and knoboo was
included as an optional spkg, I'm sure the user base for knoboo would
skyrocket, at least compared to the current user base. My guess is that
can only be really good for knoboo, and the sooner, the better.
Thanks for all your work on knoboo!
Jason
William Stein
>
> Alex Clemesha wrote:
>
>> We have spend a majority of our effort on Knoboo trying to make
>> it a robust and scalable web application (like, for example, the
>> 'frontend' is totally decoupled from the backend 'kernel').
>>
>> What's missing from Knoboo, and what is so great about the Sage Notebook,
>> is all the awesome usability features like @interact, etc.
>> I'm optimistic that we will be able to merge both our best attributes
>> in due time.
>>
>
> Like others have said, the interact and other features from the sage
> notebook are important (important enough that I can't run Knoboo for
> what I need). However, I'd *love* to run knoboo as a front end. If
> @interact and these other features were added to knoboo and knoboo was
Just to add, the interact code we wrote for Sage is GPLv2+
so I think legally it could be included in Knoboo, and socially
I am enthusiastic about you doing so. The more code sharing
between the Sage and Knoboo projects, the better.
William
> included as an optional spkg, I'm sure the user base for knoboo would
> skyrocket, at least compared to the current user base. My guess is that
> can only be really good for knoboo, and the sooner, the better.
>
> Thanks for all your work on knoboo!
>
> Jason
>
>
>
> >
>
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
Robert Bradshaw
> William Stein a écrit :
>
>>
>> If your hardware is pretty good (which the OP's hardware is), the
>> problem is definitely the webserver and notebook interface.
>> Running many sage sessions at once gets around this.
>>
>
> ok, if I understand correctly, running "many" servers (listening on
> different ports) will make the job.
Yep.
>> Note -- if the notebook servers all operated on the same
>> data (via a central database or files on the filesystem or
>> something),
>> then one could have the best of both worlds... I guess.
>> But I doubt I'm putting another month of my life into the
>> Sage notebook anytime in the near future.
>>
>
> Can you give me some hints?
See attached. I'm sure much better could be done, but it's enough to
get started.
> The notebook servers cannot share the datas?
All this means is that if someone creates an account on one server,
they can't log into another server.
> May be we can have some support to develop something...
That would be very good, it certainly is technically possible, just
no one's done it yet.
- Robert
Jason Grout
To sum up the discussion about what makes things slow, is it a
file-locking bottleneck with the sage server?
Thanks,
Jason
Mike Hansen
On Tue, Oct 7, 2008 at 1:15 PM, Jason Grout <jason...@creativetrax.com> wrote:
> To sum up the discussion about what makes things slow, is it a
> file-locking bottleneck with the sage server?
I don't think anyone has done any serious profiling of the notebook so
I think that conclusion is quite a bit premature. One thing that
might be helpful would be to run the notebook under Solaris with
DTrace.
--Mike
mabshoff
> > To sum up the discussion about what makes things slow, is it a
> > file-locking bottleneck with the sage server?
>
> I don't think anyone has done any serious profiling of the notebook so
> I think that conclusion is quite a bit premature. One thing that
> might be helpful would be to run the notebook under Solaris with
> DTrace.
notebook. I think that issue is easy to fix (RAND_MAX related
probably), but the even shorter way to profile here is either using
VTune (which is proprietary) or Systemtap (which is free).
If we can get Mike's notebook testing code to do some larger number of
operations in automated mode we could probably easily determine where
the bottlenecks are.
> --Mike
Cheers,
Michael
Justin C. Walker
FWIW, you can also use DTrace as well on Mac OS X, 10.5, as well.
Justin
--
Justin C. Walker, Curmudgeon at Large
Institute for the Absorption of Federal Funds
-----------
If it weren't for carbon-14, I wouldn't date at all.
-----------
William Stein
>
> Hello,
>
> On Tue, Oct 7, 2008 at 1:15 PM, Jason Grout <jason...@creativetrax.com> wrote:
>> To sum up the discussion about what makes things slow, is it a
>> file-locking bottleneck with the sage server?
>
> I don't think anyone has done any serious profiling of the notebook so
> I think that conclusion is quite a bit premature. One thing that
>
> --Mike
This hits the nail on the head. Mike's right on -- essentially no work
at all has gone into any profiling of the notebook. We simply don't
know why or what about it is slow or doesn't work well under a
heavy load. I could guess, since I'm intimately familiar with the code,
but I would likely be wrong. In fact, at present, the *only* "tool" we have for
observing this slowness is anecdotal situations that some of
us have experienced when teaching -- that's it. Like Michael says,
we need good tools for simulating a heavy load, etc.
William
Pierre
> notebook. I think that issue is easy to fix (RAND_MAX related
of the notebook ?
(sorry if this question is somewhat besides the point)
mabshoff
notebook on Solaris is that the key generation takes forever due to
sucky entropy caused by RANDMAX issues. I can fix that, but I want to
make sure that I do not open a gaping security issue.
Cheers,
Michael
William Stein
The way we used to fix this before GNUtls stopped totally sucking
at generating keys, was we used openssl to generate keys if it was
installed on the system, and if not only then fell back to using
GNUtls.
-- William