lost connection to maxima

[ma...@mendelu.cz]

Dear all,

I had a lecture in computer lab. There were about 15 students on one
sage server. After about 20 minutes we got messages, that Sage lost
connection to Maxima. I did not know how to establish the connection
again and I restarted the virtual server (I run sage on virtual server
under Debain lenny with Xen virtualization).

Can you guess what happened (overloaded server? problems with memory?)
and what to do with this next time?
(I know that this is strange request, but I do not remmber exact error
message, since I was in the classroom under a stress in order to
restore everything again).

My Sage version is:
Sage 3.1.2 was released on September 19th, 2008.

Thank you. Robert

[William Stein]

On Tue, Jan 13, 2009 at 8:36 AM, ma...@mendelu.cz <ma...@mendelu.cz> wrote:
>
> Dear all,
>
> I had a lecture in computer lab. There were about 15 students on one
> sage server. After about 20 minutes we got messages, that Sage lost
> connection to Maxima. I did not know how to establish the connection
> again and I restarted the virtual server (I run sage on virtual server
> under Debain lenny with Xen virtualization).
>
> Can you guess what happened (overloaded server? problems with memory?)
> and what to do with this next time?

How much RAM does the virtual server have?

Couldn't you just do

Action --> Restart worksheet

in the worksheet?

> (I know that this is strange request, but I do not remmber exact error
> message, since I was in the classroom under a stress in order to
> restore everything again).
>
> My Sage version is:
> Sage 3.1.2 was released on September 19th, 2008.
>
> Thank you. Robert
>
> >
>

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[ma...@mendelu.cz]

On 13 Led, 17:47, "William Stein" <wst...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 8:36 AM, ma...@mendelu.cz <ma...@mendelu.cz> wrote:
>
> > Dear all,
>
> > I had a lecture in computer lab. There were about 15 students on one
> > sage server.  After about 20 minutes we got messages, that Sage lost
> > connection to Maxima. I did not know how to establish the connection
> > again and I restarted the virtual server (I run sage on virtual server
> > under Debain lenny with Xen virtualization).
>
> > Can you guess what happened (overloaded server? problems with memory?)
> > and what to do with this next time?
>
> How much RAM does the virtual server have?
>

500 MB


free
total used free shared
buffers cached
Mem: 504828 396664 108164 0 134916
113796
-/+ buffers/cache: 147952 356876
Swap: 273064 0 273064

> Couldn't you just do
>
>   Action --> Restart worksheet
>
> in the worksheet?
>

Thanks for the hint, we try it next time if this problem occurs again.

Robert

[Jason Grout]

ma...@mendelu.cz wrote:
>
>
> On 13 Led, 17:47, "William Stein" <wst...@gmail.com> wrote:
>> On Tue, Jan 13, 2009 at 8:36 AM, ma...@mendelu.cz <ma...@mendelu.cz> wrote:
>>
>>> Dear all,
>>> I had a lecture in computer lab. There were about 15 students on one
>>> sage server. After about 20 minutes we got messages, that Sage lost
>>> connection to Maxima. I did not know how to establish the connection
>>> again and I restarted the virtual server (I run sage on virtual server
>>> under Debain lenny with Xen virtualization).
>>> Can you guess what happened (overloaded server? problems with memory?)
>>> and what to do with this next time?
>> How much RAM does the virtual server have?
>>
>
> 500 MB
>
>
> free
> total used free shared
> buffers cached
> Mem: 504828 396664 108164 0 134916
> 113796
> -/+ buffers/cache: 147952 356876
> Swap: 273064 0 273064

I noticed that for my Sage server, each open worksheet took about 140MB
of RAM. When I ran out of memory, the server froze and lots of errors
happened. To me, it sounds like you probably ran out of memory if you
had 15 worksheets open simultaneously (which I would calculate would
need about 15*140=2100 MB of RAM).

You can test this yourself by opening up your notebook and opening up 15
worksheets in different tabs in your browser. Run a calculation in each
one and check to see how much memory you have then. Also, you might see
how much memory is consumed by running a single worksheet (i.e., check
the memory usage, start up a single worksheet and calculate something,
and then check the memory usage again).

Jason

[ma...@mendelu.cz]

Many thanks, memory seems to be a big problem on my server (I did
tests you suggested on my laptop and each worksheet is consumig a lot
of memory). I have seen that sagenb.org has 2GB memory - like my PC.
How many users wirk simultaneously on sagenb.org?

* I think that I can set up my personal computer as second sage server
for my class
Is it sufficient to move the ~/login/.sage directory to clone
all the work to the second server?

* I will ask our webmaster to increase memory on my virtual server on
Mondays :)
But it supprised me that sage is so memory demanding :(
The students were asked to calculate nothing more than few
limits and draw some 2d graphs.

* Is there any other possibility how to decrease the memory demand? I
tried to open 17 notebooks with notebook(ulimit="-v 10240") but this
did not help. The demand for memory was the same as for notebook()
command.

I wonder how the others use sage in education. Do they have so big
memory on their servers?
Thank you very much.

Robert

[Robert Bradshaw]

On Jan 13, 2009, at 11:39 AM, ma...@mendelu.cz wrote:

> Many thanks, memory seems to be a big problem on my server (I did
> tests you suggested on my laptop and each worksheet is consumig a lot
> of memory). I have seen that sagenb.org has 2GB memory - like my PC.
> How many users wirk simultaneously on sagenb.org?
>
> * I think that I can set up my personal computer as second sage server
> for my class
> Is it sufficient to move the ~/login/.sage directory to clone
> all the work to the second server?
>
> * I will ask our webmaster to increase memory on my virtual server on
> Mondays :)
> But it supprised me that sage is so memory demanding :(
> The students were asked to calculate nothing more than few
> limits and draw some 2d graphs.

I think this is because Sage loads a lot more than your students are
using by default. Periodically, William (or others) has gone through
and audited the sage startup for speed, pruning unnecessary imports--
perhaps the same could be done with an eye towards memory usage. If
you're graphing and plotting, each notebook probably started up a
maxima process as well.

> * Is there any other possibility how to decrease the memory demand? I
> tried to open 17 notebooks with notebook(ulimit="-v 10240") but this
> did not help. The demand for memory was the same as for notebook()
> command.

I would imagine a lot of Sage is in shared object libraries--these
don't have to reside in memory for each individual process, do they?
I wonder if something like pyprocessing could be used, which seems to
effectively re-use the loaded libraries.

> I wonder how the others use sage in education. Do they have so big
> memory on their servers?
> Thank you very much.

I would imagine a couple of GB at least, but I personally don't have
any experience. The summer REU programs that have used Sage here were
on a computer with an atypically large amount of RAM, but that
shouldn't be a necessity.

- Robert

[mabshoff]

On Jan 13, 10:52 am, Jason Grout <jason-s...@creativetrax.com> wrote:
> ma...@mendelu.cz wrote:

<SNIP>

> I noticed that for my Sage server, each open worksheet took about 140MB
> of RAM.  When I ran out of memory, the server froze and lots of errors
> happened.  To me, it sounds like you probably ran out of memory if you
> had 15 worksheets open simultaneously (which I would calculate would
> need about 15*140=2100 MB of RAM).

This is not how this works, i.e. a lot of the memory Sage uses is
shared mappings between libraries, so the first notebook process is
much, much more expensive than subsequent ones. top and the default
interface it uses on Linux is dumb regarding shared mappings, i.e.
every notebook process is reported to use every library that is
dlopened by Sage.

One can reduce Sage's memory consumption by doing a couple things like
stripping its libraries and so on, but this is not done by default.

> You can test this yourself by opening up your notebook and opening up 15
> worksheets in different tabs in your browser.  Run a calculation in each
> one and check to see how much memory you have then.  Also, you might see
> how much memory is consumed by running a single worksheet (i.e., check
> the memory usage, start up a single worksheet and calculate something,
> and then check the memory usage again).

Using Python gives a certain penalty since the memory consumption is
higher than for something written more low level, but as Robert did
remark later in this thread imports a lot of functionality per default
beyond "simple" 2D plotting. And running code that creates a bunch of
graphical output on an image with 500 MB (which the operating system
has to fit in, too) is clearly rather on the low. On top of that
running many Maxima processes on an OS with little free memory will
easily make Maxima fail to allocated more memory because of the way
clisp allocates memory.

> Jason

Cheers,

Michael

[Jason Grout]

mabshoff wrote:
>
>
> On Jan 13, 10:52 am, Jason Grout <jason-s...@creativetrax.com> wrote:
>> ma...@mendelu.cz wrote:
>
> <SNIP>
>
>> I noticed that for my Sage server, each open worksheet took about 140MB
>> of RAM. When I ran out of memory, the server froze and lots of errors
>> happened. To me, it sounds like you probably ran out of memory if you
>> had 15 worksheets open simultaneously (which I would calculate would
>> need about 15*140=2100 MB of RAM).
>
> This is not how this works, i.e. a lot of the memory Sage uses is
> shared mappings between libraries, so the first notebook process is
> much, much more expensive than subsequent ones. top and the default
> interface it uses on Linux is dumb regarding shared mappings, i.e.
> every notebook process is reported to use every library that is
> dlopened by Sage.
>

Okay, I was using top to calculate this. What is a better way to
calculate the actual memory being used? I'd like to have an accurate
idea of the memory requirements of a big Sage server.

Thanks,

Jason

[William Stein]

On Tue, Jan 13, 2009 at 11:39 AM, ma...@mendelu.cz <ma...@mendelu.cz> wrote:
>
> Many thanks, memory seems to be a big problem on my server (I did
> tests you suggested on my laptop and each worksheet is consumig a lot
> of memory). I have seen that sagenb.org has 2GB memory - like my PC.
> How many users wirk simultaneously on sagenb.org?

It has 2GB RAM and *10GB* swap. Having a lot of swap is critical, because
as people leave worksheets running for a while, they can be swapped out without
the machine getting into trouble. You have only 256MB swap, which is a recipe
for disaster. Instead make it something bigger, e.g., a couple gigs at least.

William

>
> * I think that I can set up my personal computer as second sage server
> for my class
> Is it sufficient to move the ~/login/.sage directory to clone
> all the work to the second server?

It should be.

>
> * I will ask our webmaster to increase memory on my virtual server on
> Mondays :)
> But it supprised me that sage is so memory demanding :(
> The students were asked to calculate nothing more than few
> limits and draw some 2d graphs.

Sage demands a lot of memory because each worksheet launches its own
Sage process. We could change Sage so that all users use exactly the
same process. This would require way less memory, but would mean that
if user X starts a calculation running, then Y starts another one,
then Y would not see their result until X does. If X's calculation is
factor(huge number) that would be annoying.

512MB is very very piddly for a server.

> * Is there any other possibility how to decrease the memory demand? I
> tried to open 17 notebooks with notebook(ulimit="-v 10240") but this
> did not help. The demand for memory was the same as for notebook()
> command.
>
> I wonder how the others use sage in education. Do they have so big
> memory on their servers?
> Thank you very much.
>
> Robert
>
>
>> I noticed that for my Sage server, each open worksheet took about 140MB
>> of RAM. When I ran out of memory, the server froze and lots of errors
>> happened. To me, it sounds like you probably ran out of memory if you
>> had 15 worksheets open simultaneously (which I would calculate would
>> need about 15*140=2100 MB of RAM).
>>
>> You can test this yourself by opening up your notebook and opening up 15
>> worksheets in different tabs in your browser. Run a calculation in each
>> one and check to see how much memory you have then. Also, you might see
>> how much memory is consumed by running a single worksheet (i.e., check
>> the memory usage, start up a single worksheet and calculate something,
>> and then check the memory usage again).
>>
>> Jason
> >
>

[mabshoff]

On Jan 13, 3:05 pm, Jason Grout <jason-s...@creativetrax.com> wrote:
> mabshoff wrote:

<SNIP>

> > This is not how this works, i.e. a lot of the memory Sage uses is
> > shared mappings between libraries, so the first notebook process is
> > much, much more expensive than subsequent ones. top and the default
> > interface it uses on Linux is dumb regarding shared mappings, i.e.
> > every notebook process is reported to use every library that is
> > dlopened by Sage.
>
> Okay, I was using top to calculate this.  What is a better way to
> calculate the actual memory being used?  I'd like to have an accurate
> idea of the memory requirements of a big Sage server.

smap and pmap comes to mind. Some pointers to get started:

* http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html
* http://bmaurer.blogspot.com/2006/03/memory-usage-with-smaps.html

The slashdot discussion at

* http://it.slashdot.org/article.pl?sid=06/02/06/120210

also has some lively discussion, but the signal to noise ratio is
pretty bad or normal for slashdot :)

Note that smap has Python binding for Linux, so we might want to
consider using that optionally since not every kernel supports it,
even though on modern system I would guess it will be hard to find one
that doesn't.

In general we are using crappy interfaces to measure memory
consumption from Sage, i.e. get_memory_usage(). We are even parsing
the output from "top" on non-Linux platforms!

> Thanks,
>
> Jason

Cheers,

Michael

[William Stein]

Sage uses /proc/pid/status on Linux. I don't know if this is "crappy"
or not; it's code that I got from
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222
Python Cookbook, by Jean Brouwers

On OSX, indeed it just naively reads top, which is... well just as
good or bad as top at reporting a valid number. I don't know whether
OS X top is sensible or not, though I don't see why it would be wrong.

William

[mabshoff]

On Jan 13, 3:50 pm, "William Stein" <wst...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 3:42 PM, mabshoff

<SNIP>

> > In general we are using crappy interfaces to measure memory
> > consumption from Sage, i.e. get_memory_usage(). We are even parsing
> > the output from "top" on non-Linux platforms!
>
> Sage uses /proc/pid/status on Linux.  I don't know if this is "crappy"

> or not; it's code that I got fromhttp://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286222

> Python Cookbook, by Jean Brouwers

Yes, that is the same "crappy" interface that top uses :). top's
numbers are somewhat accurate if you have a single program linking
against a couple shared libs that aren't used by other programs. But
in many situations these days, i.e. running a KDE or Gnome application
with often dozens of shared libs used by a lot of different programs
the numbers are just crap and inaccurate. This is what those blog
posts complain about since top makes KDE and Gnome look much more
bloated than they actually are.

> On OSX, indeed it just naively reads top, which is... well just as
> good or bad as top at reporting a valid number.  I don't know whether
> OS X top is sensible or not, though I don't see why it would be wrong.

Solaris doesn't even ship top, you have to compile it yourself. And
even then if you compile a top in 32 bit mode it will not tell you the
amount of memory consumed by 64 bit programs. All this can and has
been worked around, but something more native on some platforms might
come in useful. I am bot sure if the answer here is getrusage(), but I
don't think it will be. If you know the limitations of
get_memory_usage() you can work around them. Case in point from 3.2.3
right after startup:

64 bit linux, i.e. sage.math:

sage: get_memory_usage()
668.32421875

----

32 bit OSX 10.5:

sage: get_memory_usage()
'131M+'

----

64 bit OSX 10.5:

sage: get_memory_usage()
'171M+'

---

32 bit Solaris/Sparc:

sage: get_memory_usage()
'81M'

While there should be a difference between 32 and 64 bit, i.e 64 bit
code is larger and consumes more memory, the result from Linux is not
even close to the truth, i.e. I don't think 32 bit Solaris is roughly
a magnitude more efficient than 64 bit Linux.

Either way, the result of get_memory_usage() should be consistent
across platforms and not return a string in some cases and something
else on Linux. It should be a float of the memory used in MB.

> William

Cheers,

Michael

[William Stein]

> sage: get_memory_usage()
> '81M'
>
> While there should be a difference between 32 and 64 bit, i.e 64 bit
> code is larger and consumes more memory, the result from Linux is not
> even close to the truth, i.e. I don't think 32 bit Solaris is roughly
> a magnitude more efficient than 64 bit Linux.
>
> Either way, the result of get_memory_usage() should be consistent
> across platforms and not return a string in some cases and something
> else on Linux. It should be a float of the memory used in MB.

+1 Create a trac ticket. This will be an easy point for the
"fix-the-most-trac-bugs-in-sage contest we're having next week.

Willam

[Jason Grout]

I should retract this statement. I just measured it again and found
that it took about 40MB per worksheet process. To measure this, I
started the sage server, created a new worksheet, and ran "1+1". I then
looked at the free memory reported at the top of top. Then I created
another new worksheet and ran "1+1", and checked the free memory again.
The free memory went down by about 45MB. Michael Abshoff says that
that numbers sounds about right. I don't know if I measured
inaccurately before or if I just misremembered the number. At any rate,
with that number, it seems that 15 simultaneous worksheets would use a
minimum of 15*45=675MB, which is still more than you allocated. And as
others point out, this is before the OS, the sage server, etc.

Jason

[mabshoff]

On Jan 13, 5:00 pm, "William Stein" <wst...@gmail.com> wrote:

<SNIP>

> > Either way, the result of get_memory_usage() should be consistent
> > across platforms and not return a string in some cases and something
> > else on Linux. It should be a float of the memory used in MB.
>
> +1  Create a trac ticket.  This will be an easy point for the
> "fix-the-most-trac-bugs-in-sage contest we're having next week.
>
> Willam

This is now #4971

Cheers,

Michael

[mabshoff]

On Jan 13, 5:50 pm, Jason Grout <jason-s...@creativetrax.com> wrote:
> Jason Grout wrote:

<SNIP>

> > I noticed that for my Sage server, each open worksheet took about 140MB
> > of RAM.  When I ran out of memory, the server froze and lots of errors
> > happened.  To me, it sounds like you probably ran out of memory if you
> > had 15 worksheets open simultaneously (which I would calculate would
> > need about 15*140=2100 MB of RAM).
>
> I should retract this statement.  I just measured it again and found
> that it took about 40MB per worksheet process.  To measure this, I
> started the sage server, created a new worksheet, and ran "1+1".  I then
> looked at the free memory reported at the top of top.  Then I created
> another new worksheet and ran "1+1", and checked the free memory again.
>   The free memory went down by about 45MB. Michael Abshoff says that
> that numbers sounds about right.  I don't know if I measured
> inaccurately before or if I just misremembered the number.  At any rate,
> with that number, it seems that 15 simultaneous worksheets would use a
> minimum of 15*45=675MB, which is still more than you allocated.  And as
> others point out, this is before the OS, the sage server, etc.

Note that a lot of that 45 MBs is a bunch of strings due to the online
help, i.e. if you look at the python heap while running Sage you will
discover that about 12 MB or so of that is the "online" help.

Another thing worth thinking about is forking the various Sage
notebook processes off a master process instead of starting new a new
python process for each one. That way on Linux for example with COW a
lot of the data and also code would be shared by all running
processes. forking either a fully working Sage session or at least
everything but twisted would certainly be a killer feature to
implement since it would allow much quicker worksheet startup as well
as insanely fast doctesting since we can skip the startup time of Sage
for each file. Gary Furnish has looked into doing this, but I am not
sure how far he got.

But maybe we want to take that discussion over to [sage-devel] since
we are getting far, far away form the original problem :)

> Jason

Cheers,

Michael

[Robert Bradshaw]

+1, that sounds like a very good idea. I'd imagine one could have a
fresh copy of sage, and just fork from there for every new doctest/
notebook session. (It would probably be a lot simpler, and still
scale much better, for the notebook to start an unused sage process
in the background and then repeatedly forking that rather than trying
to fork the notebook process itself).

- Robert

[ma...@mendelu.cz]

Thank you for all your answers. I will upgrade my hardware

> 512MB is very very piddly for a server.
>

Yes. I thought that this could be enough, because another my server,
Mathematical Assistant on Web ( http://user.mendelu.cz/marik/maw/index.php?lang=en&form=derivace
) runs pretty well with much smaller memory (
http://wood.mendelu.cz/math/phpsysinfo/index.php?disp=dynamic )
Now you, Jason, Michael and Robert helped me to find the source of the
problems and I will try to solve it. Many thanks to all again.

Robert Marik

[Jason Grout]

Nice sites. I noticed a small typo on the front page of
http://user.mendelu.cz/marik/maw/index.php?lang=en&form=derivace

In the text "The calculators are divided into several groups, the
desription is available", the word "description" is missing the "c".

Thanks,

Jason

[mabshoff]

On Jan 13, 11:28 pm, "ma...@mendelu.cz" <ma...@mendelu.cz> wrote:

Hi Robert,

> Thank you for all your answers. I will upgrade my hardware
>
> > 512MB is very very piddly for a server.
>
> Yes. I thought that this could be enough, because another my server,

> Mathematical Assistant on Web (http://user.mendelu.cz/marik/maw/index.php?lang=en&form=derivace

> ) runs pretty well with much smaller memory (http://wood.mendelu.cz/math/phpsysinfo/index.php?disp=dynamic)
> Now you, Jason, Michael and Robert helped me to find the source of the
> problems and I will try to solve it. Many thanks to all again.

Your server calls Maxima and without taking a closer look it is
unclear to me if you start a Maxima process per user or not. Overall
Maxima is lighter than Sage primarily because it is all common lisp
and does not use any external libraries per default, which is one of
the reasons Sage's memory footprint is larger than Maxima's. OTOH
Maxima is missing a lot of he functionality that Sage provides, but
the price we pay for that is a larger footprint. So if you were to use
Sage "just" as a tool to do calculus (which is mostly handled by
Maxima at the moment) we cannot be less large than Maxima by itself :)

Maybe in the long term we can make all the notebooks share one Sage
process, but I think that will be rather messy and potentially slow
for anything but non-trivial things.

> Robert Marik

Cheers,

Michael

[ma...@mendelu.cz]

>
> In the text "The calculators are divided into several groups, the
> desription is available", the word "description" is missing the "c".
>

Fixed, thanks. Robert

[ma...@mendelu.cz]

On 14 Led, 08:50, mabshoff <Michael.Absh...@mathematik.uni-

dortmund.de> wrote:
>
> Your server calls Maxima and without taking a closer look it is
> unclear to me if you start a Maxima process per user or not. Overall
> Maxima is lighter than Sage primarily because it is all common lisp
> and does not use any external libraries per default, which is one of
> the reasons Sage's memory footprint is larger than Maxima's. OTOH
> Maxima is missing a lot of he functionality that Sage provides, but
> the price we pay for that is a larger footprint. So if you were to use
> Sage "just" as a tool to do calculus (which is mostly handled by
> Maxima at the moment) we cannot be less large than Maxima by itself :)

To finish this thread: I had another lecture in computer lab today. We
had 2GB RAM, very very big swap and 15 rather slow students with
allmost no experiences with computer algebra systems ( = short
worksheets and not many requests in short time). No problem occured
today.

I hope, this information could be usefull for some other educators
using sage.

Thank you for your help.

Robert M.

[mabshoff]

On Jan 19, 10:07 am, "ma...@mendelu.cz" <ma...@mendelu.cz> wrote:
> On 14 Led, 08:50, mabshoff <Michael.Absh...@mathematik.uni-

<SNIP>

Hi Robert,

> To finish this thread: I had another lecture in computer lab today. We
> had 2GB RAM, very  very big swap and 15 rather slow students with
> allmost no experiences with computer algebra systems ( = short
> worksheets and not many requests in short time). No problem occured
> today.

Thanks for the update. Please let us know if you run into any more
trouble or want to share more of your experience with Sage with us.

> I hope, this information could be usefull for some other educators
> using sage.
>
> Thank you for your help.
>
> Robert M.

Cheers,

Michael