docbuild getting stuck

[Dima Pasechnik]

Is there anything short of "make distclean && make" to fix the stuck
docbuild:

By "stuck" I mean that I run make and

the output (and logs/dochtml.log) getting stuck at
....
[history_a] no targets are out of date.
[structure] loading cross citations... looking for now-outdated
files... none found
[structure] no targets are out of date.

while there are python processes like this:
python $SAGE_ROOT/src/doc/common/builder.py --no-pdf-links all html
sitting in the memory doing (almost) nothing for hours and hours.

"make doc-clean" does not help.

This must be a bug in sphinx, IMHO...
I have seen this many times already.

[Kannappan Sampath]

On Wed, May 28, 2014 at 3:51 AM, Dima Pasechnik <dim...@gmail.com> wrote:

Is there anything short of "make distclean && make" to fix the stuck
docbuild:

I have tried doing make doc-clean && make and confirm that it does not help in these cases! I know this is unhelpful but I still have this problem on my sage install. IIRC, Travis had this problem but he could have fixed this problem somehow... 

--Kannappan.  

 

By "stuck" I mean that I run make and

the output (and logs/dochtml.log) getting stuck at
....
[history_a] no targets are out of date.
[structure] loading cross citations... looking for now-outdated
files... none found
[structure] no targets are out of date.

while there are python processes like this:
 python $SAGE_ROOT/src/doc/common/builder.py --no-pdf-links all html
sitting in the memory doing (almost) nothing for hours and hours.

"make doc-clean" does not help.

This must be a bug in sphinx, IMHO...
I have seen this many times already.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[leif]

Dima Pasechnik wrote:
> Is there anything short of "make distclean && make" to fix the stuck
> docbuild:
>
> By "stuck" I mean that I run make and
>
> the output (and logs/dochtml.log) getting stuck at
> ....
> [history_a] no targets are out of date.
> [structure] loading cross citations... looking for now-outdated
> files... none found
> [structure] no targets are out of date.
>
> while there are python processes like this:
> python $SAGE_ROOT/src/doc/common/builder.py --no-pdf-links all html
> sitting in the memory doing (almost) nothing for hours and hours.
>
> "make doc-clean" does not help.

I'm currently having exactly the same problem (for the first time!)
after upgrading from 6.2.beta7 to 6.3.beta2 (with 'sage --upgrade ...').

'make doc' first failed with an import error though. After 'make
doc-clean && make', building the reference manual hangs in the first
pass. Interrupting it and rerunning 'make [doc]' doesn't bring me
further, with Python processes waiting forever (pipe_wait).

(Ubuntu 10.04.4 x86_64, GNU make 3.82.)


-leif

--
() The ASCII Ribbon Campaign
/\ Help Cure HTML E-Mail

[Ralf Stephan]

Short of an explanation, there is at least the Sphinx upgrade ticket #16396

[Dima Pasechnik]

for me it is OSX 10.9.3; I don't see this on Linux x86_64.

Does happen on the ARM Linux (running Ubuntu 12.10), though.

By the way, what I see on Linux often is that docbuild has finished successfully,
but has not returned control to the shell properly; hitting Enter gives me
the shell prompt.

[Ralf Stephan]

It happens here on SageMathCloud which is 3.13.0-24-generic #46-Ubuntu x86_64

but not OpenSuSE-12.3

​on ​

AMD, at least so far.

​ I am not sure if it's connected to Out of

Memory errors that I'm seeing too recently on SMC.​

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/ttzqlcNypns/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

[Volker Braun]

Is there an error further up in the log? Strace the hanging process?

Sometimes you need to run "sage -sync-build".

[leif]

Volker Braun wrote:
> Is there an error further up in the log? Strace the hanging proces
>

> --
> You received this message because you are subscribed to the Google
> Groups "sage-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send
> an email to sage-devel+...@googlegroups.com

> <mailto:sage-devel+...@googlegroups.com>.

> To post to this group, send email to sage-...@googlegroups.com

> <mailto:sage-...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[leif]

Volker Braun wrote:
> Is there an error further up in the log?

Only the usual GAP foo ("cannot extend the workspace further" or
something like that).

> Strace the hanging process?

Already did that; the problem is that it's busy waiting (i.e., polling).
Each time I kill the process it is waiting for, it spawns a new
process doing the same (nothing but waiting).

> Sometimes you need to run "sage -sync-build".

I only recall that got broken a while ago, and to that time I fixed only
parts of it. Will give it a try, but I don't see why that should be
necessary.


-leif

[leif]

leif wrote:
> Volker Braun wrote:
>> Is there an error further up in the log?
>
> Only the usual GAP foo ("cannot extend the workspace further" or
> something like that).
>
>> Strace the hanging process?
>
> Already did that; the problem is that it's busy waiting (i.e., polling).
> Each time I kill the process it is waiting for, it spawns a new
> process doing the same (nothing but waiting).
>
>
>> Sometimes you need to run "sage -sync-build".
>
> I only recall that got broken a while ago, and to that time I fixed only
> parts of it. Will give it a try, but I don't see why that should be
> necessary.

Did not make a difference, i.e., it got stuck at exactly the same point.

But after another 'make doc-clean && ./sage --docbuild ...', it now
hangs earlier:

Building reference manual, first pass.

[combinat ] loading pickled environment... not yet created
[combinat ] building [inventory]: targets for 269 source files that are
out of date
[combinat ] updating environment: 269 added, 0 changed, 0 removed


P.P.S.: Just noticed I spoke too soon. It just took ages to continue
(more quickly) after that. Now it's stuck at:

[structure] dumping object inventory... done
[structure] build succeeded.
[tensor ] loading pickled environment... not yet created
[tensor ] building [inventory]: targets for 4 source files that are
out of date
[tensor ] updating environment: 4 added, 0 changed, 0 removed
[tensor ] reading sources... [ 25%] index

reading sources... [ 50%]
sage/tensor/coordinate_patch
reading sources...
[ 75%] sage/tensor/differential_form_element
reading
sources... [100%] sage/tensor/differential_forms

loading cross citations...
[tensor ] pickling environment... done
[tensor ] checking consistency... done
[tensor ] preparing documents... done
[tensor ] writing output... [ 25%] index

writing output... [ 50%]
sage/tensor/coordinate_patch
writing output...
[ 75%] sage/tensor/differential_form_element
writing
output... [100%] sage/tensor/differential_forms
[tensor ] dumping object inventory... done
[tensor ] build succeeded.



-leif

P.S.: I tried earlier a couple of times with 'make doc', serially as
well as in parallel; doesn't make any difference either, i.e., both
behave in the same manner as directly calling './sage --docbuild ...' does.

[Dima Pasechnik]

This all sounds very familiar. (Except that I didn't try strace...)

[Volker Braun]

On Wednesday, May 28, 2014 1:28:58 PM UTC+1, leif wrote:

> Strace the hanging process?

Already did that; the problem is that it's busy waiting (i.e., polling).

  Each time I kill the process it is waiting for [...]

So whats that process doing? "strace -ff" and then sort out which subprocess didn't finish successfully. 

[leif]

Volker Braun wrote:
> On Wednesday, May 28, 2014 1:28:58 PM UTC+1, leif wrote:
>
> > Strace the hanging process?
>
> Already did that; the problem is that it's busy waiting (i.e.,
> polling).
> Each time I kill the process it is waiting for [...]
>
>
> So whats that process doing?

As mentioned, the *child* process does nothing (but pipe_wait), while
the parent is busy waiting (non-blocking, timeout 0).

> "strace -ff" and then sort out which
> subprocess didn't finish successfully.

That will just fill up my disk with n GB/s; it's already hard to follow
the screen output because of what I already mentioned at least twice... ;-)

[Travis Scrimshaw]

I was able to get it to work again (at least with 6.3.beta1) by running "make distclean && make doc-clean && make".

Best,

Travis

[Dima Pasechnik]

There are 3 processes
python /usr/local/src/sage/mav/sage/src/doc/common/builder.py --no-pdf-links all html

The dtruss (OSX clone of strace) of
the main one, with children mentioned in wait4() calls,
and the 2 children, also
python /usr/local/src/sage/mav/sage/src/doc/common/builder.py --no-pdf-links all html
are not doing anything at all

is ike this (a snapshot for 30 sec or so, on OSX):
the PID is 51814, and there are 2 threads:
0xa6c13 and 0xa6c88:

51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: gettimeofday(0x7FFF55CFFB30, 0x0, 0x4) = 1401289551 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: gettimeofday(0x7FFF55CFFB30, 0x0, 0x4) = 1401289551 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: wait4(0xCADF, 0x115E00420, 0x1) = 0 0
51814/0xa6c88: wait4(0xCADE, 0x115E00420, 0x1) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: wait4(0xCADF, 0x115E00420, 0x1) = 0 0
51814/0xa6c88: wait4(0xCADE, 0x115E00420, 0x1) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: wait4(0xCADF, 0x115E00420, 0x1) = 0 0
51814/0xa6c88: wait4(0xCADE, 0x115E00420, 0x1) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: wait4(0xCADF, 0x115E00420, 0x1) = 0 0
51814/0xa6c88: wait4(0xCADE, 0x115E00420, 0x1) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: wait4(0xCADF, 0x115E00420, 0x1) = 0 0
51814/0xa6c88: wait4(0xCADE, 0x115E00420, 0x1) = 0 0

etc etc

with occasional bursts of:
.........
51814/0xa6c13: psynch_mutexdrop(0x7FEBDA0610C8, 0xFF13003, 0xFF12F00) = 0 0
51814/0xa6c13: psynch_cvsignal(0x7FEBDA061098, 0xE6600000E6700, 0xE6600) = 257 0
51814/0xa6c13: psynch_cvsignal(0x7FEBDA061098, 0xE6700000E6800, 0xE6700) = 257 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: psynch_mutexdrop(0x7FEBDA0610C8, 0xFF14203, 0xFF14100) = 0 0
51814/0xa6c13: psynch_cvsignal(0x7FEBDA061098, 0xE6800000E6900, 0xE6800) = 257 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: psynch_mutexdrop(0x7FEBDA0610C8, 0xFF15303, 0xFF15200) = 0 0
51814/0xa6c13: psynch_cvsignal(0x7FEBDA061098, 0xE6900000E6A00, 0xE6900) = 257 0
51814/0xa6c13: select(0x0, 0x0, 0x0, 0x0, 0x7FFF55CFFB30) = 0 0
51814/0xa6c13: psynch_mutexdrop(0x7FEBDA0610C8, 0xFF16403, 0xFF16300) = 0 0
51814/0xa6c13: psynch_cvsignal(0x7FEBDA061098, 0xE6A00000E6B00, 0xE6A00) = 257 0
51814/0xa6c88: select(0x0, 0x0, 0x0, 0x0, 0x115E01030) = 0 0
51814/0xa6c88: psynch_mutexwait(0x7FEBDA0610C8, 0xFF17503, 0xFF17300) = 267482371 0
51814/0xa6c88: psynch_cvwait(0x7FEBDA061098, 0xE6B01000E6C00, 0xE6B00) = 0 0
.......

(appears to me like some multithreading stuff going on...)

HTH,
Dima

[Dima Pasechnik]

On 2014-05-28, Travis Scrimshaw <tsc...@ucdavis.edu> wrote:
> I was able to get it to work again (at least with 6.3.beta1) by running
> "make distclean && make doc-clean && make".

yes, I mentioned that "make distclean..." does fix things,
but this is not what one wants to do, IMHO, unless you work on a
100-core monster of a machine...

[Volker Braun]

This is multiprocessing.pool waiting for children to finish. The question is what happened to the children.

[Ralf Stephan]

How do you exclude out of memory? On SMC for 12 hours now, there are no more doc builds being stuck, but also no more out-of-memory which I got yesterday.

On Wed, May 28, 2014 at 5:56 PM, Volker Braun <vbrau...@gmail.com> wrote:

This is multiprocessing.pool waiting for children to finish. The question is what happened to the children.

--

You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/ttzqlcNypns/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

To post to this group, send email to sage-...@googlegroups.com.

[leif]

Ralf Stephan wrote:
> How do you exclude out of memory?

Having more than 16 GB of free memory available?


-leif

> On SMC for 12 hours now, there are no
> more doc builds being stuck, but also no more out-of-memory which I got
> yesterday.
>
>
> On Wed, May 28, 2014 at 5:56 PM, Volker Braun <vbrau...@gmail.com
> <mailto:vbrau...@gmail.com>> wrote:
>
> This is multiprocessing.pool waiting for children to finish. The
> question is what happened to the children.

--

[Dima Pasechnik]

On 2014-05-28, Volker Braun <vbrau...@gmail.com> wrote:

> This is multiprocessing.pool waiting for children to finish.

To finish, or to start?

> The question is what happened to the children.

They are seemingly alive, but don't do anything at all, according to
dtruss.


The main process occupies over 100Mb of RAM, which is kind of a
lot for a process that builds docs... Its children don't occupy much.

Perhaps the children wait for memory to become available, or something
like this.

>

[Dima Pasechnik]

I also tried building pdf docs, and the result is very similar:
a master docbuild process waiting for a slave (in this case, just one)
to do something, and the slave doing nothing at all.
Using sage's python (running docbuild from Sage's shell)
rather than system's python does not change the
picture either.
Hitting ctrl-C in this case results in something remotely meaningful:

...
[history_a] loading cross citations... looking for now-outdated
files... none found

[history_a] no targets are out of date.

^CProcess PoolWorker-63:
Traceback (most recent call last):
File "/usr/local/src/sage/mav/sage/src/doc/common/builder.py", line 1477, in <module>
Traceback (most recent call last):
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/process.py", line 258, in _bootstrap
getattr(get_builder(name), type)()
File "/usr/local/src/sage/mav/sage/src/doc/common/builder.py",
line 276, in _wrapper getattr(get_builder(document), 'inventory')(*args, **kwds)
File "/usr/local/src/sage/mav/sage/src/doc/common/builder.py",
line 487, in _wrapper self.run()
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/process.py",
line 114, in run self._target(*self._args, **self._kwargs)
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/pool.py",
line 102, in worker x.get(99999)
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/pool.py",
line 548, in get self.wait(timeout)
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/pool.py",
line 543, in wait task = get()
File "/usr/local/src/sage/mav/sage/local/lib/python/multiprocessing/queues.py",
line 376, in get self._cond.wait(timeout)
File "/usr/local/src/sage/mav/sage/local/lib/python/threading.py",
line 358, in wait _sleep(delay)
File "c_lib.pyx", line 176, in sage.ext.c_lib.sage_python_check_interrupt
(sage/ext/c_lib.c:1696) return recv()
File "c_lib.pyx", line 176, in sage.ext.c_lib.sage_python_check_interrupt
(sage/ext/c_lib.c:1696) File "c_lib.pyx", line 73, in sage.ext.c_lib.sig_raise_exception
(sage/ext/c_lib.c:872) KeyboardInterrupt
File "c_lib.pyx", line 73, in sage.ext.c_lib.sig_raise_exception (sage/ext/c_lib.c:872)
KeyboardInterrupt (sage-sh)

Is there a (hidden?) option or an easily tweakable parameter in a
config file to turn the multiprocessing of docbuild off?

Thanks,
Dima

[Volker Braun]

If its really because you are running out of memory then the OOM killer will send a SIGTERM, do you see that in strace?

The doc builder respects the usual SAGE_NUM_THREADS environment variable.

[Dima Pasechnik]

On 2014-05-29, Volker Braun <vbrau...@gmail.com> wrote:
> If its really because you are running out of memory then the OOM killer
> will send a SIGTERM, do you see that in strace?

I have this on OSX, which I guess doesn't have an OOM killer;
anyway, no, the processes just hang, and no,
nothing gets killed.
OSX should pop up a desktop warning saying you're low on RAM, but I
see none of this.

I can imagine a deadlock when a slave waits for a chunk of`
RAM from the OS, while the master waits for the slave...

Perhaps if you make overcommitting on a Linux system really
conservative you'd be able to reproduce this.

[Dima Pasechnik]

On 2014-05-29, Volker Braun <vbrau...@gmail.com> wrote:

> If its really because you are running out of memory then the OOM killer
> will send a SIGTERM, do you see that in strace?
>
> The doc builder respects the usual SAGE_NUM_THREADS environment variable.

Really? With

$ SAGE_NUM_THREADS=1 make

I still get *two*

python $SAGEROOT/src/doc/common/builder.py --no-pdf-links all html

processes, one master, and one slave, getting stuck the same way as
before.

[leif]

Volker Braun wrote:
> The doc builder respects the usual SAGE_NUM_THREADS environment variable.

Well, nevertheless, AFAIK it uses Python's multiprocessing even with
SAGE_NUM_THREADS=1, just like 'sage -b' does.

I've been considering the latter a bug for years, but others apparently
didn't care.

[Volker Braun]

On Thursday, May 29, 2014 9:45:52 PM UTC+1, leif wrote:

Well, nevertheless, AFAIK it uses Python's multiprocessing even with
SAGE_NUM_THREADS=1, just like 'sage -b' does.

Whats wrong with that? You want an extra untested codepath to handle that special value?

[leif]

Volker Braun wrote:
> On Thursday, May 29, 2014 9:45:52 PM UTC+1, leif wrote:
>
> Well, nevertheless, AFAIK it uses Python's multiprocessing even with
> SAGE_NUM_THREADS=1, just like 'sage -b' does.
>
>
> Whats wrong with that?

As Dima pointed out, it doesn't only (needlessly) require specific
resources (such as e.g. /dev/shm on Linux), it also uses more resources
(because of forking). I haven't counted how many times people reported
to sage-{devel,support} not being able to build Sage (i.e., the Sage
library) or the docs because of issues with Python's multiprocessing.
When trying to build the Sage library, the error message isn't very
helpful either.

> You want an extra untested codepath to handle
> that special value?

Why should it be untested? And it isn't special in any way either; it's
rather the default.

[leif]

FWIW, little "update":

This is what I got after

make doc-clean && ./sage --docbuild --no-pdf-links reference html

...

writing output... [ 93%] sage/combinat/tutorial

writing output... [ 93%]
sage/combinat/vector_partition
writing output...
[ 94%] sage/combinat/words/abstract_word
writing
output... [ 94%] sage/combinat/words/alphabet

writing output... [ 94%] sage/combinat/words/finite_word

writing output... [ 95%] sage/combinat/words/infinite_word

writing output... [ 95%] sage/combinat/words/morphism

writing output... [ 95%]
sage/combinat/words/paths
writing output...
[ 96%] sage/combinat/words/shuffle_product
writing
output... [ 96%] sage/combinat/words/suffix_trees

writing output... [ 97%] sage/combinat/words/word

writing output... [ 97%] sage/combinat/words/word_generators

writing output... [ 97%] sage/combinat/words/words

writing output... [ 98%]
sage/combinat/yang_baxter_graph
writing output...
[ 98%] sage/databases/oeis
writing
output... [ 98%] species

writing output... [ 99%] symmetric_functions

writing output... [ 99%] tableaux

writing output... [100%] words
[combinat ] None:38: WARNING: citation not found: Schiffmann
Error building the documentation.

Traceback (most recent call last):

File "/foo/sage-6.2.beta7/src/doc/common/builder.py", line 1477, in
<module>
getattr(get_builder(name), type)()
File "/foo/sage-6.2.beta7/src/doc/common/builder.py", line 487, in
_wrapper
x.get(99999)
File "/foo/sage-6.2.beta7/local/lib/python/multiprocessing/pool.py",
line 554, in get
raise self._value
OSError: [combinat ] None:38: WARNING: citation not found: Schiffmann


Probably someone can make sense of that.

[John Cremona]

It looks as if the citation Schiffmann is defined in
algebras/hall_algebra.py:210 and referred to in that file but also in
combinat/hall_polynomial.py so perhaps there should be a second
definition in the second file.

John

>
>
>
> -leif
>
> --
> () The ASCII Ribbon Campaign
> /\ Help Cure HTML E-Mail
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[leif]

Not sure. Isn't the purpose of building the reference manual in two
passes (which I don't see there by the way) to resolve cross-references
(and also to not have to define the same [external] reference more than
once)?

I'm getting the same with a clean install of Sage 6.3.beta2 (built from
scratch, on another system), so to me it seems 'sage --docbuild <other
than all at least> ...' is broken (opposed to 'make doc' and what it
calls, namely 'sage --docbuild *all* html').


-leif

P.S.: Also the output (...%) doesn't get flushed; I'm getting

[combinat ] loading pickled environment... not yet created

[combinat ] building [html]: targets for 269 source files that are out

of date
[combinat ] updating environment: 269 added, 0 changed, 0 removed

then a burst of "reading sources... [0%] ... reading sources... [100%]",
then

loading cross citations...
[combinat ] pickling environment... done
[combinat ] checking consistency... done
[combinat ] preparing documents... done

followed by a burst of "writing output... [0%] ... writing output...
[100%]", followed by the error message:

[combinat ] None:38: WARNING: citation not found: Schiffmann
Error building the documentation.
Traceback (most recent call last):

File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 1477, in
<module>
getattr(get_builder(name), type)()
File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 487, in
_wrapper
x.get(99999)
File "/foo/sage-6.3.beta2/local/lib/python/multiprocessing/pool.py",

line 554, in get
raise self._value
OSError: [combinat ] None:38: WARNING: citation not found: Schiffmann

The 'None' there btw. is exceptionally useful.

[leif]

When running 'make doc' (without 'make doc-clean') afterwards, a lot
succeeds (and I see a couple of "building [inventory]" and "dumping
object inventory" which I didn't see with the plain './sage --docbuild
reference html'), but it ends with

[tensor ] build succeeded.
[reference] loading pickled environment... not yet created
[reference] Compiling the master document
[reference] WARNING: intersphinx inventory
'/foo/sage-6.3.beta2/src/doc/output/inventory/en/reference/objects.inv'
not fetchable due to <type 'exceptions.IOError'>: [Errno 2] No such file
or directory:
'/foo/sage-6.3.beta2/src/doc/output/inventory/en/reference/objects.inv'
Error building the documentation.

Note: incremental documentation builds sometimes cause spurious
error messages. To be certain that these are real errors, run
"make doc-clean" first and try again.

Traceback (most recent call last):
File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 1477, in
<module>
getattr(get_builder(name), type)()

File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 276, in

_wrapper
getattr(get_builder(document), 'inventory')(*args, **kwds)

File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 500, in
_wrapper
getattr(DocBuilder(self.name, lang), format)(*args, **kwds)
File "/foo/sage-6.3.beta2/src/doc/common/builder.py", line 94, in f
execfile(sys.argv[0])
File "/foo/sage-6.3.beta2/src/doc/common/custom-sphinx-build.py",
line 210, in <module>
raise OSError(ERROR_MESSAGE)
OSError: [reference] WARNING: intersphinx inventory
'/foo/sage-6.3.beta2/src/doc/output/inventory/en/reference/objects.inv'
not fetchable due to <type 'exceptions.IOError'>: [Errno 2] No such file
or directory:
'/foo/sage-6.3.beta2/src/doc/output/inventory/en/reference/objects.inv'

make: *** [doc-html] Error 1


-leif

[leif]

FWIW,

find src/doc/ -type d -empty -exec rmdir {} \;
make doc

solved that error, but I'm of course not sure whether removing empty
folders was necessary, or just the second attempt then worked.

[Dima Pasechnik]

On 2014-06-01, leif <not.r...@online.de> wrote:
> leif wrote:

[...]

>
> FWIW,
>
> find src/doc/ -type d -empty -exec rmdir {} \;
> make doc
>
> solved that error, but I'm of course not sure whether removing empty
> folders was necessary, or just the second attempt then worked.

It does not help my totally f-d installation :[

[leif]

It doesn't in my f'd up installation (6.2.beta7 upgraded to 6.3.beta2)
either.

(It just worked in the /clean/ 6.3.beta2 where I reproduced the error
with './sage --docbuild reference html' I first got in the broken one.)


-leif

[kcrisman]

FWIW, I also run into this occasionally and have no idea how to fix it.  It's particular annoying when trying to review tickets.  http://trac.sagemath.org/ticket/14669 doesn't seem to have made it worse or better, has it?

[Travis Scrimshaw]

Apparently this is related to Gap running out of memory and will (should) be fixed with the merging of #16440. See also https://groups.google.com/forum/#!topic/sage-devel/4r0QuzzDlg0.

Best,

Travis

[kcrisman]

On Tuesday, June 17, 2014 10:41:39 PM UTC-4, Travis Scrimshaw wrote:

Apparently this is related to Gap running out of memory and will (should) be fixed with the merging of #16440. See also https://groups.google.com/forum/#!topic/sage-devel/4r0QuzzDlg0.

Best,

Possibly... but I haven't seen that particular message, nor have I installed any gap packages :) 

[Dima Pasechnik]

you don't need gap *packages* to enjoy this fix.
(which at least fixes docbuilding hangups for me)

Just base your ticket reviewing etc off this branch.
(note that 'git trac try' does not currently support this workflow,
cf. https://github.com/sagemath/git-trac-command/issues/15)

>

[Travis Scrimshaw]

  I just build 6.3.beta4 and when building the doc, I noticed large memory usage spikes when compiling the "master document"s. I ended up eating up all of my RAM (4 GB) plus swap space (2 GB) when running it in parallel (4 cores/threads) and it was trying to build 3-4 different master documents. So it may be prudent when building the entire doc from scratch on 1-2 threads.

Best,

Travis