recommendation: delete local/share/mathjax/mathjax, run make doc-clean, then make

[John H Palmieri]

I found that my Sage documentation directory was taking up 15GB of hard drive space, because of a bug (which I think has been fixed, but I don't remember where). The bug was caused by a recursive symlink

    local/share/mathjax/mathjax -> local/share/mathjax

When the contents of local/share/mathjax were copied to various _static directories in the html documentation, rather than just reproducing the symlink, it created many layers of subdirectories; for example, ca/intro/_static/mathjax/mathjax/mathjax/... Basically mathjax all the way down. I don't know if this happened all at once, or once each time I built the documentation.

To see if you have this problem, see if you have the symlink. You might also find out how much disk space local/share/doc/sage takes up (for example, "du -s -h /path/to/SAGE_ROOT/local/share/doc/sage"). On my computer, it is 1.9GB. If you are using up a lot more space than that, you might have hit this bug.

To fix: remove the symlink SAGE_ROOT/local/share/mathjax/mathjax. Then run "make doc-clean" to remove the old documentation, and then run "make" to rebuild the documentation.

Question:

- has the bug been fixed, or is that symlink still being created?

--

John

[Travis Scrimshaw]

Running the du command (which took quite a bit of time to run), mine was 21GB. That might have been why the docbuild for me was hanging after it had finished tutorial... Trying now.

Best,

Travis

[Travis Scrimshaw]

Running the du command (which took quite a bit of time to run), mine was 21GB. That might have been why the docbuild for me was hanging after it had finished tutorial... Trying now.

Yep, that seems to have done the trick.

Best,

Travis

[Dima Pasechnik]

Yeah, I also got hit by this, gigabytes of extra stuff :-(

The symlink comes from sagenb/spkg-install, see

https://trac.sagemath.org/ticket/18011

Some symlinking is still happening on 8.4.beta1, see

$ git show 32f5bbeac80

commit 32f5bbeac804b37776007407eb00b8acd39559e7

Author: Erik M. Bray

Date:   Wed May 16 09:52:27 2018 +0000

    try installing the symlink anyways and don't poke around SAGE_LOCAL

[Erik Bray]

I believe that in the current version this is fixed, but if you did
several builds of sagenb before the issue was fixed, this symlink
might have been created, or if switching to an older branch and
rebuilding it might still get recreated. The best solution seems to
be to simply `rm -rf $SAGE_LOCAL/share/mathjax` then reinstall mathjax
and rebuild.

[Sébastien Labbé]

On one machine, I get this:

$ du -s -h local/share/doc/sage

20G    /local/share/doc/sage

on the other, I get this:

$ du -s -h local/share/doc/sage
1,9G    local/share/doc/sage

I do not know why the bug affects only one? Thanks for sharing.

Sébastien

[John H Palmieri]

If somehow the bad symlink didn't get created on one machine (you did 'make distclean', for example), but it did on the other, that would probably explain it.

[Erik Bray]

On Sat, Aug 25, 2018 at 3:39 PM Sébastien Labbé <sla...@gmail.com> wrote:
>
> On one machine, I get this:
>
> $ du -s -h local/share/doc/sage
> 20G /local/share/doc/sage
>
> on the other, I get this:
>
> $ du -s -h local/share/doc/sage
> 1,9G local/share/doc/sage
>
> I do not know why the bug affects only one? Thanks for sharing.

This bug is already fixed, but if you did a build on the first machine
with a version where the bug did exist (specifically in the sagenb
package) then it will persist until and unless you delete
$SAGE_LOCAL/share/mathjax and reinstall the mathjax package.
Otherwise you'll just never see it in the first place.

> On Thursday, August 23, 2018 at 10:13:11 PM UTC+2, John H Palmieri wrote:
>>
>> I found that my Sage documentation directory was taking up 15GB of hard drive space, because of a bug (which I think has been fixed, but I don't remember where). The bug was caused by a recursive symlink
>>
>> local/share/mathjax/mathjax -> local/share/mathjax
>>
>> When the contents of local/share/mathjax were copied to various _static directories in the html documentation, rather than just reproducing the symlink, it created many layers of subdirectories; for example, ca/intro/_static/mathjax/mathjax/mathjax/... Basically mathjax all the way down. I don't know if this happened all at once, or once each time I built the documentation.
>>
>> To see if you have this problem, see if you have the symlink. You might also find out how much disk space local/share/doc/sage takes up (for example, "du -s -h /path/to/SAGE_ROOT/local/share/doc/sage"). On my computer, it is 1.9GB. If you are using up a lot more space than that, you might have hit this bug.
>>
>> To fix: remove the symlink SAGE_ROOT/local/share/mathjax/mathjax. Then run "make doc-clean" to remove the old documentation, and then run "make" to rebuild the documentation.
>>
>> Question:
>>
>> - has the bug been fixed, or is that symlink still being created?
>>
>> --
>> John
>>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[John H Palmieri]

I thought that this bug was fixed, but it has reappeared: the directory 'SAGE_ROOT/local/share/doc/sage' may be taking up many gigabytes of space. Please check to see if you're affected. See also #26612 for a possible fix.

  John

[Erik Bray]

On Wed, Oct 31, 2018 at 8:07 PM John H Palmieri <jhpalm...@gmail.com> wrote:
>
> I thought that this bug was fixed, but it has reappeared: the directory 'SAGE_ROOT/local/share/doc/sage' may be taking up many gigabytes of space. Please check to see if you're affected. See also #26612 for a possible fix.

It might have re-occurred for you (or anyone) if you changed to an old
branch and re-built.

[John H Palmieri]

On Thursday, November 8, 2018 at 7:37:12 AM UTC-8, Erik Bray wrote:

On Wed, Oct 31, 2018 at 8:07 PM John H Palmieri <jhpalm...@gmail.com> wrote:
>
> I thought that this bug was fixed, but it has reappeared: the directory 'SAGE_ROOT/local/share/doc/sage' may be taking up many gigabytes of space. Please check to see if you're affected. See also #26612 for a possible fix.

It might have re-occurred for you (or anyone) if you changed to an old
branch and re-built.

I don't know if I did that; maybe it depends on what "old" means. It happened to me with three different computers, at least one of which I don't use that much for Sage development, so either I got unlucky or it is easy for this error to reappear.

To what extent should Sage try to catch the error and repair it if it occurs, or should it be up to individual users to notice and fix it on their own? I would suggest that if a bug in Sage is responsible for using up an extra 15 GB of disk space, then Sage should try to detect whether that disk space is being used (for example, by checking for the presence of the symlink) and either deleting all of the bad stuff, or at least warning the user that it's there. People who do not read sage-devel may not know about this, and if they have been doing incremental upgrades for a while, they may have this problem. Or they thought that it was fixed before and don't realize it's a problem again.

[Dima Pasechnik]

It might have been a bug in sagenb 1.1.0, fixed by Jeroen in
https://trac.sagemath.org/ticket/26641
Basically, sagenb 1.1.0 came in a tarball with mathjax, was my fault in part
(and in part a deficiency in the procedure to create sagenb releases,
hopefully fixed by now).

[John H Palmieri]

On Thursday, November 8, 2018 at 8:32:32 AM UTC-8, Dima Pasechnik wrote:

It might have been a bug in sagenb 1.1.0, fixed by Jeroen in
https://trac.sagemath.org/ticket/26641
Basically, sagenb 1.1.0 came in a tarball with mathjax, was my fault in part
(and in part a deficiency in the procedure to create sagenb releases,
hopefully fixed by now).

I agree, but fixed in the sense that if you didn't have the bad symlink, then it shouldn't be created. Not fixed in the sense that if you already have the bad symlink and the docs that take up 15 extra GB, then that symlink will remain, and the disk usage will still be bad, until you manually remove the symlink and do 'make doc-clean'. Should Sage do that clean-up for everyone? (It can be more targeted, just removing the bad symlink and the bad files/directories in the documentation.)