Factoring for Fun and Profit

[ph h]

Dear All,

This hitch hiker has been enjoying scratching her itch (caused by a virus from the Forth camp she visited many moons ago, smiley)

She has been wondering, it cannot be this simple. She must have missed something very important. Please help her to find out what she is missing.

Her itch was to reduce the number of occurrence of 'resolvelinks()' from 3 (1 in $SAGE_ROOT/sage, 1 in $SAGE_SRC_BIN/sage and 1 in $SAGE_SRC_BIN/sage-env) to 1.

• $SAGE_ROOT/sage was renamed to $SAGE_ROOT/sage.sage and the function 'resolvelinks()' was removed.
  
• $SAGE_SRC_BIN/sage  was renamed to $SAGE_SRC_BIN/sage.src.bin.sage and the function 'resolvelinks()' was removed.
  
• $SAGE_SRC_BIN/sage-env was factored into $SAGE_SRC_BIN/sage.src.bin.sage-env and $SAGE_SRC_BIN/resolvelinks
  

You can name these files whatever you like, just modify the scripts below (just one place to change for each name)

Then, 3 glue files were created, it is critical that these files have these names (all or just one of them?), otherwise the configuration scripts will break:

1. $SAGE_ROOT/sage
   
2. $SAGE_SRC_BIN/sage
   
3. $SAGE_SRC_BIN/sage-env
   

When copy&paste is needed(?) should only the glue files be so? They will find out where are the files to be sourced based on these sign posts.

The code as it is now will break when there are fake sign posts in front of true $SAGE_ROOT.  It is just a proof of concept, right now.  (Heavy duty sed, awk scripts should be able to fix this?)

If you build your own 'Sage' and like to have some fun with this factoring, please try it out  with your favorite use case scenarios to see what is broken (Sage team might be able to help in fixing it?).
If the Sage team thinks Sage might benefit from this idea, please feel free to do whatever they like to with it.

Please find appended below the glue codes and your help is much appreciated.

Regards,

phiho



<$SAGE_ROOT/sage>

SELF=$(readlink -f $0)
SAGE_ROOT=`dirname $SELF`

RESOLVELINKS=$SAGE_ROOT/src/bin/resolvelinks
SAGE_SAGE=$SAGE_ROOT/sage.sage

if [ ! -f $RESOLVELINKS -o ! -f $SAGE_SAGE ]; then
    echo "$RESOLVELINKS and/or $SAGE_SAGE not found" >&2
    exit 1
fi

. $RESOLVELINKS
. $SAGE_SAGE

</$SAGE_ROOT/sage>


<$SAGE_SRC_BIN/sage>

#
# Please note the following line need be replaced with a copy&paste
#
# . $SAGE_SRC_BIN/sage.src.bin.sage-sage-env.common
#

SAGE_SRC_BIN_SAGE=$SAGE_SRC_BIN/sage.src.bin.sage

if [ ! -f $RESOLVELINKS ] || [ ! -f $SAGE_SRC_BIN_SAGE ]; then
    echo "$RESOLVELINKS and/or $SAGE_SRC_BIN_SAGE not found" >&2
    exit 1
fi

. $RESOLVELINKS
. $SAGE_SRC_BIN_SAGE

</$SAGE_SRC_BIN/sage>


<$SAGE_SRC_BIN/sage-env>

#
# Please note the following line need be replaced with a copy&paste
#
#  . $SAGE_SRC_BIN/sage.src.bin.sage-sage-env.common
#

SAGE_ENV=$SAGE_SRC_BIN/sage.src.bin.sage-env

if [ ! -f $RESOLVELINKS ] || [ ! -f $SAGE_ENV ]; then
    echo "$RESOLVELINKS and/or $SAGE_ENV not found"  >&2
    return
fi

. $RESOLVELINKS
. $SAGE_ENV

</$SAGE_SRC_BIN/sage-env>


<$SAGE_SRC_BIN/sage.src.bin.sage-sage-env.common>

SELF=$(readlink -f $0)

#
#  Maybe the Sage team can help to verify if the list of sing posts is exhaustive.

#

if [ ! -d "$SAGE_ROOT" ]; then
    for signpost in  \
        "/src/bin/" \
        "/build/bin/" \
        "/local/var" \
        "/build/pkgs/sagelib/src/bin/" \
        "/build/pkgs/sagemath_categories/src/bin/" \
        "/build/pkgs/sagemath_objects/src/bin/" \
        "/pkgs/sagemath-categories/bin/" \
        "/pkgs/sagemath-objects/bin/" \
        "/pkgs/sagemath-standard/bin/" \
        "/pkgs/sage-conf_pypi/sage_root/" \
        "/pkgs/sagemath-standard/bin/" \
        "/pkgs/sage-conf_pypi/sage_root/"
    do
        SAGE_ROOT=$(echo $SELF | sed "s:$pat.*$::")

        if [ -f $SAGE_ROOT/src/bin/resolvelinks ]; then
            break
        fi
    done
fi

SAGE_SRC_BIN=$SAGE_ROOT/src/bin
RESOLVELINKS=$SAGE_SRC_BIN/resolvelinks

#################################################################################################################
#   The above snippet is common to sage and sage-env  copied and pasted from sage.src.bin.sage-sage-env.common  #
#################################################################################################################

</$SAGE_SRC_BIN/sage.src.bin.sage-sage-env.common>

[Michael Orlitzky]

On Tue, 2022-05-03 at 14:30 -0700, ph h wrote:
>
> *SELF=$(readlink -f $0)

Sadly, readlink is not a POSIX standard utility; otherwise we would
already be using it in place of resolvelinks().

[ph h]

Hi,

> Sadly, readlink is not a POSIX standard utility; otherwise we would
> already be using it in place of resolvelinks().

Thank you so much. That's the constraint that Sage must face.

BTW, which of the platforms that Sage supports do not have readlink?

Regards,

phiho

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/CuYOEY5pfdI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/5e9947445de86811f34c05e204f8e45c100e421d.camel%40orlitzky.com.

[ph h]

P.S:  What does this mean? Thanks.

https://man7.org/linux/man-pages/man2/readlink.2.html

CONFORMING TO         top

       readlink(): 4.4BSD (readlink() first appeared in 4.2BSD),
       POSIX.1-2001, POSIX.1-2008.

       readlinkat(): POSIX.1-2008.

[Michael Orlitzky]

On Tue, 2022-05-03 at 21:33 -0400, ph h wrote:
> P.S: What does this mean? Thanks.
>
> https://man7.org/linux/man-pages/man2/readlink.2.html
> CONFORMING TO top

> <https://man7.org/linux/man-pages/man2/readlink.2.html#top_of_page>
>
> *readlink*(): 4.4BSD (*readlink*() first appeared in 4.2BSD),
> POSIX.1-2001, POSIX.1-2008.
>
> *readlinkat*(): POSIX.1-2008.
>
>

Those are C functions:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/readlink.html

[ph h]

> Those are C functions:
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/readlink.html

Is it installable at configuration time?

Which of the platforms that Sage supports do not have readlink?

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/d623b4c02c132d66b9f6cbe4ad0c256f58fa26f1.camel%40orlitzky.com.

[Michael Orlitzky]

On Tue, 2022-05-03 at 21:07 -0400, ph h wrote:
> Hi,
>
> > Sadly, readlink is not a POSIX standard utility; otherwise we would
> > already be using it in place of resolvelinks().
>
> Thank you so much. That's the constraint that Sage must face.
>
> BTW, which of the platforms that Sage supports do not have readlink?
>

I think the macOS/BSD implementation does something different or is
missing the "-f" flag or something like that.

We would have the same problem with any shell function though. Python
packaging just isn't equipped to handle it, and probably never will be.

[Michael Orlitzky]

On Tue, 2022-05-03 at 22:21 -0400, ph h wrote:
> > Those are C functions:
> > https://pubs.opengroup.org/onlinepubs/9699919799/functions/readlink.html
>
> Is it installable at configuration time?

It just means that those functions are "guaranteed" to be present in
the C standard library on most unix-like systems. It's not something
that can be used directly by a shell script.

[ph h]

Hi,

> Sadly, readlink is not a POSIX standard utility; otherwise we would
> already be using it in place of resolvelinks().

In that case, please try this:

#SELF=$(readlink -f $0)

SELF=$(cd `dirname $0` ; pwd)/`basename $0`

So far 'sage --coverageall', 'sage --sdist', './configure' and make is churning along happily:

~/sage-9.6/git/rc3/sage.mod8$ ls -l logs/pkgs/
total 524
-rw-rw-rw- 1 hph hph  13791 May  4 01:43 ccache-3.3.4.log
-rw-rw-rw- 1 hph hph 107491 May  4 01:51 cython-0.29.28.log
-rw-rw-rw- 1 hph hph      0 May  4 01:43 flint-2.8.4.log
-rw-rw-rw- 1 hph hph   4728 May  4 01:45 flit_core-3.4.0.log
-rw-rw-rw- 1 hph hph   9167 May  4 01:47 gmpy2-2.1.1.log
-rw-rw-rw- 1 hph hph  13366 May  4 01:46 packaging-21.3.log
-rw-rw-rw- 1 hph hph   4473 May  4 01:52 pari-2.13.3.log
-rw-rw-rw- 1 hph hph   3352 May  4 01:51 pari_galdata-20080411.p0.log
-rw-rw-rw- 1 hph hph   3430 May  4 01:51 pari_seadata_small-20090618.p0.log
-rw-rw-rw- 1 hph hph 128307 May  4 01:45 pip-22.0.2.log
-rw-rw-rw- 1 hph hph   9690 May  4 01:46 pyparsing-3.0.6.log
-rw-rw-rw- 1 hph hph 107180 May  4 01:44 setuptools-59.8.0.log
-rw-rw-rw- 1 hph hph  11881 May  4 01:47 setuptools_scm-6.3.2.log
-rw-rw-rw- 1 hph hph  40502 May  4 01:46 setuptools_wheel-59.8.0.log
-rw-rw-rw- 1 hph hph   4625 May  4 01:45 tomli-1.2.1.log
-rw-rw-rw- 1 hph hph  19554 May  4 01:44 wheel-0.37.0.log
~/sage-9.6/git/rc3/sage.mod8$ ls -l logs/pkgs/ | wc -l
17
~/sage-9.6/git/rc3/sage.mod8$

She will be out of brane in the next few weeks, please allow her to catch up upon her return.

Thank you for your help.

Regards,

phiho

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/CuYOEY5pfdI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/5e9947445de86811f34c05e204f8e45c100e421d.camel%40orlitzky.com.

[Michael Orlitzky]

On Wed, 2022-05-04 at 01:56 -0400, ph h wrote:
> Hi,
>
> > Sadly, readlink is not a POSIX standard utility; otherwise we would
> > already be using it in place of resolvelinks().
>
> In that case, please try this:
>
> #SELF=$(readlink -f $0)
> SELF=$(cd `dirname $0` ; pwd)/`basename $0`
>

That will fail if $0 is a symlink, because `dirname $0` will get the
directory that contains the symlink itself and not its target.

Our installation instructions suggest creating such a symlink:

https://doc.sagemath.org/html/en/installation/source.html

I don't have much nice to say about that installation procedure, but
running sage from a symlink is something that should be supported
regardless of how it was installed.

[ph h]

  Hi,

>  running sage from a symlink is something that should be supported

Absolutely, yes.

In a hurry she forgot to mention that :

SELF=$(cd `dirname $0` ; pwd)/`basename $0`

is for the links from within $SAGE_ROOT only, of course any link from outside would need to be resolved with 'resolvelinks'.

Unless this 'readlink.py' works on all platforms that 'sage' supports:

<readlink.py>

#!/usr/bin/python
import sys, os

thispath=sys.argv[1]

if os.path.islink( thispath  ):
     thispath  =os.readlink( thispath  )
else:
     thispath  =os.path.abspath( thispath  )

print( thispath  )

</readlink.py>

and in top level 'sage', it is embedded:

readlink() { cat << READLINK

import sys, os

thispath="$0"

if os.path.islink(thispath):

    thispath=os.readlink(thispath)

else:

    thispath=os.path.abspath(thispath)

print(thispath)

READLINK

}

SELF=`readlink | python3`

and exported:

export READLINKPY=$SAGE_ROOT/readlink.py

in 'sage.src.bin.sage':

SELF=`$READLINKPY "${0}"`

In 'sage.src.bin.sage-env':

DOT_SAGE=`$READLINKPY "$HOME/.sage"

nodeenv_activate=`$READLINKPY "$nodeenv_activate"`

Is 'resolvelinks' better than Python 'os.readlink()'? Are there any issues with  the latter?

Would it be nice if 'Sage' has more Python code?

Regards,

phiho

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c4295d4a6af9e3b69c72b4f4ba425325d52e5634.camel%40orlitzky.com.

[Michael Orlitzky]

On Tue, 2022-05-10 at 00:41 -0400, ph h wrote:
>
> and exported:
>
> export READLINKPY=$SAGE_ROOT/readlink.py
>
>

How do you know $SAGE_ROOT? That's what we're trying to determine in
the first place.

This is a code smell, and an easy problem to fix, but to be blunt:
you're not going to fix it. We need an adult in charge of the sage
build system first. Other packages don't have this problem because they
can substitute the correct path into the scripts at build/install time.

[John H Palmieri]

Regarding the very last question: the decision many years ago was that the startup scripts for Sage should be shell scripts, because using Python scripts seemed to add to the startup time. If anyone wants to change this, they need to do some careful evaluations and comparisons regarding startup times.

[Matthias Koeppe]

https://trac.sagemath.org/ticket/33786 makes some related changes. Everyone is welcome to participate in the review of the ticket.

[Michael Orlitzky]

On Tue, 2022-05-10 at 09:54 -0700, John H Palmieri wrote:
> Regarding the very last question: the decision many years ago was that the
> startup scripts for Sage should be shell scripts, because using Python
> scripts seemed to add to the startup time. If anyone wants to change this,
> they need to do some careful evaluations and comparisons regarding startup
> times.
> > > > >

Keep in mind also the recent thread about "make build". Many people
still believe that "sage -i" should be able to install python for them.

[Matthias Koeppe]

On Tuesday, May 10, 2022 at 1:46:55 PM UTC-7 Michael Orlitzky wrote:

Keep in mind also the recent thread about "make build". Many people
still believe that "sage -i" should be able to install python for them.

This is not actually related because Sage needs a version of python as a build prerequisite.

This is "sage-bootstrap-python".

[ph h]

Dear All,

A plain vanilla Sage 9.7 Beta1 was built from "~/sage-9.7$/beta1" which then is renamed to "~/sage-9.7$/beta1_"

"root.beta1" is a symlink to "$SAGE_ROOT/sage" and "local.beta1" is a symlink to "$SAGE_LOCAL/bin/sage".

Please find appended below the scenario when some of the 'sage' options were tried.

Is the outcome expected or is it just something that happens only on this machine?

Please note that "--coverageall" and "--testall"  are available, according to "sage --advanced" which does not show "--sdist" (but this user is familiar with it and has been using it)

Thank you for your help.

   

Regards,

phiho

~/sage-9.7$ mv beta1 beta1_
~/sage-9.7$ root.beta1
-bash: /home/hph/bin/root.beta1: No such file or directory

~/sage-9.7$ local.beta1 --coverageall
Traceback (most recent call last):
  File "/mnt/g/Maths/sage-9.7/beta1/bin/sage-coverage", line 60, in <module>
    coverage_all(os.path.join(os.environ["SAGE_SRC"], 'sage'))
  File "/usr/lib/python3.10/os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: 'SAGE_SRC'

~/sage-9.7$ local.beta1 --sdist
/home/hph/bin/local.beta1: line 994: exec: sage-sdist: not found

~/sage-9.7$ local.beta1 --testall
too many failed tests, not using stored timings
Running doctests with ID 2022-05-31-12-38-54-59a8fb70.
Using --optional=pip,sage
Features to be detected: 4ti2,benzene,bliss,buckygen,conway_polynomials,csdp,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_jones_numfield,database_knotinfo,dvipng,gfan,graphviz,imagemagick,jupymake,kenzo,latte_int,lrslib,mcqd,meataxe,nauty,palp,pandoc,pdf2svg,pdftocairo,phitigra,plantri,polytopes_db,polytopes_db_4d,pynormaliz,python_igraph,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,sphinx,tdlib
Doctesting entire Sage library.
Traceback (most recent call last):
  File "/mnt/g/Maths/sage-9.7/beta1/bin/sage-runtests", line 151, in <module>
    err = DC.run()
  File "/mnt/g/Maths/sage-9.7/beta1/lib/python3.10/site-packages/sage/doctest/control.py", line 1343, in run
    self.add_files()
  File "/mnt/g/Maths/sage-9.7/beta1/lib/python3.10/site-packages/sage/doctest/control.py", line 840, in add_files
    all_files()
  File "/mnt/g/Maths/sage-9.7/beta1/lib/python3.10/site-packages/sage/doctest/control.py", line 823, in all_files
    import sage_docbuild
  File "/mnt/g/Maths/sage-9.7/beta1/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 59, in <module>
    from .build_options import (LANGUAGES, SPHINXOPTS, OMIT,
  File "/mnt/g/Maths/sage-9.7/beta1/lib/python3.10/site-packages/sage_docbuild/build_options.py", line 9, in <module>
    LANGUAGES = [d for d in os.listdir(SAGE_DOC_SRC) if re.match('^[a-z][a-z]$', d)]
FileNotFoundError: [Errno 2] No such file or directory: 'src/doc'
~/sage-9.7$

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/298a04c7a9808fd8f61a0588ce1ae014e4a53db7.camel%40orlitzky.com.

[Matthias Koeppe]

https://github.com/sagemath/sage#relocation

"It is not supported to move the SAGE_ROOT or SAGE_LOCAL directory after building Sage. If you do move the directories, you will have to run make distclean and build Sage again from scratch."

[ph h]

Hi,

> "It is not supported to move the SAGE_ROOT or SAGE_LOCAL directory after building Sage. If you do move the directories, you will have to run make distclean and build Sage again from scratch."

So, practically speaking, BOTH "$SAGE_ROOT" and "$SAGE_LOCAL" are part of the 'sage' installation?

However, when both are available:

~/sage-9.7$ root.beta1 --advanced | wc -l
242

~/sage-9.7$ local.beta1 --advanced | wc -l

242

and when "$SAGE_ROOT" is renamed to "$SAGE_ROOT"_:

~/sage-9.7$ mv beta1 beta1_
~/sage-9.7$ root.beta1 --advanced

-bash: /home/hph/bin/root.beta1: No such file or directory

~/sage-9.7$ local1 --advanced | wc -l
154
~/sage-9.7$

What are these missing 88(?) lines in "sage --advanced"?

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/07b2aa47-7707-43a1-ae4e-62b26bf23049n%40googlegroups.com.

[Matthias Koeppe]

On Tuesday, May 31, 2022 at 10:45:47 AM UTC-7 hohoa...@gmail.com wrote:

> "It is not supported to move the SAGE_ROOT or SAGE_LOCAL directory after building Sage. If you do move the directories, you will have to run make distclean and build Sage again from scratch."

So, practically speaking, BOTH "$SAGE_ROOT" and "$SAGE_LOCAL" are part of the 'sage' installation?

SAGE_LOCAL is the self-contained Sage installation.

 

You need SAGE_ROOT only if you want to refer back to the source or rebuild Sage.

However, when both are available:

~/sage-9.7$ local.beta1 --advanced | wc -l

242

and when "$SAGE_ROOT" is renamed to "$SAGE_ROOT"_:

~/sage-9.7$ local1 --advanced | wc -l

154

What are these missing 88(?) lines in "sage --advanced"?

When SAGE_ROOT is not available, "sage --advanced" will not show the commands for rebuilding Sage, which need SAGE_ROOT.

 

[ph h]

Hi,

> When SAGE_ROOT is not available, "sage --advanced" will not show the commands for rebuilding Sage, which need SAGE_ROOT.

and any other options like "sage --coverageall", :sage --testall", ... and what else?

Not only that  "sage --advanced" will not show the options but also 'sage' should NOT attempt to do them (when the users insist) with error messages.

'Sage' should instead give some wise advice and gracefully exits(?):

" For this option to work, you need '$SAGE_ROOT' which is not available on your system right now. 

(In case you are testing me, you know that I know what you are doing, please 'mv beta1_ beta1' (smiley))"

Cheers,

phiho

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/0be69090-6dc3-487c-be56-ad68fb0fa87an%40googlegroups.com.