Mysterious .sage behavior

[Marc Culler]

This is a follow-up to a user's query in a Sage_macOS issue. 

The current sage-env script contains the excerpt below.  It seems pretty confusing that  Sage would create a directory named .sage/ipython-5.0.0 when Sage is shipping IPython 8.18.1.  What would be wrong with calling the directory .sage/profiles/ipython--v1, and moving to ipython--v2 when necessary?  Similarly for jupyter and matplotlib.

- Marc

if [ -z "$IPYTHONDIR" ]; then
    # We hardcode a version number in the directory name. The idea is
    # that we keep using the same version number as long as that is
    # possible. Only when some future IPython version really requires
    # a new structure for the $IPYTHONDIR should this version number be
    # changed to the new IPython version.
    export IPYTHONDIR="$DOT_SAGE/ipython-5.0.0"
fi

if [ -z "$JUPYTER_CONFIG_DIR" ]; then
    # We hardcode a version number in the directory name. The idea is
    # that we keep using the same version number as long as that is
    # possible. Only when some future Jupyter version really requires
    # a new structure for the $JUPYTER_CONFIG_DIR should this version
    # number be changed to the new jupyter_core version.
    export JUPYTER_CONFIG_DIR="$DOT_SAGE/jupyter-4.1"
fi

if [ -z "$MPLCONFIGDIR" ]; then
    # We hardcode a version number in the directory name. The idea is
    # that we keep using the same version number as long as that is
    # possible. Only when some future Matplotlib version really requires
    # a new structure for the $MPLCONFIGDIR should this version
    # number be changed to the new matplotlib version.
    export MPLCONFIGDIR="$DOT_SAGE/matplotlib-1.5.1"
fi

[Dima Pasechnik]

On 31 March 2024 15:23:24 CEST, Marc Culler <marc....@gmail.com> wrote:
>This is a follow-up to a user's query in a Sage_macOS issue.
>
>The current sage-env script contains the excerpt below. It seems pretty
>confusing that Sage would create a directory named .sage/ipython-5.0.0
>when Sage is shipping IPython 8.18.1. What would be wrong with calling the
>directory .sage/profiles/ipython--v1, and moving to ipython--v2 when
>necessary? Similarly for jupyter and matplotlib.

I must say I don't know what kind of problems these versioned names are meant to solve.

Dima

[Nils Bruin]

One scenario where I could see this being beneficial:

These configurations are written into `$HOME/.sage` (normally), so that is not a location that is predicated by a venv or a localized subdir. If a user is using two different sagemath installs (e.g., one system, the other for development) that have incompatible ipython configs, they need to live in places that allow peaceful co-existence. This would do it. I agree the version number can look confusing, It seems like a historic artifact that we didn't need to change our ipython configs since version 5.0.0, probably. It looks to me like a fairly arbitrary choice how to mark that directory name. Not changing it would be the easiest solution, but if someone wants to make an argument that another naming convention has benefits (following common practices that have since evolved?) it could be changed, at the expense of everyone with an install needing to change locations of files they already customized. So based on that, I think the best time to change it (if at all) would be when an incompatible change in config is happening anyway.

On Monday 1 April 2024 at 12:20:16 UTC-7 Dima Pasechnik wrote:

I must say I don't know what kind of problems these versioned names are meant to solve.

 

[Marc Culler]

The problem that it *causes* is that people think that Sage has somehow managed to install an ancient ipython-5.0.0 in their ~/.sage directory.  The user that reported this was indeed having problems because of an out of date ipython thet they had installed with %pip.  The python package was installed as a --user package  in  ~/.sage.   That is where the macOS app installs all pip packages, being disallowed from putting them into the app bundle itself because that would break the signature and macOS would then refuse to run the app..

The old version of ipython caused the new app to crash with the rather amazing error message "object of type List is not iterable".  When the user removed their ~/.sage directory everything worked as expected.  But then, after starting Sage,  they checked to see what was actually in the directory, and were alarmed to ",find" that Sage had re-installed ipython-5.0.0.

As I said, this idea confuses users, and understandably so.  It was not a great idea. There were so many other, better, ways of naming the directory.

Maybe for the next Sage release I will move the pip packages to ~/Library/SageMath-X.Y.  That won't stop Sage from creating ~/.sage/ipython-5.0.0 but it will remove the motivation for users to look in the .sage directory and get confused.

- Marc

[John H Palmieri]

I searched for "ipython-5.0.0" in the source and found this:

if [ -z "$IPYTHONDIR" ]; then
    # We hardcode a version number in the directory name. The idea is
    # that we keep using the same version number as long as that is
    # possible. Only when some future IPython version really requires
    # a new structure for the $IPYTHONDIR should this version number be
    # changed to the new IPython version.
    export IPYTHONDIR="$DOT_SAGE/ipython-5.0.0"
fi

So it should be viewed as "IPython, version 5.0.0 or later, as long as things remain compatible". Maybe it would be better to have a directory "ipython-config" (for example) with subdirectories that somehow convey the version information. We could also, or in addition, create a README file the existing directory with some sort of explanation.

--

John