[Yet again] Sage's R vs system's R

[Emmanuel Charpentier]

While trying my hand at porting R 3.3.1 to Sage (needs_review, by the way), I found this in the current R Installation and Administration manual :

> C.8 Cygwin
>
> The 32-bit version has never worked well enough to pass R’s make check, and residual support from
> earlier experiments was removed in R 3.3.0.
>
> The 64-bit version is completely unsupported.

Maybe we should consider to have an interface to system's R rather than our own version (and therefore make it an optional package)

It would be more difficult (but not impossible) to offer the same kind of access to an external R as to a tightly-knit R. I'm thinking graphics, notebook(s) integration, etc...

OTOH, the existence of a good R interface (Rpy2) could make things easier. The largest problem that one can forecast is, of course, backwards compatibility in existing programs.


On the third hand, what can be said if the future of a native Windows port of Sage, which could do without Cygwin monkeying ? I understand that a) it's not really easy and b) some significant advances have been made recently...

Ideas, suggestions, etc ?

[leif]

Emmanuel Charpentier wrote:
> While trying my hand <https://trac.sagemath.org/ticket/20523> at porting

> R 3.3.1 to Sage (needs_review, by the way), I found this in the
> current R Installation and Administration manual

> <https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Cygwin> :

>
>> C.8 Cygwin
>>
>> The 32-bit version has never worked well enough to pass R’s make
> check, and residual support from
>> earlier experiments was removed in R 3.3.0.
>>
>> The 64-bit version is completely unsupported.
>
> Maybe we should consider to have an interface to system's R rather than
> our own version (and therefore make it an optional package)

+1 for making it an optional package. (It's by the way not what I'd
call a small package, and also takes some time to build.)


The Cygwin developers certainly have to say something regarding support
as a standard package. (I don't think it would make sense to upgrade
Sage's R version *and* keep it a standard package if it does no longer
build on Cygwin. We could presumably still keep Rpy and let it use the
system version of R, also on Windoze, if present and suited.)


-leif

[Jean-Pierre Flori]

On Saturday, August 20, 2016 at 7:05:27 PM UTC+2, leif wrote:

Emmanuel Charpentier wrote:
> While trying my hand <https://trac.sagemath.org/ticket/20523> at porting
> R 3.3.1 to Sage (needs_review, by the way), I found this in the
> current R Installation and Administration manual
> <https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Cygwin> :
>
>> C.8 Cygwin
>>
>> The 32-bit version has never worked well enough to pass R’s make
> check, and residual support from
>> earlier experiments was removed in R 3.3.0.

I wonder what the residual support was...

>>
>> The 64-bit version is completely unsupported.

I concur, but that was for bad reasons, or let's be fair, one could say the lack of manpower maybe?

>
> Maybe we should consider to have an interface to system's R rather than
> our own version (and therefore make it an optional package)

+1 for making it an optional package.  (It's by the way not what I'd
call a small package, and also takes some time to build.)
 

The Cygwin developers certainly have to say something regarding support
as a standard package.  (I don't think it would make sense to upgrade
Sage's R version *and* keep it a standard package if it does no longer
build on Cygwin.  We could presumably still keep Rpy and let it use the

system version of R, also on Windoze, if present and suited.) g

It's not my impression that The R folk really supported any version of Cygwin recently.
I even got bashed out when proposing a simple and non-intrusive patch for Cygwin64.
The point is that it seems hopeless to push patches upstream which is a very bad point.
gwthat hard at all, one just needs a setup and a very little bit of good will:
ftp://cygwin.com/pub/cygwin/x86_64/release/R/
So yes R still builds on Cygwin32/64.

The other solution on Windows is to try to use the R provided binaries, but that surely means no real library interface.

Concerning the status of Sage on Cygwin, it's always the same deal: we need buildbots, I thought it was one of the ODK goals.
I don't have the time anymore to play with my VM's and torture Cygwin and Sage.
I could still do it next week at SD75.

-leif


> It would be more difficult (but not impossible) to offer the same kind
> of access to an external R as to a tightly-knit R. I'm thinking
> graphics, notebook(s) integration, etc...
>
> OTOH, the existence of a good R interface (Rpy2) could make things
> easier. The largest problem that one can forecast is, of course,
> backwards compatibility in existing programs.

Would this be possible on Cygwin, mixing a Cygwin compiled rpy2 and a Windows compiled R?

[Jean-Pierre Flori]

And frankly the set of patches shipped by Cygwin is really small...
One would need to make a diplomatic move toward R folk.

[Dima Pasechnik]

Every R user on Windows I saw was using rstudio. And most probably this explains lack of interest for Cygwin.

Besides, there are several R packages that use Java, and this with Cygwin seems to be an extra pain.

[Emmanuel Charpentier]

Did you try to use the cygwin tarball as a source for Sage's R version ?

BTW : could it be acceptable to have multiple tarballs as a source for R on different platforms ? Or different set of patches ?

Another alternative : can the spkg-install script use only certain patches (as a function of the platform he's running on) ? In that case, a diff between the original tarball and the Cygwin-patced tarball could be applied if and only if one is installing on cygwin.

What do you think ? I do not understand the Sage build system well enough to understand if this is possible, much less how... I need your advice...
 

One would need to make a diplomatic move toward R folk.

I doubt it : the set of Sage's R users is probably fairly small compared to the number of R users...

--
Emmanuel Charpentier

[Jean-Pierre Flori]

That would be complicated.

Another alternative : can the spkg-install script use only certain patches (as a function of the platform he's running on) ? In that case, a diff between the original tarball and the Cygwin-patced tarball could be applied if and only if one is installing on cygwin.

Very easy, I can try to do it.

But the main point is that the Cygwyn's folk R patches only modify the build system behavior when run on Cygwin.
If you apply the patches and build on Linux it would make no differences.
 

[leif]

Well, then if the Cygwin folks keep maintaining it, we could simply
"import" their patch(es), not (significantly) upgrading R before they do.

I'd still like R being an /optional/ package though... ;-)


-leif

> What do you think ? I do not understand the Sage build system well
> enough to understand if this is possible, much less how... I need
> your advice...
>
>
> One would need to make a diplomatic move toward R folk.
>
>
> I doubt it : the set of Sage's R users is probably fairly small

> compared to the number of R users....
>
> --
> Emmanuel Charpentier

[Emmanuel Charpentier]

From Trac#20523 :

Replying to [comment:9 tscrim]:
> I get a failure trying to install this on Cygwin32 with
> {{{
> checking for lzma_version_number in -llzma... no
> configure: error: "liblzma library and headers are required"
> Error configuring R.
> }}}
> which is the same failure I had on #20190.

From the R 3.3.0 release notes :
 :

The previously included versions of zlib, bzip2, xz and PCRE have been removed, so suitable external (usually system) versions are required (see the ‘R Installation and Administration’ manual).

So it's either :

• add xz-tools (né lzma-utils) to the list of prerequisites for Sage (i. e. relying on system's liblzma) ;
• adding xz-tools (at least the lzma library) to sage (cluttering it with yet one more library to maintain) ;
• or relying on system's R through an interface that remains to be written...

Pick your poison...

[leif]

Emmanuel Charpentier wrote:
> From Trac#20523 <https://trac.sagemath.org/ticket/20523> :

>
> Replying to [comment:9 tscrim]:
>> I get a failure trying to install this on Cygwin32 with
>> {{{
>> checking for lzma_version_number in -llzma... no
>> configure: error: "liblzma library and headers are required"
>> Error configuring R.
>> }}}
>> which is the same failure I had on #20190.
>
> From the R 3.3.0 release notes

> <https://cran.r-project.org/doc/manuals/r-release/NEWS.html> :
> :

>
> The previously included versions of zlib, bzip2, xz and PCRE have
> been removed, so suitable external (usually system) versions are
> required (see the ‘R Installation and Administration’ manual).
>
>
> So it's either :
>

> * add xz-tools (né lzma-utils) to the list of prerequisites for Sage

> (i. e. relying on system's liblzma) ;

> * adding xz-tools (at least the lzma library) to sage (cluttering it

> with yet one more library to maintain) ;

> * or relying on system's R through an interface that remains to be
> written...
>
>
> Pick your poison...

Make R optional? (Nothing in Sage depends on it, except for the
interface to it, including Rpy2.)

xz is already an /optional/ Sage package; we'd also need PCRE (or make
it a prerequisite, just for R). (I'd personally make xz standard
anyway, and recompress our upstream tarballs with it, probably even
dropping bzip2.)

In case we made R optional, we could re-include some stuff (such as
PCRE) into the "upstream" tarball as well; not sure if we should do so
if we keep R standard.


My 2ct,

-leif

[kcrisman]

 

Make R optional?  (Nothing in Sage depends on it, except for the
interface to it, including Rpy2.)

Gosh, R has been standard for*ever*, practically, and is often heavily advertised as a good reason to use Sage.  There are certainly many who have been using them together (as mentioned, obviously nowhere near the number of "pure" R users, but still we definitely get queries about this regularly) and of course optional=untested=broken all too often.  Take the Maple or Mathematica interfaces and their on-again, off-again nature... Is this only a Cygwin problem, or on other platforms?  I couldn't see anything about other problems on this thread. 

[Emmanuel Charpentier]

I agree that losing a standard package for failing to build it on one particular target would be bad.

From Trac#20523 :

Replying to [comment:11 embray]:

On Cygwin I needed to `apt-cyg install liblzma-devel` (if you have `apt-cyg`, otherwise use setup as normal).  Without the `-devel` package it doesn't install the import library, so `-llzma` doesn't work (`-llzma.dll` should though).

Okay. I just checked on my (Linux) system : xz is not installed in the $(SAGE_ROOT) tree, probably because I have it installed at system level and the build script detects that somehow. (BTW : it is an optional package.

Therefore, what is needed is :
a. ensure that xz and its development files are installed, either in the system or in the Sage package) ;
b. that this is done before the R installation begins ; and
c. that R installation depends on xz being somehow available.

This can be simplified, at the expense of making xz a standard package, just by ensuring that xz is installed before R compilation.

I have little idea on how to proceed, since the Sage build system is still a (large) bit obscure to me. What I need is a way to check in the Makefile that xz and its development files are available, install xz if not (can an optional package be installed before the compilation of standard packages is finished ?), and make the R compilation dependent on this target.

Any ideas or advice ?

[leif]

Emmanuel Charpentier wrote:
> I agree that losing a standard package for failing to build it on one
> particular target would be bad.
>

> From Trac#20523 <https://trac.sagemath.org/ticket/20523> :

>
> Replying to [comment:11 embray]:
> On Cygwin I needed to `apt-cyg install liblzma-devel` (if you have
> `apt-cyg`, otherwise use setup as normal). Without the `-devel` package
> it doesn't install the import library, so `-llzma` doesn't work
> (`-llzma.dll` should though).
>
> Okay. I just checked on my (Linux) system : xz is not installed in the
> $(SAGE_ROOT) tree, probably because I have it installed at system level

> and the build script detects that somehow. (BTW : it is an /optional/

> package.
>
> Therefore, what is needed is :

> a. ensure that xz /and its development files/ are installed, either in

> the system or in the Sage package) ;

> b. that this is done /before/ the R installation begins ; and
> c. that R installation /depends/ on xz being somehow available.
>
> This can be simplified, at the expense of making xz a /standard/
> package, just by ensuring that xz is installed /before/ R compilation.

I'm all for making xz standard, although for different reason.

> I have little idea on how to proceed, since the Sage build system is
> still a (large) bit obscure to me. What I need is a way to check in the
> Makefile that xz and its development files are available, install xz if
> not (can an optional package be installed before the compilation of
> standard packages is finished ?), and make the R compilation dependent
> on this target.
>
> Any ideas or advice ?

Checking whether xz libs and headers are available (in other words,
whether Sage's xz package needs to get installed) belongs to Sage's
top-level 'configure'. (A probably simpler but dumb way I don't really
like is to act in xz's spkg-install like we do -- for historical reasons
-- in e.g. iconv's: Do nothing in some cases. This would still require
making xz standard of course.)

We still would have to deal with other stuff R no longer includes, OTOH
at least libpcre.

(And standard packages must not depend on optional ones, just saying.)


-leif

[leif]

kcrisman wrote:
>
> Make R optional? (Nothing in Sage depends on it, except for the
> interface to it, including Rpy2.)
>
> Gosh, R has been standard for*ever*, practically,

Hört sich nach Schwäbischem Dreiklang an.

> and is often heavily
> advertised as a good reason to use Sage. There are certainly many who
> have been using them together (as mentioned, obviously nowhere near the
> number of "pure" R users, but still we definitely get queries about this
> regularly)

Well, I guess the ratio of R-thru-Sage users to Sage users is as
"negligible" as that to pure R users. ;-)

I know of exactly /one/ person who reported errors concerning R because
he was using (or trying to use) R; all others just had build issues
(some also doctest failures) with R, and just because it was/is a
standard package.

> and of course optional=untested=broken all too often.

While that's true to some extent, I'd say you confuse cause and effect
here. If hardly anybody is interested in a package, it will presumably
rotten with time, orthogonal to what its type is (except that build and
test issues with /standard/ packages bug every developer and user, no
matter whether anybody actually uses them).

What happened to the role of an spkg maintainer by the way?

> Take
> the Maple or Mathematica interfaces and their on-again, off-again
> nature...

If I'm not mistaken, Sage never shipped Maple nor Mathematica, nor have
there ever been optional packages of them, unfortunately. (So we had no
influence on which version was used either, besides that most developers
and buildbots simply couldn't test, not to mention develop further.)

Also, Rpy wasn't invented by Sage, and is developed independently by others.

> Is this only a Cygwin problem, or on other platforms? I
> couldn't see anything about other problems on this thread.

Wait and see. The prerequisites R removed from its tarball certainly
won't be present on every system. We'd at least have to make them
explicit prerequisites for building Sage(!) if we keep R standard,
despite (my impression being) that only few people at all need Sage's R.


-leif

[Erik Bray]

I think almost any dependency that Sage-the-Python-package can work
without should be considered "optional" insofar as installing Sage is
concerned. I think it's fine for it to be a stadard part of
Sage-the-Distribution.

But this gets almost off-topic and in to my preference that Sage
development make a stronger distinction between those two things. End
of the day though, I should be able to install Sage-the-Python-package
with as few dependencies as possible in order to use it to build
applications and code directly on Sage. Whereas Sage-the-Distribution
is more of an end-user thing (in fact I think the OS packaging would
do well to make this distinction as well--have a minimal Sage that
works, but doesn't necessarily support *all* features, plus a
sage-full that is more of a metapackage including all possible
dependencies.

[William Stein]

On Tue, Aug 23, 2016 at 10:45 PM, Erik Bray <erik....@gmail.com> wrote:
> I think almost any dependency that Sage-the-Python-package can work
> without should be considered "optional" insofar as installing Sage is
> concerned. I think it's fine for it to be a stadard part of
> Sage-the-Distribution.
>
> But this gets almost off-topic and in to my preference that Sage
> development make a stronger distinction between those two things. End
> of the day though, I should be able to install Sage-the-Python-package
> with as few dependencies as possible in order to use it to build
> applications and code directly on Sage. Whereas Sage-the-Distribution
> is more of an end-user thing (in fact I think the OS packaging would
> do well to make this distinction as well--have a minimal Sage that
> works, but doesn't necessarily support *all* features, plus a
> sage-full that is more of a metapackage including all possible
> dependencies.

Huge +1. I strongly agree with all of the above.

William

[leif]

A matter of introducing more package "types" (such as "mandatory",
"shipped by default", "installed by default" etc.), and changing mainly
some (build, release and dist) scripts accordingly. (In addition,
portions of the Sage library would of course have to get adapted as
well, but that's more or less copy-pasting from what we already have,
although we might improve the existing code -- including doctests -- to
become more generic.)


Erik, have you opened some ticket (or thread, wiki page) regarding that?
(Sage "repackagers" will certainly be interested in such as well.)


-leif

P.S.: Alternatively using system-provided packages for optional or
mandatory components (and improving the build system w.r.t. version
requirements) is related, but can IMHO be addressed independently.

'./configure ... && make download && make' should just work as one would
perhaps expect, namely making sure all prerequisites (depending on the
configuration) are present (modulo things like e.g. Xcode of course, but
in some cases we could create 'sudo apt-get install ...' scripts for
example).

[kcrisman]

On Tuesday, August 23, 2016 at 4:54:45 PM UTC-4, leif wrote:

kcrisman wrote:
>
>     Make R optional?  (Nothing in Sage depends on it, except for the
>     interface to it, including Rpy2.)
>
> Gosh, R has been standard for*ever*, practically,

Hört sich nach Schwäbischem Dreiklang an.

Not being from southern Germany, I have no idea what you're talking about ...

 

> and is often heavily
> advertised as a good reason to use Sage.  There are certainly many who
> have been using them together (as mentioned, obviously nowhere near the
> number of "pure" R users, but still we definitely get queries about this
> regularly)

Well, I guess the ratio of R-thru-Sage users to Sage users is as
"negligible" as that to pure R users. ;-)

Just today:

http://ask.sagemath.org/question/34571/linear-regression-with-r-in-sagemath/

I'm not saying it's a huge user group, but if Sage is actually mathematics software and not "people in number theory" software, it would be nice to have good stats and the tons of optional R packages just waiting.  It has definitely been a selling point in many discussions I've had, and I and others have used it ourselves.  Note that even for "brial"/polybori which presumably is not a huge user base either we made things work out.

How this relates to "Sage-the-package" versus "Sage-The-distro" I don't care as long as it's still in "sage-the-distro".

[Erik Bray]

Absolutely it should be--by default in fact, since it already has
been. It would be a major regression to remove it.

I think maybe even a better distinction than "sage-the-distro" is to
call it "SageMath, the Application". As far as users of the
application are concerned I agree the it should have the goal of being
a general purpose stack of mathematics software. For short I'll call
the application/distribution SageMath in (camel-case) and the Python
package "sage" since that's what the actual package is called.

But treating that larger software stack, and the `sage` Python package
as one in the same makes development on the latter more difficult, and
probably ultimately the former as well. For example--continuing with
R as the test particle--say I'm a SageMath user who makes heavy use of
R for my statistics, and like being able to use R with sage. What
happens when a critical bug is fixed in R, and a new patch release
comes out (something Sage, by the way, doesn't even have but should).
How do I get that patch release? Well I have to wait for a new
version of sage (the package, and everything else that comes with it).
That might include a bunch of other new dependencies that I didn't
really think I needed. Possibly even API breakage elsewhere (though
fortunately sage itself is normally pretty good about that).

Better would be to have a stable version of SageMath the application
stack, with a stable version of sage at its core, to which one can
receive small patch upgrades both to sage, as well as to the other
software in the stack, and only make "big" upgrades if I need to, or
want to be on the bleeding edge (it's also perfectly possible and
reasonable to have several at once--a stable version used for work,
and a development or experimental version in a different install
path).

Another option of course is to provide one's own R installation and
have sage use that instead of the R installed in the SageMath stack.
This should definitely be doable for most of sage's dependencies,
especially the optional, non-core ones (and there has been some
progress on that front). In sage there should be at most a way to
get the path to the R to use, and a minimal (best guess at least)
version check for whether it's supported by the interface.

Jeroen and others have pointed out examples where it isn't always so
nice--for example getting important bug fixes bundled up with major
API change bombs in ways that are difficult to extricate. This is
sort of thing is certainly a pain to deal with--sometimes it can force
an otherwise unwanted upgrade. But a model that separates SageMath
from sage doesn't preclude that either. It just means that the stable
release of the former may not get all fixes all the time. In practice
this is rare. Many packages are well-behaved in terms of their
versions, and maintaining stable release lines for a decent amount of
time. For those that don't, most of the time at worst it is the job
of a downstream integrator (like us) to manually cherry-pick a
sensible patch set. Most bug fixes are not hard to backport if
they're relevant to older releases.

On Wed, Aug 24, 2016 at 6:57 PM, leif <not.r...@online.de> wrote:
> A matter of introducing more package "types" (such as "mandatory",
> "shipped by default", "installed by default" etc.), and changing mainly
> some (build, release and dist) scripts accordingly. (In addition,
> portions of the Sage library would of course have to get adapted as
> well, but that's more or less copy-pasting from what we already have,
> although we might improve the existing code -- including doctests -- to
> become more generic.)

Yep. It also helps to better make a distinction between
(required/optional) build dependencies, and (required/optional)
runtime dependencies.

And on top of that it would really help to have a more complete list
of which dependencies are (required/optional) for sage itself, as
opposed to being secondary or tertiary dependencies.

> Erik, have you opened some ticket (or thread, wiki page) regarding that?
> (Sage "repackagers" will certainly be interested in such as well.)

I tried to start a discussion about this a long time ago on this
mailing list, but I dropped the ball on it. Got spread too thin on
other issues and this was maybe a bit "big" to tackle at the time.
Having a ticket in Trac would be a nice way to prevent the issue from
being lost though. Of course, what we're talking about here is really
several issues--the biggest of all being the proposal to better
separate SageMath and sage. But a metaticket could get the ball
rolling and spawn child tickets.

> P.S.: Alternatively using system-provided packages for optional or
> mandatory components (and improving the build system w.r.t. version
> requirements) is related, but can IMHO be addressed independently.

Ah, right! I meant to bring that up above. I have now edited my
message to do so. I agree it should be possible, but is also slightly
tangential from separating SageMath and sage.

> './configure ... && make download && make' should just work as one would
> perhaps expect, namely making sure all prerequisites (depending on the
> configuration) are present (modulo things like e.g. Xcode of course, but
> in some cases we could create 'sudo apt-get install ...' scripts for
> example).

Yep.

Best
Erik

[kcrisman]

I think maybe even a better distinction than "sage-the-distro" is to
call it "SageMath, the Application".  As far as users of the
application are concerned I agree the it should have the goal of being
a general purpose stack of mathematics software.  For short I'll call
the application/distribution SageMath in (camel-case) and the Python
package "sage" since that's what the actual package is called.

But treating that larger software stack, and the `sage` Python package
as one in the same makes development on the latter more difficult, and
probably ultimately the former as well.  For example--continuing with
R as the test particle--say I'm a SageMath user who makes heavy use of
R for my statistics, and like being able to use R with sage.  What
happens when a critical bug is fixed in R, and a new patch release
comes out (something Sage, by the way, doesn't even have but should).
How do I get that patch release?  Well I have to wait for a new
version of sage (the package, and everything else that comes with it).
That might include a bunch of other new dependencies that I didn't
really think I needed.  Possibly even API breakage elsewhere (though
fortunately sage itself is normally pretty good about that).

Yes, I agree that is a very significant issue and of course has been for some time (with respect to many, many packages, I couldn't even list them all for you).  Perhaps unreasonably, I want us to be able to have our cake and eat it too, at least in this regard.  Which is why it is awesome you are on board since maybe eventually this will happen :)

[aishen]

There is a SAGE accounting with is very interfering if you don't write sagemath in a research !

[leif]

kcrisman wrote:
>
> I think maybe even a better distinction than "sage-the-distro" is to
> call it "SageMath, the Application". As far as users of the
> application are concerned I agree the it should have the goal of being
> a general purpose stack of mathematics software. For short I'll call
> the application/distribution SageMath in (camel-case) and the Python
> package "sage" since that's what the actual package is called.
>
> But treating that larger software stack, and the `sage` Python package
> as one in the same makes development on the latter more difficult, and
> probably ultimately the former as well. For example--continuing with
> R as the test particle--say I'm a SageMath user who makes heavy use of
> R for my statistics, and like being able to use R with sage. What
> happens when a critical bug is fixed in R, and a new patch release
> comes out (something Sage, by the way, doesn't even have but should).
> How do I get that patch release? Well I have to wait for a new
> version of sage (the package, and everything else that comes with it).
> That might include a bunch of other new dependencies that I didn't
> really think I needed. Possibly even API breakage elsewhere (though
> fortunately sage itself is normally pretty good about that).
>
>
> Yes, I agree that is a very significant issue and of course has been for
> some time (with respect to many, many packages, I couldn't even list
> them all for you).

... which was less of a problem with our "legacy" spkgs, as you in many
cases could simply install an older or a more recent version when
available /somewhere/, or even easily create your own, just replacing
the upstream version in the src/ folder.

More or less just out of curiosity, did we ever upgrade R because of
bugs in R (besides build/installation issues) R-thru-Sage users
complained about?

Closer on topic, is there urgent need to upgrade our current version of
R? (The 'configure' bug I reported upstream May last year is by the way
still in 3.3.1, although the patch I submitted is pretty trivial.)


-leif

[Jean-Pierre Flori]

By the way I've got all we need for the upgrade. Just got to put it into a branch.

[kcrisman]

... which was less of a problem with our "legacy" spkgs, as you in many
cases could simply install an older or a more recent version when
available /somewhere/, or even easily create your own, just replacing
the upstream version in the src/ folder.

More or less just out of curiosity, did we ever upgrade R because of
bugs in R (besides build/installation issues) R-thru-Sage users
complained about?

R is pretty stable in its core set, so I would find that highly unlikely.  However, we have definitely upgraded R because without having the latest R one would not have access to many (MANY) of the R user packages.  R is pretty aggressive on allowing projects to specify e.g. R version > 2.10 to function properly, and so we've had to upgrade for that reason.  I guess that is not a bug, though.

 

Closer on topic, is there urgent need to upgrade our current version of
R?  (The 'configure' bug I reported upstream May last year is by the way
still in 3.3.1, although the patch I submitted is pretty trivial.)

Like Maxima and others, in principle it would be great to have "rolling" upgrades that semi-automatically happened once passing some tests (in R's case, including graphics, maybe).  Which presumably supports the general notion of decoupling things better? 

[Emmanuel Charpentier]

Well... Tha was ... instructive ...

We see to have two consistent options to keep R a standard package :

1. Keep R as a standard package, including the binaries. This involves :
• make xz a standard package ;
• port the old-style pcre package as a new-style standard package ;
• make R depend on xz and pcre ;
2. Keep an interface to R as a standard package. This involves :
• make a working system's R installation a a prerequisite of R ;
• possibly making this prerequisite (R's)-version-specific.
  

The first solution guarantees that any existing code using R can still be run (modulo R's evolution) but involves maintaining a not-so-small R binary for an indefinite future... It also involves a duplication of R packages libraries between system's and Sage's installations, which can be a non-trivial amount (my *small* use of R packages amounts to about 360 packages, 29 among them being "standard" R packages). This cvcan be alleviated by using Sage's R as the system's R installation (I didn't do that until now for fear of sage's problems : R is my daily bread and butter...).

The second one breaks this guarantee, and introduces the possibility of version incompatibilities between R and Sage.

We could also make R an optional package, with (again) two options :

• Maintain a R optional binary, depending on xz and pcre.
• Maintain an R optional interface.

Making R optional *will* break existing code.

This boils down to :

• R binary standard
• R interface standard
• R binary optional
• R interface optional

 My vote goes to R binary standard for now (dissecting the current package to separate binary and interface and creating the required Makefile modifications is not trivial) ; my second choice would be R interface standard, if we can accept the possibility of R-induced inconsistencies...)

What are your choices ?

[leif]

Emmanuel Charpentier wrote:
> Well... Tha was ... instructive ...
>
> We see to have two consistent options to keep R a standard package :
>

> 1. Keep R as a standard package, including the binaries. This involves :
> * make xz a standard package ;
> * port the old-style pcre package as a new-style standard package ;
> * make R depend on xz and pcre ;
> 2. Keep an interface to R as a standard package. This involves :
> * make a working system's R installation a a prerequisite of R ;
> * possibly making this prerequisite (R's)-version-specific.

>
> The first solution guarantees that any existing code using R can still
> be run (modulo R's evolution) but involves maintaining a not-so-small R
> binary for an indefinite future... It also involves a duplication of R
> packages libraries between system's and Sage's installations, which can
> be a non-trivial amount (my *small* use of R packages amounts to about
> 360 packages, 29 among them being "standard" R packages). This cvcan be
> alleviated by using Sage's R as the system's R installation (I didn't do
> that until now for fear of sage's problems : R is my daily bread and
> butter...).
>
> The second one breaks this guarantee, and introduces the possibility of

> version incompatibilities between R and Sage..

>
> We could also make R an optional package, with (again) two options :
>

> * Maintain a R optional binary, depending on xz and pcre.
> * Maintain an R optional interface.

>
>
> Making R optional *will* break existing code.

How do you come to that conclusion?

Making R optional just means people using R (or the interface to it)
would have to "manually" (=explicitly) install it then (or in the
future, perhaps have to configure Sage '--with-R' or something like that).


-leif

> This boils down to :
>
> * R binary standard
> * R interface standard
> * R binary optional
> * R interface optional

[Emmanuel Charpentier]

Le mardi 30 août 2016 19:33:20 UTC+2, leif a écrit :

> Making R optional *will* break existing code.

How do you come to that conclusion?

To be precise : any code relying on R being standard and using it without bothering checking this assumption.
I have such code myself (e. g. a couple of pedagogic notebooks confronting explicit symbolic integration + function evaluation, numerical integration and Monte-Carlo integration, which uses R for the latter).

Making R optional just means people using R (or the interface to it)
would have to "manually" (=explicitly) install it then (or in the
future, perhaps have to configure Sage '--with-R' or something like that).

(Wouldn't that be "sage -i R" ? ) Indeed. Yet another chore during Sge maintainance.... easily avoided with standard R binary, partially avoided with a standard R interface.

HTH,

--
Emmanuel Charpentier