"replacement" optional packages

[Thierry]

Hi,

having the possibility to install all optional packages is interesting for
easy deployment as well as for testing (we need more automated tests of
all optional packages, in particular more patchbots with optional packages
installed). Currently, some optional packages, like gmp/mpir,
atlas/openblas, boost/boost_cropped (what else ?) are alternatives to some
existing standard packages. Hence, those should not be installed when we
want to set up a patchbot to test optional packages (unless we want to
specifically test those). There is also the special case of openssl, which
should not be installed when openssl-dev is present on the system.

I would like to suggest to have such
replacement/duplicate/redundent/alternative/... packages be tagged not as
'optional' but with another word, so as to ease the selection and
installation of all optional packages, in order to easily provide some
kind of "sage-full" binaries and patchbots. I currently use a hand-made
list of optional packages for Sage Debian Live but it requires to be
updated at each release.

- What do you think about distinguish such packages to optional ones ?
- Which name should be given to such packages ?
- What are your current strategies to install all meaningful optional
packages ?

Ciao,
Thierry

[Travis Scrimshaw]

+1 to having another marker for those alternative-to-standard packages, but I think this is a bit of a problem to do in some general framework. These packages are suppose to come in sets and one of them will have to be installed. So it might be better to think of ways to combine these packages under common descriptions that the user could swap out by rebuilding.

Best,
Travis

[Dima Pasechnik]

On Tuesday, February 28, 2017 at 5:20:23 PM UTC, Thierry (sage-googlesucks@xxx) wrote:

Hi,

having the possibility to install all optional packages is interesting for
easy deployment as well as for testing (we need more automated tests of
all optional packages, in particular more patchbots with optional packages
installed). Currently, some optional packages, like gmp/mpir,
atlas/openblas, boost/boost_cropped (what else ?) are alternatives to some
existing standard packages. Hence, those should not be installed when we
want to set up a patchbot to test optional packages (unless we want to
specifically test those). There is also the special case of openssl, which
should not be installed when openssl-dev is present on the system.

I would like to suggest to have such
replacement/duplicate/redundent/alternative/...  packages be tagged not as
'optional' but with another word, so as to ease the selection and
installation of all optional packages, in order to easily provide some
kind of "sage-full" binaries and patchbots. I currently use a hand-made
list of optional packages for Sage Debian Live but it requires to be
updated at each release.

IMHO it's something to be encoded---already encoded, right?---

in dependencies rather than in type.

So all this can be recovered by parsing dependencies.

[Jeroen Demeyer]

I have been thinking about this too. My personal conclusion was that the
"type" enumeration (standard, optional, experimental, pip, script) is
simply too restricted and that we need additional metadata with more
degrees of freedom.

Currently, the "type" field is relevant for:
- which packages are installed by default
- which packages should be packed in the source tarball
- which --optional tags are given when doctesting
- whether a warning message is given when installing the package
- the Make rules of a package
- the automatic dependencies of a package

I think that's bordering on being too much already. So +1 to more
metadata but -1 to inventing yet another type.

[Dima Pasechnik]

Probably one can have an optional `replacements` file

in the package directory of each package that can be replaced by some others; or,

in build/packages/, a file named `alternatives` listing alternative packages, e.g.

altas openblas

foo bar baz

... <another list of alternatives>...

...

 

[Erik Bray]

Agreed. What we want here is "provides". For example:

"provides: blas"

where "blas" is an abstract dependency which may be fulfilled by
multiple packages (more or less like we already do, but more
concretely).

In Debian the alternatives [1] system is used to manage what concrete
package is being used to provide an abstract dependency.

Coincidentally I posted #22509 [2], which might be useful for this, a
little before reading this thread. My motivation there was to support
uninstallation better. But another case for it could be managing
package alternatives in such a way that obviates needing to rebuild
every one switches packages. For example, one could have builds of
both openblas and atlas living in directories outside $SAGE_LOCAL (but
that are *not* temporary as my ticket suggests). Then one could use a
modified form of the uninstall procedure described in #22510 [3] to
swap one package out for the other.

[1] https://wiki.debian.org/DebianAlternatives
[2] https://trac.sagemath.org/ticket/22509
[3] https://trac.sagemath.org/ticket/22510

[Jeroen Demeyer]

Speaking of optional packages, we should also think of a strategy to
deal with optional packages which have dependencies which are not part
of Sage.

For example, the optional rst2ipynb package requires pandoc, which is
not in Sage.

On the other hand, in #22378 I wanted to package surf as optional
package and got complains from Dima that I couldn't because it had
dependencies not in Sage (such as libjpeg).

[Dima Pasechnik]

FYI, pandoc installs and runs on all the platforms that sage supports.
But surf does not.

in this sense dependence on pandoc is OK,
dependence on surf - much less so.

[Justin C. Walker]

If you can get surf to build/run on macOS that would be terrific. I have not managed to do it, and AFAICT, no one else has either.

Justin

--
Justin C. Walker, Curmudgeon-At-Large
Director
Institute for the Enhancement of the Director's Income
--------
"Weaseling out of things is what separates us from the animals.
Well, except the weasel."
- Homer J Simpson
--------

[Jeroen Demeyer]

On 2017-03-11 20:18, Dima Pasechnik wrote:
> in this sense dependence on pandoc is OK,
> dependence on surf - much less so.

Isn't the issue with surf the same as pandoc: it requires dependencies

[Dima Pasechnik]

there is no "issue" with pandoc. Most linux distros allow you to install it using package manager, and it is easy to install it on OSX or Windows.

On the other hand, it appears that surf had never been ported to OSX.

[Francois Bissey]

Apart from dependencies like jpeg what’s the problem with surf?

François

> On 12/03/2017, at 21:27, Dima Pasechnik <dim...@gmail.com> wrote:
>
> there is no "issue" with pandoc. Most linux distros allow you to install it using package manager, and it is easy to install it on OSX or Windows.
>
> On the other hand, it appears that surf had never been ported to OSX.
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Dima Pasechnik]

X11?

[Jeroen Demeyer]

On 2017-03-12 09:35, Dima Pasechnik wrote:
> X11?

That can be disabled with --disable-gui

[Jeroen Demeyer]

On 2017-03-12 09:27, Dima Pasechnik wrote:
> On the other hand, it appears that surf had never been ported to OSX.

What does that mean? It just means that pandoc is more popular than surf.

I somebody has really tried to compile surf on OS X and didn't manage,
that would be a different story. But if nobody has even tried, it means
nothing.

[Francois Bissey]

Indeed X absence is not fatal in configure unlike libjpeg.
I guess my point is that before saying it needs porting it should
be tested in something like brew that can bring dependencies
like jpeg.

François

[Justin C. Walker]

On Mar 12, 2017, at 00:27 , Dima Pasechnik wrote:

> there is no "issue" with pandoc. Most linux distros allow you to install it using package manager, and it is easy to install it on OSX or Windows.
>
> On the other hand, it appears that surf had never been ported to OSX.

No: in fact, MacPorts had a port a while back (maybe 2013), but it no longer works, and AFAIK, it's not supported there. I did have a port from the surf team in a farther-away land, and they gave up on macOS after a bit. The last copy I have was from ~2005-6. That was before 10.6 (maybe the 10.4 epoch). I think the problem may be with the version of Qt required, but I'm not sure.

Justin

--
Justin C. Walker, Curmudgeon at Large
Institute for the Absorption of Federal Funds
-----------
If it weren't for carbon-14, I wouldn't date at all.
-----------

[Francois Bissey]

> On 13/03/2017, at 07:34, Justin C. Walker <jus...@mac.com> wrote:
>
>
> On Mar 12, 2017, at 00:27 , Dima Pasechnik wrote:
>
>> there is no "issue" with pandoc. Most linux distros allow you to install it using package manager, and it is easy to install it on OSX or Windows.
>>
>> On the other hand, it appears that surf had never been ported to OSX.
>
> No: in fact, MacPorts had a port a while back (maybe 2013), but it no longer works, and AFAIK, it's not supported there. I did have a port from the surf team in a farther-away land, and they gave up on macOS after a bit. The last copy I have was from ~2005-6. That was before 10.6 (maybe the 10.4 epoch). I think the problem may be with the version of Qt required, but I'm not sure.
>

Not qt but gtk. But that’s the right feeling since we are
talking about gtk-1.2.x here.
However this is optional, but may be you don’t find it very useful
without the gui. My main complaint is that the project is dead
as far as anyone can see. The last release is from 2010.
And I suspect the tarball Jeroen put in experimental has been
tweaked as it has a gcc6 ending in its name.

François

[Justin C. Walker]

On Mar 12, 2017, at 11:43 , Francois Bissey wrote:

>
>> On 13/03/2017, at 07:34, Justin C. Walker <jus...@mac.com> wrote:
>>
>>
>> On Mar 12, 2017, at 00:27 , Dima Pasechnik wrote:
>>
>>> there is no "issue" with pandoc. Most linux distros allow you to install it using package manager, and it is easy to install it on OSX or Windows.
>>>
>>> On the other hand, it appears that surf had never been ported to OSX.
>>
>> No: in fact, MacPorts had a port a while back (maybe 2013), but it no longer works, and AFAIK, it's not supported there. I did have a port from the surf team in a farther-away land, and they gave up on macOS after a bit. The last copy I have was from ~2005-6. That was before 10.6 (maybe the 10.4 epoch). I think the problem may be with the version of Qt required, but I'm not sure.
>>
>
> Not qt but gtk. But that’s the right feeling since we are
> talking about gtk-1.2.x here.

Right. Qt arose in farbling with CoCoA-GUI on macOS - beyond my capabilities to grok. Just like gtk...

> However this is optional, but may be you don’t find it very useful
> without the gui. My main complaint is that the project is dead
> as far as anyone can see. The last release is from 2010.
> And I suspect the tarball Jeroen put in experimental has been
> tweaked as it has a gcc6 ending in its name.

I'm trying to imagine surf without graphics (:-}), but perhaps that's not what you mean exactly.

Thanks for clarifying.

Justin

--
Justin C. Walker, Curmudgeon-At-Large, Director

Institute for the Enhancement of the Director's Income
--------

The path of least resistance:
it's not just for electricity any more.
--------

[François Bissey]

On 13/03/17 11:00, Justin C. Walker wrote:
> I'm trying to imagine surf without graphics (:-}), but perhaps that's not what you mean exactly.
>
> Thanks for clarifying.

I don't know surf :) but I imagine it can output
to jpeg and tiff directly. And if X is found to
a basic x-window.

If the gui offer navigation inside the window kind
of thing. Then, yes you don't have that, which may be
annoying.

Francois

[Dima Pasechnik]

On Sunday, March 12, 2017 at 10:05:56 PM UTC, François wrote:

On 13/03/17 11:00, Justin C. Walker wrote:
> I'm trying to imagine surf without graphics (:-}), but perhaps that's not what you mean exactly.
>
> Thanks for clarifying.

I don't know surf :) but I imagine it can output
to jpeg and tiff directly. And if X is found to
a basic x-window.

AFAIK this is about mostly being able to use surf in jupyter kernel for singular.

Then it suffices that surf produces a graphics file, to be shown in a web browser.

[Dima Pasechnik]

this does not work, still (here is a slightly weird error I get on OSX (with Sage's gcc 5.4)):

g++ -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"surf\" -DVERSION=\"1.0.6\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DYYTEXT_POINTER=1 -DHAVE_EXPLICIT_TEMPLATE_INSTANTIATION=1 -DNO_GUI=1 -DX_DISPLAY_MISSING=1 -DHAVE_LIBM=1 -DHAVE_LIBZ=1 -DHAVE_LIBJPEG=1 -DHAVE_LIBTIFF=1 -DHAVE_LIBGMP=1 -I. -I.  -I../src -I../drawfunc -I../yaccsrc -I../image-formats -I../curve -I../mt -I../draw -I../misc -I../debug -I../xpm -I../dither  -DDATADIR=\"\"  -g -O2 -fno-rtti -fno-exceptions -Wall -W -Wwrite-strings -Wpointer-arith -Wconversion -Wno-unused -Woverloaded-virtual -Wno-deprecated  -c xwd.cc

g++ -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"surf\" -DVERSION=\"1.0.6\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DYYTEXT_POINTER=1 -DHAVE_EXPLICIT_TEMPLATE_INSTANTIATION=1 -DNO_GUI=1 -DX_DISPLAY_MISSING=1 -DHAVE_LIBM=1 -DHAVE_LIBZ=1 -DHAVE_LIBJPEG=1 -DHAVE_LIBTIFF=1 -DHAVE_LIBGMP=1 -I. -I.  -I../src -I../drawfunc -I../yaccsrc -I../image-formats -I../curve -I../mt -I../draw -I../misc -I../debug -I../xpm -I../dither  -DDATADIR=\"\"  -g -O2 -fno-rtti -fno-exceptions -Wall -W -Wwrite-strings -Wpointer-arith -Wconversion -Wno-unused -Woverloaded-virtual -Wno-deprecated  -c jpeg.cc

jpeg.cc: In function 'bool write_jpeg_file(byte*, byte*, byte*, int, int, FILE*)':

jpeg.cc:59:41: error: cannot convert 'bool' to 'boolean' for argument '3' to 'void jpeg_set_quality(j_compress_ptr, int, boolean)'

  jpeg_set_quality (&cinfo, quality, true);

                                         ^

jpeg.cc:60:35: error: cannot convert 'bool' to 'boolean' for argument '2' to 'void jpeg_start_compress(j_compress_ptr, boolean)'

  jpeg_start_compress (&cinfo, true);

                                   ^

make[1]: *** [jpeg.o] Error 1

make[1]: *** Waiting for unfinished jobs....

xwd.cc:54:22: fatal error: X11/Xlib.h: No such file or directory

compilation terminated.

make[1]: *** [xwd.o] Error 1