Conda package for Sage

[Isuru Fernando]

I've packaged sage 7.5.1 for conda for linux and have uploaded it to anaconda.org.

To try it out you can do on linux with miniconda3, (Downloads about 1 GB)

    conda create -n sage sage sage-spkg-sources -c isuruf -c conda-forge

This was a hacky build and I don't expect everything to work properly. sage command, sage notebook, gap, maxima and a few packages I've checked works.

There were 86 standard packages in conda-forge and default channel. (conda-forge is a community effort, see https://conda-forge.github.io/). These conda packages were used instead of sage's packages

74 standard packages were built in the docker container condaforge/linux-anvil with SAGE_FAT_BINARY=yes, which should make the packages compatible with distros as old as CentOS 6. I used sage's build process to create these, kept track of which files were created and copied them over to a conda package.

Things to note are,

1. anaconda.org r channel is incompatible with conda-forge. So rpy2 and r is not bundled, but conda-forge has started packaging the r packages.

2. only the SAGE_LOCAL folder is packaged. So no development.

3. sage-env is sourced when the env is activated, but the deactivation doesn't seem to work properly. Your shell session may become unusable.

4. pinning needs to happen so that ABI compatibility is there.

5. Packages are built with gmp.

6. miniconda2 has issues with path names having non-ASCII characters, so you have to use miniconda3 as your root environment.

Anybody interested in making proper conda packages for the remaining 74 packages and sage itself?

Isuru Fernando

[Samuel Lelievre]

Sun 2017-02-05 13:27:02 UTC+1, Isuru Fernando:

Hi Isuru Fernando,

Thanks for this packaging and for reporting to sage-devel.

I'm cc-ing sage-packaging and sage-release which are also
appropriate place to announce this.

I updated the "Distribution" wiki page at

    https://wiki.sagemath.org/Distribution

Best, Samuel

[Erik Bray]

Thanks a lot for this! Some of us have talked a lot about trying to
make conda packages for Sage and its dependencies. It's definitely an
idea I support, at least in principle. Just nobody until now (that I
know of) has taken the time to sit down and actually try to do it.
Part of the problem is that not many of us are that familiar with
conda. I'm mostly familiar with it by using it, but it's been years
since I've made a package for it (last time I did the project was
still in its infancy and I had to ask directly for their build tools).

I would definitely be interested in helping with this.

[Isuru Fernando]

Hi Erik,

Here's a issue to track the packages in conda-forge, https://github.com/conda-forge/staged-recipes/issues/2357

Conda-forge is a community effort to distribute conda packages. What's better than Continuum's own conda packages is that these packages are much more up-to-date and the maintainers of scientific python stack are maintaining the conda recipe itself. Also you have control of how and when the packages get built.

Conda-forge uses CircleCI, Travis-CI and Appveyor to build packages for Linux, OSX and Windows. You have to submit a PR to https://github.com/conda-forge/staged-recipes and these CI will try to build the packges for you. Here's a PR that adds arb, https://github.com/conda-forge/staged-recipes/pull/1758. After the PR has been merged, it will be moved to a git repo like https://github.com/conda-forge/arb-feedstock. You become a maintainer and you can invite others to maintain the recipe as well. Maintainers will be given access to these git repos (called feedstocks).

Conda-forge has an active gitter channel at https://gitter.im/conda-forge/conda-forge.github.io. Let me know if you need any more info.

Isuru Fernando

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Matthias Koeppe]

On Sunday, February 5, 2017 at 4:27:02 AM UTC-8, Isuru Fernando wrote:

Anybody interested in making proper conda packages for the remaining 74 packages and sage itself?

People interested in this effort may want to look at:

https://trac.sagemath.org/ticket/21507

Task ticket: Make sagelib a pip-installable Python source package, listed on PyPI 

[Isuru Fernando]

Update:

All the build dependencies of sagelib except libgap is in conda-forge now for Linux and OS X. See https://github.com/conda-forge/staged-recipes/issues/2357 for details on what has to be done. If there are volunteers who would like to help me in maintaining these conda recipes, please let me know. Each recipe lives in its own git repo like https://github.com/conda-forge/ntl-feedstock and I can give you push access.

On OS X, the dependencies were compiled with clang and linked to libc++. Fortran packages are compiled with gfortran. What's the status of OS X with clang anyway?

For libgap and gap, I don't know how to fix this issue, https://github.com/conda-forge/gap-feedstock/issues/3 or whether that is a gap issue or a conda-build issue.

Isuru Fernando

[Nicolas M. Thiery]

On Mon, Feb 13, 2017 at 07:50:08AM +0530, Isuru Fernando wrote:
> Update:
> All the build dependencies of sagelib except libgap is in conda-forge
> now for Linux and OS X.

Wow, excellent! Congratulations :-)

> See [1]https://github.com/conda-forge/staged-recipes/issues/2357 for

> details on what has to be done. If there are volunteers who would like
> to help me in maintaining these conda recipes, please let me know. Each
> recipe lives in its own git repo

> like [2]https://github.com/conda-forge/ntl-feedstock and I can give you

> push access.
> On OS X, the dependencies were compiled with clang and linked to
> libc++. Fortran packages are compiled with gfortran. What's the status
> of OS X with clang anyway?
> For libgap and gap, I don't know how to fix this

> issue, [3]https://github.com/conda-forge/gap-feedstock/issues/3 or

> whether that is a gap issue or a conda-build issue.

Sage Days 85 [1] in one week near Paris from now might be a good place
to work on this, since Matthias, Erik, and GAP devs will all be in the
same room.

You would be very welcome to join and we could possibly pay for
some/all of the trip; it's pretty late though, so it's probably
complicated for you to arrange a trip and to get reasonably priced
plane tickets. Let me know in case you'd be interested.

Cheers,
Nicolas

[1] https://wiki.sagemath.org/days85
--
Nicolas M. Thiéry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/

[Isuru Fernando]

Hi,

I've identified the gap issue here, https://github.com/gap-system/gap/issues/1171. So, that's working now.

I've only got a few packages to go before I start testing sagelib. (sympow, maxima and giacpy_sage). Any help is appreciated. There's still the sagenb related packages and optional packages. I'll not be packaging those (sagenb is getting phased out, right?), but I can help out if anybody is interested.

> You would be very welcome to join and we could possibly pay for

some/all of the trip;

Thanks. There's no time to get visas and arrange everything. (I previously thought I'd be busy that week, but now that's cleared up.) I'll join remotely.

Isuru Fernando

[Isuru Fernando]

Hi,

Update: With the help of Erik and Julian, we now have all the runtime dependencies of sage in conda-forge.

There are some more things to be done until it is ready. (like patching sagelib for https://trac.sagemath.org/ticket/20382)

You can install sage using the following commands (Downloads about 1 GB). Note that this is still experimental.

conda config --add channels conda-forge

conda create -n sage sage

Regards,
Isuru Fernando

[mmarco]

Does that work on all the platforms supported by anaconda, or is it linux only?

[Isuru Fernando]

Linux only.

For OSX, all the dependencies are there. Once https://trac.sagemath.org/ticket/12426 is resolved, I can make a package for OSX as well.

--

[Erik Bray]

On Thu, Mar 23, 2017 at 12:25 PM, mmarco <mma...@unizar.es> wrote:
> Does that work on all the platforms supported by anaconda, or is it linux
> only?

For Windows I hope to one day package Sage and its dependencies for
Cygwin too (now that most of them are working). Isuru's work on this
will be a big help for that.

Unfortunately that doesn't help with Anaconda for Windows. One
possibility that I might look into at some point is actually packaging
Cygwin (by which I mean the core Cygwin DLLs) for conda. The downside
to that is that with Cygwin it's kind of all-or-nothing, so I would
also need to make separate packages for versions of most dependencies
built against Cygwin. So yet another possibility would be for conda
to make Cygwin an officially-supported platform for which binaries can
be built but I'm not sure how likely that is either...

> El jueves, 23 de marzo de 2017, 5:29:31 (UTC+1), Isuru Fernando escribió:
>>
>> Hi,
>>
>> Update: With the help of Erik and Julian, we now have all the runtime
>> dependencies of sage in conda-forge.
>>
>> There are some more things to be done until it is ready. (like patching
>> sagelib for https://trac.sagemath.org/ticket/20382)
>>
>> You can install sage using the following commands (Downloads about 1 GB).
>> Note that this is still experimental.
>>
>> conda config --add channels conda-forge
>> conda create -n sage sage
>>
>> Regards,
>> Isuru Fernando
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.

[Isuru Fernando]

Here's a branch for osx with sage 7.6. https://github.com/isuruf/sagelib-feedstock/tree/osx3
Unfortunately travis-ci timeouts due to the build time restrictions. conda-forge's policy has been to do it locally and upload to the anaconda channel.

Since I don't have access to osx, it would be great if somebody could build it on an OSX machine with 10.9 sdk and upload it somewhere. You only need to make sure the clang on the PATH is the apple one and do `conda build recipe` on the repo root. You can set CPU_COUNT variable to set number of threads to be used when building sagelib.

Isuru Fernando

> email to sage-devel+unsubscribe@googlegroups.com.

> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.

[Erik Bray]

On Sun, Jun 11, 2017 at 3:57 PM, Isuru Fernando <isu...@gmail.com> wrote:
> Here's a branch for osx with sage 7.6.
> https://github.com/isuruf/sagelib-feedstock/tree/osx3
> Unfortunately travis-ci timeouts due to the build time restrictions.
> conda-forge's policy has been to do it locally and upload to the anaconda
> channel.
>
> Since I don't have access to osx, it would be great if somebody could build
> it on an OSX machine with 10.9 sdk and upload it somewhere. You only need to
> make sure the clang on the PATH is the apple one and do `conda build recipe`
> on the repo root. You can set CPU_COUNT variable to set number of threads to
> be used when building sagelib.

I wonder if OpenDreamKit could pay for an OSX machine to test on,
because I have no good way to test either. Granted, I feel with every
OSX version there's a completely different XCode that changes
everything radically (that was my experience, at least, back when I
had a lot of astronomer users who all used OSX--every OS upgrade broke
everything). How do people get VMs for testing different OSX / XCode
versions?

>> > email to sage-devel+...@googlegroups.com.

>> > To post to this group, send email to sage-...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sage-devel.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to sage-devel+...@googlegroups.com.

>> To post to this group, send email to sage-...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sage-devel.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to sage-devel+...@googlegroups.com.

[Dima Pasechnik]

On Wednesday, June 14, 2017 at 10:00:05 AM UTC+1, Erik Bray wrote:

On Sun, Jun 11, 2017 at 3:57 PM, Isuru Fernando <isu...@gmail.com> wrote:
> Here's a branch for osx with sage 7.6.
> https://github.com/isuruf/sagelib-feedstock/tree/osx3
> Unfortunately travis-ci timeouts due to the build time restrictions.
> conda-forge's policy has been to do it locally and upload to the anaconda
> channel.
>
> Since I don't have access to osx, it would be great if somebody could build
> it on an OSX machine with 10.9 sdk and upload it somewhere. You only need to
> make sure the clang on the PATH is the apple one and do `conda build recipe`
> on the repo root. You can set CPU_COUNT variable to set number of threads to
> be used when building sagelib.

I wonder if OpenDreamKit could pay for an OSX machine to test on,
because I have no good way to test either.  Granted, I feel with every
OSX version there's a completely different XCode that changes
everything radically (that was my experience, at least, back when I
had a lot of astronomer users who all used OSX--every OS upgrade broke
everything).  How do people get VMs for testing different OSX / XCode
versions?

 

I've just asked an Apple insider (the guy works for Apple as a senior dev, in fact)

I happen to know, I just cannot imagine they don't have some meaningful tips...

Anyhow, I could buy a Mac Mini with ODK money and hook it up on the net.

We can also have a look at getting a VM, e.g. something like https://xcloud.me/ (not cheap though...)

Dima

[Nathan Dunfield]

 How do people get VMs for testing different OSX / XCode versions?

Both VirtualBox and VMWare (Fusion) fully support OS X VMs with the important caveat you have to use actual Apple hardware for the host.

I've used both without major problems on a Mac Pro.  Currently I mostly use VMWare since it's a little more polished (my university has a site-license).  You can configure VMWare so you that you can both ssh into the VM from the host as well as interact with the full GUI, the latter even remotely using VMWare's VNC capability.   With fairly minimal installs and only the occasional snapshot, I find you need about 40-50Gb of disk space for each VM.  

I believe Travis-CI offers OS X nodes, though building Sage might well time out on their free tier.  

Best,

Nathan

[Dima Pasechnik]

problem is, Apple headless hardware range currently available consists of Mac Mini's, which are very much underpowered.

they promise something better in Dec.

Internet is offering details on how to hack OSX to run in a VM which is not running on Apple hardware. Perhaps this is the best way out.

[Jean-Pierre Flori]

From what I tried and heard that more or less works depending on OS X version.
And is unfortunately illegal according to Apple though I'm not a lawyer.

[Dima Pasechnik]

https://www.apple.com/fr/shop/buy-mac/mac-pro

I don't see why ODK cannot spend 4600 euro to buy one of these...

(it is way above my ODK budget, but would still be peanuts for the whole project)

[Erik Bray]

On Thu, Jun 15, 2017 at 11:49 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> https://www.apple.com/fr/shop/buy-mac/mac-pro
>
> I don't see why ODK cannot spend 4600 euro to buy one of these...
> (it is way above my ODK budget, but would still be peanuts for the whole
> project)

Plus licenses for a range of OSX versions (at least those that are
still supported by Apple...)

> On Thursday, June 15, 2017 at 10:25:57 AM UTC+1, Jean-Pierre Flori wrote:
>>
>>
>>
>> On Thursday, June 15, 2017 at 8:29:37 AM UTC+2, Dima Pasechnik wrote:
>>>
>>> problem is, Apple headless hardware range currently available consists
>>> of Mac Mini's, which are very much underpowered.
>>>
>>> they promise something better in Dec.
>>>
>>> Internet is offering details on how to hack OSX to run in a VM which is
>>> not running on Apple hardware. Perhaps this is the best way out.
>>
>> From what I tried and heard that more or less works depending on OS X
>> version.
>> And is unfortunately illegal according to Apple though I'm not a lawyer.
>

[Dima Pasechnik]

On Thursday, June 15, 2017 at 11:03:11 AM UTC+1, Erik Bray wrote:

On Thu, Jun 15, 2017 at 11:49 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> https://www.apple.com/fr/shop/buy-mac/mac-pro
>
> I don't see why ODK cannot spend 4600 euro to buy one of these...
> (it is way above my ODK budget, but would still be peanuts for the whole
> project)

Plus licenses for a range of OSX versions (at least those that are
still supported by Apple...)

these are peanuts, e.g. £20 for 10.7

https://www.apple.com/uk/shop/product/D6106ZM/A/os-x-lion

(but 1st thing 1st, let's start from the current version being well-supported...)

[Jean-Pierre Flori]

On Thursday, June 15, 2017 at 12:13:20 PM UTC+2, Dima Pasechnik wrote:

On Thursday, June 15, 2017 at 11:03:11 AM UTC+1, Erik Bray wrote:

On Thu, Jun 15, 2017 at 11:49 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> https://www.apple.com/fr/shop/buy-mac/mac-pro
>
> I don't see why ODK cannot spend 4600 euro to buy one of these...
> (it is way above my ODK budget, but would still be peanuts for the whole
> project)

Plus licenses for a range of OSX versions (at least those that are
still supported by Apple...)

these are peanuts, e.g. £20 for 10.7

https://www.apple.com/uk/shop/product/D6106ZM/A/os-x-lion

(but 1st thing 1st, let's start from the current version being well-supported...)

Or rather let's find someone from ODK willing to buy the hardware!

(Next we could buy some sparc and itanium CPUs and Solaris licenses and a power8/9 :))

[William Stein]

We literally used to have such a cluster for sage testing, including itanium!

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--

-- William Stein

[Jean-Pierre Flori]

On Thursday, June 15, 2017 at 2:31:58 PM UTC+2, William wrote:

(but 1st thing 1st, let's start from the current version being well-supported...)

Or rather let's find someone from ODK willing to buy the hardware!

(Next we could buy some sparc and itanium CPUs and Solaris licenses and a power8/9 :))

We literally used to have such a cluster for sage testing, including itanium!

Yes I remember and I advocated that one of ODK goals should  be portability and such hardware should be bought back, including hardware for testing cygwin portability, but I was never an official member of ODK.

[Nathan Dunfield]

Plus licenses for a range of OSX versions (at least those that are
still supported by Apple...)

these are peanuts, e.g. £20 for 10.7

I don't know if this is still the case, but as of a year ago if you become a registered Apple Developer then you got access to all the old versions of OS X and XCode as well as the pre-releases.  That costs US$100/year, but once you have the complete set you could drop the subscription since it's easy to keep current: just copy the VM for version n when n+1 comes out and the let the system update itself. 

With regards to hardware, the single-core performance of the Mac Pro isn't that much better than the Mac Mini, see:

https://browser.primatelabs.com/mac-benchmarks

but of course the Mac Mini has only 2 cores.  If space is not an issue, you might think about the iMac with the i7-7700K, or, if it is, the top of the line Mac Book Pro is quite a bit more powerful than the Mini at this point.

Nathan

 

[Volker Braun]

On Thursday, June 15, 2017 at 11:25:57 AM UTC+2, Jean-Pierre Flori wrote:

And is unfortunately illegal according to Apple though I'm not a lawyer.

May or may not be legal depending on your jurisdiction. In the US its illegal according to the Psystar case. In Germany, most click-through terms and conditions are void as your consumer rights cannot be restricted post-sale. Most of the EU probably has similar consumer protection laws. 

[Volker Braun]

On Thursday, June 15, 2017 at 3:10:22 PM UTC+2, Nathan Dunfield wrote:

but of course the Mac Mini has only 2 cores.

For the record, our OSX buildbot is a quad-core mac mini (i7-3720QM CPU @ 2.60GHz).

CPU-wise its actually reasonable, but the box is just too small. Fan goes crazy if you actually compute something, and no space for disks. On the plus side, it looks good on your desk when idle.

I did build a hackintosh for fun from some spare parts that I had lying around (i.e. faster cpu, more memory, bigger harddisk) than the mac mini. It takes some fiddling to get it working. Tough at the end of the day I still don't like OSX that much, too much eye candy and not enough work horse.

[Nathan Dunfield]

On Thursday, June 15, 2017 at 12:38:40 PM UTC-5, Volker Braun wrote:

On Thursday, June 15, 2017 at 3:10:22 PM UTC+2, Nathan Dunfield wrote:

but of course the Mac Mini has only 2 cores.

For the record, our OSX buildbot is a quad-core mac mini (i7-3720QM CPU @ 2.60GHz).

Inexplicably, the option of a quad-core processor was dropped with the current Mac mini revision that came out in late 2014.

Nathan