I was trying to find if it was discussed before, but didn't find
anything -- what is your view on spkg dependencies?
It is not that bad so far, but still I need to remember in which order
to install all my aditional software, e.g. cmake first, then the
fortran package, then my wrappers, it'd be convenient if the user
could just do "sage -i qsnake" and it would do the right thing.
I also spent couple hours investigating some package managers that
handle dependencies and don't require root access. I quite liked:
There are packages in almost all distributions, but then I tried to
install it on our department cluster:
http://0install.net/install-source.html
and it utterly failed:
$ python setup.py install --home ~ --install-data ~/.local
running install
error: invalid Python installation: unable to open
/usr/lib/python2.3/config/Makefile (No such file or directory)
Yes, I know there is some old python. So that's a show stopper.
Ideally, I would like to bootstrap like Sage, from nothing. E.g. the
user would download a small tarball, that would contain basically just
a python based package manager + python itself (if it's not installed
on the system). And then he could install and remove any packages he
wants, they would download & install into ~/.cache or something. So it
would work like a source distribution, that runs everywhere. The
advantage is that libraries like numpy, lapack, blas, etc. would get
installed only once and all the upgrades would just download a small
thing. The Sage current approach is to download and compile everything all
over again.
But I don't want to start anything new, so I am just curious about
Sage plans in the future about this. Basically, what I need from Sage
is atlas, lapack, numpy, scipy, python and then the possibility to
install all my additional packages.
Another thing --- I'd like to create some repository with my packages,
so that people can just "sage -i" install them, without having to
first wget all the spkg and install them manually. So I thought I
would get my packages to sage experimental, but is there any procedure
for that?
I know that all of this is reinventing the wheel and basically doing
what linux distributions are doing, but Sage imho is a distribution --
a source distribution that runs everywhere and actually compiles ---
well, it's true that each time I tried to compile sage on some cluster
(2x so far), it failed :), but I think I am an exception, since I used
some older g++, or some other stuff was broken.
Ondrej
Thanks for the information.
>
>> Ideally, I would like to bootstrap like Sage, from nothing. E.g. the
>> user would download a small tarball, that would contain basically just
>> a python based package manager + python itself (if it's not installed
>> on the system). And then he could install and remove any packages he
>> wants, they would download & install into ~/.cache or something. So it
>> would work like a source distribution, that runs everywhere. The
>> advantage is that libraries like numpy, lapack, blas, etc. would get
>> installed only once and all the upgrades would just download a small
>> thing. The Sage current approach is to download and compile everything all
>> over again.
>
> This is wrong, you can upgrade from release to release. And you can
> drop spkgs on any webserver and set some env variable and Sage will
> pull from it and get spkg updates.
Ah, that's cool. I'll do that.
But I meant more for users who install it on their computers --- so
I'll submit my packages into experimental, I guess that's the way.
>> But I don't want to start anything new, so I am just curious about
>> Sage plans in the future about this. Basically, what I need from Sage
>> is atlas, lapack, numpy, scipy, python and then the possibility to
>> install all my additional packages.
>
> You want to make this stand alone? Take a look at local/bin.
sage/local/bin? Or /usr/loca/bin?
>
>> Another thing --- I'd like to create some repository with my packages,
>> so that people can just "sage -i" install them, without having to
>> first wget all the spkg and install them manually. So I thought I
>> would get my packages to sage experimental, but is there any procedure
>> for that?
>
> Yes, submit them for inclusion. But you should add dependency checks
> inside the spkg so things do not blow up.
Ok, I will, when I polish them more.
>
>> I know that all of this is reinventing the wheel and basically doing
>> what linux distributions are doing, but Sage imho is a distribution --
>> a source distribution that runs everywhere and actually compiles ---
>> well, it's true that each time I tried to compile sage on some cluster
>> (2x so far), it failed :), but I think I am an exception, since I used
>> some older g++, or some other stuff was broken.
>
> Well, in once case you used a truly screwed up distribution and the
Yes, but I didn't have root access on that boxes.
> bugs got fixed with your help. What do you expect from 5,000,000 lines
> of code? It works much more reliable than any other project I have
> ever build from sources in that size range.
Exactly, that's why I want to use Sage and not something else, because
it's the best.
Ondrej
You can toss some "sage -i's" in the beginning of the spkg-install for qsnake.
> Another thing --- I'd like to create some repository with my packages,
> so that people can just "sage -i" install them, without having to
> first wget all the spkg and install them manually. So I thought I
> would get my packages to sage experimental, but is there any procedure
> for that?
A while ago I added a feature so one can do
sage -i http://url.to.an.spkg/anywhere/on/the/net
and sage will download the spkg and install it. That might be good for
you so that users do *not* have to use wget (which they might not
have, e.g., it isn't on OS X).
> I know that all of this is reinventing the wheel and basically doing
> what linux distributions are doing, but Sage imho is a distribution --
> a source distribution that runs everywhere and actually compiles ---
We definitely try very hard and it will only keep getting better.
William
This is really helpful. I just posted the spkg to our google code page
and then it can be installed with:
MAKE="make -j9" sage -i http://qsnake.googlecode.com/files/openmx.spkg
Well, not now, because I haven't yet updated the g95 problem that
Michael just helped me fix and google code is read only now for
maintenance. :( I think I depend too much on google, when I needed to
send an important email, gmail went down for couple hours and I was
literally stuck and had to use some other webmail.
One more question: -- I am trying to take sage-3.3.tar and strip it
from unnecessary spkgs that I don't need -- seems to me that I can
just delete them? I am learning the dependencies in
spkg/standard/deps, because it's a bit tricky -- as my first attempt I
deleted almost everything and it didn't build, so now I am more
careful. I want to create some minimal build, so that people can take
it (hopefully it will be just couple MB) and then install only
packages that they need. I will be experimenting some more with this,
until I figure it out.
Ondrej
I won't.
> * what license do you want to use for the code from the local/bin
> repo in Sage? The code in that area is not BSD compatible.
GPL2+.
I will use BSD for code that I write, unless I am forced otherwise. I
think as long as my code runs standalone, I can use BSD and when I
include one package that bundles everything, I will just use GPL2+ and
I should be fine.
Ondrej
But, we will give Sage credit :)
> Yes, absolutely. I just wanted to make sure no one slabbed BSD license
> headers on top of scripts I have significantly contributed to without
> asking (and I did not think of you in that case, but there are other
> people in your project).
No fears, I am the only other person involved in this project and I
(Ondrej as well) am intimately aware of all the licensing issues.
Thanks for the help though.
Brian
Either that, or if I put it into spkg, it can be gpl, I don't care,
because spkgs will be used in Sage that is GPL. What I care is that my
programs can be used as BSD if someone wants (e.g. without Sage).
>> I can use BSD and when I
>> include one package that bundles everything, I will just use GPL2+ and
>> I should be fine.
>
> Yes, absolutely. I just wanted to make sure no one slabbed BSD license
> headers on top of scripts I have significantly contributed to without
> asking (and I did not think of you in that case, but there are other
> people in your project).
As Brian said, we don't do such things. Besides it's not even my way
of thinking to change somebody's mind, so if I use your work, I use it
under the conditions that you set. If I don't like it, I will have to
use somebody's else work or do the work myself.
Ondrej
If building openmx really benefits from "make -j" you might consider
putting something clever in spkg-install to detect "available cores" and
maybe build using them...
>
> Well, not now, because I haven't yet updated the g95 problem that
> Michael just helped me fix and google code is read only now for
> maintenance. :( I think I depend too much on google, when I needed to
> send an important email, gmail went down for couple hours and I was
> literally stuck and had to use some other webmail.
You should use Sage in such cases. Type
sage: email?
for more details. That uses Twisted behind the scenes.
>
> One more question: -- I am trying to take sage-3.3.tar and strip it
> from unnecessary spkgs that I don't need -- seems to me that I can
> just delete them? I am learning the dependencies in
> spkg/standard/deps, because it's a bit tricky -- as my first attempt I
> deleted almost everything and it didn't build, so now I am more
> careful. I want to create some minimal build, so that people can take
> it (hopefully it will be just couple MB) and then install only
> packages that they need. I will be experimenting some more with this,
> until I figure it out.
>
> Ondrej
>
Go for it. That's an awesome idea. We should make this more
generically do-able though.
This reminds of how Debian is a Linux distribution that dozens
(hundreds?) of other custom Linux distributions derive from. Sage
could be the same. That would be cool.
William
Especially since he asked earlier in the list for permission to
rebrand the Sage notebook as something else :-).
>
>> > * what license do you want to use for the code from the local/bin
>> > repo in Sage? The code in that area is not BSD compatible.
>>
>> GPL2+.
>>
>> I will use BSD for code that I write, unless I am forced otherwise. I
>> think as long as my code runs standalone,
>
> I assume you will have an spkg that contains your BSD licensed code,
> but just uses some bits of the Sage building system to build it. That
> is perfectly legal and does not cause any issues with the GPL.
Yep.
>> I can use BSD and when I
>> include one package that bundles everything, I will just use GPL2+ and
>> I should be fine.
>
> Yes, absolutely. I just wanted to make sure no one slabbed BSD license
> headers on top of scripts I have significantly contributed to without
> asking (and I did not think of you in that case, but there are other
> people in your project).
We should make sure there are clear GPL headers on all
local/bin/sage-* scripts....
William
Yes, but I need this for all packages, so I think I'll rather
customize my own derived sage to do this automatically. For Michael:
we'll it's not sage. :)
>> Well, not now, because I haven't yet updated the g95 problem that
>> Michael just helped me fix and google code is read only now for
>> maintenance. :( I think I depend too much on google, when I needed to
>> send an important email, gmail went down for couple hours and I was
>> literally stuck and had to use some other webmail.
>
> You should use Sage in such cases. Type
>
> sage: email?
>
> for more details. That uses Twisted behind the scenes.
I think my sage is a little bit old for emails:)
$ sage
----------------------------------------------------------------------
| SAGE Version 3.1.2, Release Date: 2008-09-19 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage: email?
Object `email` not found.
>> One more question: -- I am trying to take sage-3.3.tar and strip it
>> from unnecessary spkgs that I don't need -- seems to me that I can
>> just delete them? I am learning the dependencies in
>> spkg/standard/deps, because it's a bit tricky -- as my first attempt I
>> deleted almost everything and it didn't build, so now I am more
>> careful. I want to create some minimal build, so that people can take
>> it (hopefully it will be just couple MB) and then install only
>> packages that they need. I will be experimenting some more with this,
>> until I figure it out.
>>
>> Ondrej
>>
>
> Go for it. That's an awesome idea. We should make this more
> generically do-able though.
>
> This reminds of how Debian is a Linux distribution that dozens
> (hundreds?) of other custom Linux distributions derive from. Sage
> could be the same. That would be cool.
Ok. I did some progress yesterday:
http://code.google.com/p/qsnake/wiki/qsnake_all
Basically you download 3.2MB tarball and then install everything else
using "sage -i". That's what I want! There is not even python in my
qsnake_all. If you type ./sage, it will enter an infinite loop, see my
wiki above for instructions how to fix it.
However, there is a big problem with dependencies, as you need to
install the packages in correct order. So I will create some meta
packages that will do that for me, e.g. scipy-all-0.1, that would
install all packages needed for scipy in correct order. Or
sage-all-0.1, that would install Sage+packages. Or mayavi2-all for
mayavi.
Secretly I hope that it will spark interest of other people, so that
we can maintain this stripped sage (michael: not sage) together, as I
think it can do the job for all the people who wanted something like
python(x,y) for linux. Well, here you have it.
When Sage is ported to windows, I'll try to do the same, even though,
I don't have any windows here, so maybe someone else will do it for
me. :)
Ondrej
Yes.
> Is there a security feature to prevent this?
Not at sagenb.org right now, but it is easy to add prevention of this
by adding a firewall to the virtual server to not allow any outgoing
ports to be opened.
William
sage@sagenb:~$ mail
The program 'mail' can be found in the following packages:
* mailx
* mailutils
Ask your administrator to install one of them
-bash: mail: command not found
>
>> > Is there a security feature to prevent this?
>>
>> Not at sagenb.org right now, but it is easy to add prevention of this
>> by adding a firewall to the virtual server to not allow any outgoing
>> ports to be opened.
>
> Email gets out in many cases where you don't expect it. You have to do
> a really tight set of firewall rules which will likely make the box
> not that usable any more anyway :)
I would turn off opening of any ports at all by non-root users. Does
that make the
box unusable? Note that sendmail, etc., isn't installed systemwide.
William