mixing package install with non-package installs
kilian
Thanks for doing this work. This project is extremely useful!
I was wondering what the preferred method method is for installing a
python module for which no .spg file exists yet (e.g. nose). I am not
familiar with the sage package manager, so I am not sure if installing
additional packages using 'python setup.py install' will break things.
Maybe a better way would be to create the .spg package. Do you have a
pointer how to do that and did you think of a way to share these newly
created packages?
I am really impressed that I was able to compile scipy in two
different cluster environments using spd, whereas before it used to be
a big pain...
Thanks,
Kilian
Ondrej Certik
On Sun, Jul 5, 2009 at 12:59 PM, kilian<koep...@gmail.com> wrote:
>
> Hi,
>
> Thanks for doing this work. This project is extremely useful!
Many thanks for the feedback!
>
> I was wondering what the preferred method method is for installing a
> python module for which no .spg file exists yet (e.g. nose). I am not
> familiar with the sage package manager, so I am not sure if installing
> additional packages using 'python setup.py install' will break things.
python setup.py install
will install everything in the right location and it should not break
anything. If it does, please report it.
>
> Maybe a better way would be to create the .spg package. Do you have a
> pointer how to do that and did you think of a way to share these newly
> created packages?
The best thing is to read Sage docs:
http://www.sagemath.org/doc/developer/producing_spkgs.html
If you want, just use my template here:
http://wiki.sagemath.org/OndrejCertik
(at the bottom of it)
this works for a general package. for python package, just see the
spkg-install from a python package, like sympy or ipython.
To share the new packages, I suggest that for the time being we just
add them to the downloads section here:
http://code.google.com/p/spdproject/downloads/list
then anyone can easily download it. Ping me when you have something,
I'll add you to the google code project, so that you can upload it.
>
> I am really impressed that I was able to compile scipy in two
> different cluster environments using spd, whereas before it used to be
> a big pain...
Exactly, it's a huge pain, that's one of the motivations behind SPD.
If you have other ideas how to improve SPD, let us know. I also
created femhub, which is a finite element distribution, based on SPD:
http://code.google.com/p/femhub/
just for an inspiration what can be done.
Ondrej
kilian
Thanks for the quick reply.
> > I was wondering what the preferred method method is for installing a
> > python module for which no .spg file exists yet (e.g. nose). I am not
> > familiar with the sage package manager, so I am not sure if installing
> > additional packages using 'python setup.py install' will break things.
>
> python setup.py install
>
> will install everything in the right location and it should not break
> anything. If it does, please report it.
with that option I think.
> To share the new packages, I suggest that for the time being we just
> add them to the downloads section here:
>
> http://code.google.com/p/spdproject/downloads/list
>
> then anyone can easily download it. Ping me when you have something,
> I'll add you to the google code project, so that you can upload it.
be
good to include as one of the standard packages, since it is neede for
numpy.test()
and scipy.test(). It is very reassuring that scipyt.test() runs
without errors!
> If you have other ideas how to improve SPD, let us know. I also
> created femhub, which is a finite element distribution, based on SPD:
>
> http://code.google.com/p/femhub/
One thing that I think would be great on the longer run would be to
make package description files,
just like the spkg files but without the src directory, and that
contain the url for the upstream package.
This way, people could contribute package descriptions and you could
include these (small) files in
the distribution. This is how the fink project (http://fink.sf.net)
distributes the patches necessary to
compile a variety of unix programs under OSX.
Kilian
Ondrej Certik
>
> Hi Ondrej,
>
> Thanks for the quick reply.
>
>> > I was wondering what the preferred method method is for installing a
>> > python module for which no .spg file exists yet (e.g. nose). I am not
>> > familiar with the sage package manager, so I am not sure if installing
>> > additional packages using 'python setup.py install' will break things.
>>
>> python setup.py install
>>
>> will install everything in the right location and it should not break
>> anything. If it does, please report it.
>
> I see. However, given how easy it is to make new packages, I will go
> with that option I think.
>
>> To share the new packages, I suggest that for the time being we just
>> add them to the downloads section here:
>>
>> http://code.google.com/p/spdproject/downloads/list
>>
>> then anyone can easily download it. Ping me when you have something,
>> I'll add you to the google code project, so that you can upload it.
>
> I created a package for nose (nose-0.11.1.p0.spkg) which I think would
> be
> good to include as one of the standard packages, since it is neede for
> numpy.test()
> and scipy.test(). It is very reassuring that scipyt.test() runs
> without errors!
Excellent, I have added you to the project, so just upload your spkg
package into the Downloads (hit new download and it should work).
>
>> If you have other ideas how to improve SPD, let us know. I also
>> created femhub, which is a finite element distribution, based on SPD:
>>
>> http://code.google.com/p/femhub/
>
> That looks great!
>
> One thing that I think would be great on the longer run would be to
> make package description files,
> just like the spkg files but without the src directory, and that
> contain the url for the upstream package.
> This way, people could contribute package descriptions and you could
> include these (small) files in
> the distribution. This is how the fink project (http://fink.sf.net)
> distributes the patches necessary to
> compile a variety of unix programs under OSX.
Interesting --- so you think that SPD itself should include most of
those small files for most of packages and it would now how to
download the src directory, thus installing itself?
I think we would have to keep it updated all the time, isn't it easier
to just point to the real spkg packages, that are known to work? E.g.
create a pool of supported spkg packages and then just do something
like:
./spd -i nose
and it would try to download the latest spkg package. In fact, this
is how Sage works already (it tries to download the package form
sagemath.org).
Ondrej
kilian
On Jul 5, 6:28 pm, Ondrej Certik <ond...@certik.cz> wrote:
>
> Excellent, I have added you to the project, so just upload your spkg
> package into the Downloads (hit new download and it should work).
>
> > One thing that I think would be great on the longer run would be to
> > make package description files,
> > just like the spkg files but without the src directory, and that
> > contain the url for the upstream package.
> > This way, people could contribute package descriptions and you could
> > include these (small) files in
> > the distribution. This is how the fink project (http://fink.sf.net)
> > distributes the patches necessary to
> > compile a variety of unix programs under OSX.
>
> Interesting --- so you think that SPD itself should include most of
> those small files for most of packages and it would now how to
> download the src directory, thus installing itself?
>
> I think we would have to keep it updated all the time, isn't it easier
> to just point to the real spkg packages, that are known to work? E.g.
> create a pool of supported spkg packages and then just do something
> like:
>
> ./spd -i nose
>
> and it would try to download the latest spkg package. In fact, this
> is how Sage works already (it tries to download the package form
> sagemath.org).
would be that it would be easier to see, what the package actually
does
and therefore, it would be easier to monitor many packages from many
contributors. And it would be easier to put a tree of these
descriptions under
source control, maybe even two trees: testing and stable. You could
still
keep a copy of the original source tar file as a fall-back solution.
Each package description file would contain the url of the original
source,
its MD5 sum, a patch against this source code and maybe the email
address
of the package maintainer,
On the other hand, the fact that the new format would deviate from the
sage
format would be a clear drawback. Maybe, we should have this
discussion
rather on the sage email list...
Kilian
Ondrej Certik
Let's discuss this on the mailinglist from now on.
Ondrej
William Stein
>
> On Mon, Jul 6, 2009 at 8:14 AM, David Joyner<wdjo...@gmail.com> wrote:
>>
>> On Mon, Jul 6, 2009 at 2:10 AM, Ondrej Certik<ond...@certik.cz> wrote:
>>>
>>
>> ...
>>
>>>>>
>>>>> On the other hand, the fact that the new format would deviate from the
>>>>> sage
>>>>> format would be a clear drawback. Maybe, we should have this
>>>>> discussion
>>>>> rather on the sage email list...
>>>>
>>>> Let's discuss this on the mailinglist from now on.
>>>
>>
>>
>> My 2 cents: I'm confused what the proposal is exactly. Also, I'm concerned that
>> the thread will be hard to follow for some on sage-devel who are not on
>> spd-dev. Maybe a separate thread on sage-devel with your question might
>> be clearer?
>
> That's right, Kilian, do you think you could please start such a thread?
>
My quick thoughts are that:
(1) the proposal isn't sufficiently clear, and
(2) there are a lot of Sage spkg's already, and trying to change
everything is a great deal of error-prone work, unless it is
automated, and
(3) does the proposal do anything that the existing SPKG.txt files
don't do already?
William