new SPKG.txt format and spkg tracking in the wiki

[mabshoff]

Hi,

since I am currently testing out the latest eclib (see #1058) I am
rewriting the SPKG.txt to reflect some of the issue raised in the
"Most Sage spkg's are out of date!" thread. The SPKG.txt is now
written in wiki text and has a couple standard sections:

[begin example]
= eclib [i.e. name of spkg] =

== Description ==

John Cremona's programs for enumerating and computating with elliptic
curves defined over the rational numbers. This is the culmination of
over 30 years of hard work and careful polish.

== Maintainers ==

* William Stein
* John Cremona
* Ralph Philip Weinmann
* Michael Abshoff

== Upstream Contact ==

* Author: John Cremona
* Email: john.c...@gmail.com
* Website: http://www.warwick.ac.uk/~masgaj/mwrank/index.html

== Distribution ==

=== Padus ===
* Contact: Ismail Dönmez
* EMail: ism...@pardus.org.tr
* Website: N/A

== Changelog ==

=== eclib-20071231 (John Cremona) ===

* renamed to eclib
* allows elliptic curves as input with rational (as opposed to just
integer) coefficients.

=== cremona-20071219.p1 (Michael Abshoff) ===

* patch to fix "Internal error: can't free this _ntl_gbigint" (John
Cremona)

=== cremona-20071219.p0 (John Cremona) ===

* fix main Makefile mismerge (Michael Abshoff)
* add missing export to g0n/Makefile (John Cremona)
* fix permission issue (Michael Abshoff)

=== cremona-20071219 (John Cremona) ===

* update to latest source
* fix mwrank error on non-minimal curves (#1233)

=== cremona-20071124.p4 (Michael Abshoff) ===

* apply John Cremoan's second patch for #1403
* delete $SAGE_LOCAL/include/mwrank (#1410)
* strip the mwrank binaries and link dynamically (#1410)
* delete $SAGE_LOCAL/lib/libmwrank.[so|dylib] (#1410)

=== cremona-20071124.p3 (Michael Abshoff) ===

* apply John Cremoan's patch for #1403
* fix #1256, i.e. remove the now obsolete mwrank.spkg

=== previous versions ===

* lost to history
[end example]

I pasted the verbatim text file into http://wiki.sagemath.org/spkg/eclib

Is there anything missing? Should anything be removed?

But now there are a couple issues we need to resolve:

* How do we keep SPKG.txt and the wiki page in sync? In an ideal world
we would just copy over the updated SPKG.txt into the wiki and be done
with it. But people will edit the wiki page, i.e. to add contact info
or correct issues. One way would be for the maintainers to subscribe
to the pages of the spkgs they handle and sync it manually. Since the
wiki preserves all edits and offers an interface to diff this should
be relatively easy.

* We currently have two couple pages in the wiki that list spkgs:

http://wiki.sagemath.org/standard_packages_available_for_SAGE
http://wiki.sagemath.org/Sage_Spkg_Tracking

I think both should be merged into one page while preserving the info
from both pages. "standard_packages_available_for_SAGE" is slightly
older than "Sage_Spkg_Tracking", but I llike the format of
"Sage_Spkg_Tracking" better. It also has all current components
listed.

* We would potentially have two wiki pages for each spkg. Take for
example Givaro. We have a page at

http://wiki.sagemath.org/Givaro

That page lists some examples and compares the performance of GF(2^8)
to Magma. That information shouldn't be in SPKG.txt and the example
section could potentially be expanded.

The not yet existing page at

http://wiki.sagemath.org/spkg/Givaro

on the other hand would then contain a much more technical changelog.

I think that we also should merge both pages for each spkg into some
part of the manual. It is possible to export wiki pages to latex which
in turn then can be stuck into some part like the developer's manual.
We should do that via some script so that prior to a release somebody
can execute that script. If there is the need to do something manual
it won't happen.

Thought? Ideas?

Cheers,

Michael

[John Cremona]

Good work!

Can I change the second sentence of this text please?

> John Cremona's programs for enumerating and computating with elliptic
> curves defined over the rational numbers. This is the culmination of
> over 30 years of hard work and careful polish.

It's more like 25... and since I am still fixing bugs I find the
"careful polish" a little embarrassing (but thank you, William!)

John

On 31/12/2007, mabshoff

--
John Cremona

[mabshoff]

Hi John,

On Dec 31, 10:32 pm, "John Cremona" <john.crem...@gmail.com> wrote:
> Good work!
>
> Can I change the second sentence of this text please?
>
> > John Cremona's programs for enumerating and computating with elliptic
> > curves defined over the rational numbers. This is the culmination of
> > over 30 years of hard work and careful polish.
>
> It's more like 25... and since I am still fixing bugs I find the
> "careful polish" a little embarrassing (but thank you, William!)

:) - I changed that sentence to

John Cremona's programs for enumerating and computating with elliptic
curves defined over the rational numbers. This is the culmination of

over 25 years of hard work.

Feel free to send me an extended description if there is a better one.

> John

I am currently [finally] diffing all the changes for the cygwin build
I did out of my two Cygwin build trees. I am doing the right thing and
also fixing the make [very]clean targets and so on, so this will take
a while. I will post some patches later and also merge them into the
eclib.spkg. You can then decide if you just want to take
eclib-20071231.p0.spkg or apply the patches manually.

In addition I also discovered that on FreeBSD you really need gmake to
build the library [due to FOO ?=BLAS operators which BSD's make
doesn't understand]. But I will merge a fix for that in spkg-install
once we start to officially support FreeBSD around the 2.10.x
timeframe.

Cheers,

Michael

> On 31/12/2007, mabshoff

>
>
>
> <Michael.Absh...@fsmath.mathematik.uni-dortmund.de> wrote:
>
> > Hi,
>
> > since I am currently testing out the latest eclib (see #1058) I am
> > rewriting the SPKG.txt to reflect some of the issue raised in the
> > "Most Sage spkg's are out of date!" thread. The SPKG.txt is now
> > written in wiki text and has a couple standard sections:
>
> > [begin example]
> > = eclib [i.e. name of spkg] =
>
> > == Description ==
>
> > John Cremona's programs for enumerating and computating with elliptic
> > curves defined over the rational numbers. This is the culmination of
> > over 30 years of hard work and careful polish.
>
> > == Maintainers ==
>
> > * William Stein
> > * John Cremona
> > * Ralph Philip Weinmann
> > * Michael Abshoff
>
> > == Upstream Contact ==
>
> > * Author: John Cremona

> > * Email: john.crem...@gmail.com
> > * Website:http://www.warwick.ac.uk/~masgaj/mwrank/index.html

Since John mentioned the dependencies with Ismail in the other thread
I also think we should list dependencies in the SPKG.txt. In case of
eclib that would be:

== Dependencies ==

* gmp
* pari
* NTL

> > I pasted the verbatim text file intohttp://wiki.sagemath.org/spkg/eclib

[Ismail Dönmez]

Monday 31 December 2007 23:14:01 tarihinde mabshoff şunları yazmıştı:

> === Padus ===
>  * Contact: Ismail Dönmez
>  * EMail: ism...@pardus.org.tr

s/Padus/Pardus and thank you! I am honored!

Regards,
ismail

--
Never learn by your mistakes, if you do you may never dare to try again.

[mabshoff]

On Jan 1, 12:11 am, Ismail Dönmez <ism...@pardus.org.tr> wrote:
> Monday 31 December 2007 23:14:01 tarihinde mabshoff şunları yazmıştı:

Hi,

> > === Padus ===
> > * Contact: Ismail Dönmez
> > * EMail: ism...@pardus.org.tr
>
> s/Padus/Pardus and thank you! I am honored!

Oops, fixed. Well, you asked for a tarball to package it and so I
figured I would throw your name & distribution in there. Sooner or
later others will follow, but for now it looks like Pardus will be the
first distribution to ship Sage. If you have some URLs to Pardus
itself and/or a package specific page let me know and I will add it to
the SPKG.txt. The latest version has been updated at

http://wiki.sagemath.org/spkg/eclib

> Regards,
> ismail

Cheers,

Michael

[Ismail Dönmez]

Tuesday 01 January 2008 02:14:30 tarihinde mabshoff şunları yazmıştı:

> On Jan 1, 12:11 am, Ismail Dönmez <ism...@pardus.org.tr> wrote:
> > Monday 31 December 2007 23:14:01 tarihinde mabshoff şunları yazmıştı:
>
> Hi,
>
> > > === Padus ===
> > > * Contact: Ismail Dönmez
> > > * EMail: ism...@pardus.org.tr
> >
> > s/Padus/Pardus and thank you! I am honored!
>
> Oops, fixed. Well, you asked for a tarball to package it and so I
> figured I would throw your name & distribution in there. Sooner or
> later others will follow, but for now it looks like Pardus will be the
> first distribution to ship Sage. If you have some URLs to Pardus
> itself and/or a package specific page let me know and I will add it to
> the SPKG.txt. The latest version has been updated at

Well I just packaged python-2.5 branch for Pardus, so we are ready to rock for
Pardus 2008, soon I we'll finish all Sage deps, and actually we are waiting
for mwrank/ecllib issue to settle :-)

[didier deshommes]

2007/12/31, mabshoff <Michael...@fsmath.mathematik.uni-dortmund.de>:

> == Dependencies ==
>
> * gmp
> * pari
> * NTL
>

I feel like this is too much information required for an spkg. I think
that only the name, upstream contact and maintainers fields should be
required to distribute one. The changelog could be inferred from the
hg changelog, the distribution field does not seem to be of any value
to me as a user.

> * How do we keep SPKG.txt and the wiki page in sync? In an ideal world
> we would just copy over the updated SPKG.txt into the wiki and be done
> with it. But people will edit the wiki page, i.e. to add contact info
> or correct issues. One way would be for the maintainers to subscribe
> to the pages of the spkgs they handle and sync it manually. Since the
> wiki preserves all edits and offers an interface to diff this should
> be relatively easy.

That's a great solution.

didier

[mabshoff]

hi Didier,

On Jan 1, 2:41 am, "didier deshommes" <dfdes...@gmail.com> wrote:
> 2007/12/31, mabshoff <Michael.Absh...@fsmath.mathematik.uni-dortmund.de>:

> > == Dependencies ==
>
>
>
> > * gmp
> > * pari
> > * NTL
>
> I feel like this is too much information required for an spkg.

It is information that should change rarely and is already present in
deps, but will help out the packagers for distributions. Ismail for
example has inquired about those issues repeatedly, so writing them
down is a big plus in my opinion.

> I think
> that only the name, upstream contact and maintainers fields should be
> required to distribute one. The changelog could be inferred from the
> hg changelog,

The problem is that the hg changelog itself tells only half the story,
i.e. what changed in the base directory. But the SPKG.txt should
reflect also what changes in the src directory and those changes
should be on a very high level. The whole changelog is supposed to be
in the wiki and I don't see any convenient way to extract that
information from the hg repo in some automated way. And I do believe
that if you take the time to summarize the changes you made to some
spkg it will be better than a set of change log messages. And reading
the diffs of spkg-install isn't an option in my opinion either. One
should just spend a minimal amount of time to catch up with the
changes and reading it in some text format or the wiki is the most
convenient in my opinion.

> the distribution field does not seem to be of any value to me as a user.

Well, maybe not to you, but in case of problems with Sage packaged by
a distributor in the future it will come in handy in my opinion. If
somebody has a problem with Sage x.y.z packaged for FUBAR Linux when
we are at Sage x.y+3.z+2 I would just point that person to said link/
contact if the person doesn't want to update to Sage x,y+3,z+2. I
seriously doubt that we will ever support a "stable" release. William
has speculated that we will do that "once things settle down", but I
don't see that happening any time soon. Maintaining a stable release
for a said amount of time could happen via some commercial support
contract, but I see little value in patching some old forked code
base.

I looked at the Gentoo ticket to create an ebuild for Sage today and
they had some patch that made sure that SAGE_FORTRAN didn't have a
trailing slash or something. If I have all the URLs to the
distribution packages in one location I can easily check them for
patches. In a perfect world that wouldn't be needed since those should
flow back freely from packagers, but in reality this takes time. I
have been sitting on a lot of patches for Cygwin/FreeBSD/Solaris
recently for a variety of spkgs and I will push all of those upstream
in the 2.10 merge cycle.

Another section that should be mandatory are all changes made to the
original upstream sources, i.e. removal of documentation to shrink the
size and so on. That section should be quite detailed, preferably with
some script that does the job. That way somebody can adopt a package
quickly when the original maintainer is busy or goes missing for a
while.

> > * How do we keep SPKG.txt and the wiki page in sync? In an ideal world
> > we would just copy over the updated SPKG.txt into the wiki and be done
> > with it. But people will edit the wiki page, i.e. to add contact info
> > or correct issues. One way would be for the maintainers to subscribe
> > to the pages of the spkgs they handle and sync it manually. Since the
> > wiki preserves all edits and offers an interface to diff this should
> > be relatively easy.
>
> That's a great solution.
>
> didier

Cheers,

Michael