sympow

14 views
Skip to first unread message

John Cremona

unread,
Aug 16, 2010, 6:51:05 AM8/16/10
to SAGE devel, sage-nt
The spkg sympow (Mark Watkins's C library for computing symmetric
power L-functions, which applications) is causing more and more
problems (see #9705 for example) but it provides functionality which
is unavailable elsewhere in Sage. I have only used it for modular
degrees of elliptic curves over Q, but it does other stuff. I suspect
that some of that could be provided via lcalc (C library) and/or
dokchitser's gp scripts.

How realistic do people think it to rewrite what's needed?

And (this question directed at the others of you who know him) is it
worth it to try to get Mark Watkins to be more cooperative? Or is
that hopeless while he remains at Magma, at least?

John

Dr. David Kirkby

unread,
Aug 16, 2010, 8:38:20 AM8/16/10
to sage-...@googlegroups.com
On 08/16/10 11:51 AM, John Cremona wrote:
> The spkg sympow (Mark Watkins's C library for computing symmetric
> power L-functions, which applications) is causing more and more
> problems (see #9705 for example)

John,

I think you mean #9703 - #9705 looks unrelated to SYMPOW to me.

#9166 (Cygwin related) is another relevant ticket, though there's probably more
information on #9703.

Dave

John Cremona

unread,
Aug 16, 2010, 8:55:42 AM8/16/10
to sage-...@googlegroups.com
On 16 August 2010 13:38, Dr. David Kirkby <david....@onetel.net> wrote:
> On 08/16/10 11:51 AM, John Cremona wrote:
>>
>> The spkg sympow (Mark Watkins's C library for computing symmetric
>> power L-functions, which applications) is causing more and more
>> problems (see #9705 for example)
>
> John,
>
> I think you mean #9703 - #9705 looks unrelated to SYMPOW to me.

Correct. (Eyesight not what it was...)

John

>
> #9166 (Cygwin related) is another relevant ticket, though there's probably
> more information on #9703.
>
> Dave
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to
> sage-devel+...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

Bill Hart

unread,
Aug 16, 2010, 5:32:14 PM8/16/10
to sage-devel
Is there even a link from which Sympow can be downloaded so that one
can look through the code to see if it is worth salvaging. I can't
even find a webpage for Mark Watkins at the moment, let alone sympow.

I honestly suspect that Mark has just moved on to other projects. I
doubt very much if his work on Magma precludes him from maintaining
this package. I've personally worked with him on projects since he
joined the Magma group and we both made use of each other's code
without even asking the question.

But, we should be more selective when including packages. IMHO, from
the start, sympow should have been merged to become part of a larger
library of functionality, though to be honest, I'm not quite sure
where it exactly belongs. I don't think it belongs in flint as such.
Maybe Pari?

I'm also really wondering why cephes is in Sage. Unless there are two
packages with the same name, it was last updated circa 1994 and uses
non-portable long doubles. We've booted packages from Sage for much
less felonious offences.

Is there a modularity problem here? I always thought that given Sage
sees itself as more of a distribution (like linux) than a single
project, then all but a very small number of core libs should be able
to be added/removed from Sage at the flick of a switch. It sure
would've made a Windows port easier! Is it too late to fix the spkg
system to be more modular. Is there even an efficient method in Python
of conditionally including code based on availability of prereqs? I
know this exists at some levels, for example GMP vs MPIR or NTL vs
FLINT or zn_poly vs FLINT vs Pari for Z/nZ[x], etc. But this only
seems to be where one library can substitute for another, which is
rare. Could this be extended and still efficient? For example I don't
think you want to be checking in Python whether you have GMP loaded
every time you go to add two integers together. But then maybe with
the possibility of a choice between MPIR and GMP at the spkg level,
GMP/MPIR should be viewed as essential to Sage and thus a core lib?
Maybe ALL the libraries in Sage are considered core and only those
packages which are optional spkgs are considered truly.... optional.
If so, I think the core is potentially unwieldy and may be
contributing to some of the maintenance issues.....

Bill.

On 16 Aug, 13:55, John Cremona <john.crem...@gmail.com> wrote:

William Stein

unread,
Aug 16, 2010, 5:50:57 PM8/16/10
to sage-devel
On Mon, Aug 16, 2010 at 2:32 PM, Bill Hart <goodwi...@googlemail.com> wrote:
> Is there even a link from which Sympow can be downloaded so that one
> can look through the code to see if it is worth salvaging. I can't
> even find a webpage for Mark Watkins at the moment, let alone sympow.
>
> I honestly suspect that Mark has just moved on to other projects. I
> doubt very much if his work on Magma precludes him from maintaining
> this package. I've personally worked with him on projects since he
> joined the Magma group and we both made use of each other's code
> without even asking the question.

The above is just speculation without even asking him.

> But, we should be more selective when including packages.

No new packages ever again :-)

> IMHO, from
> the start, sympow should have been merged to become part of a larger
> library of functionality, though to be honest, I'm not quite sure
> where it exactly belongs. I don't think it belongs in flint as such.
> Maybe Pari?

And whose going to do this work? You?

> I'm also really wondering why cephes is in Sage. Unless there are two
> packages with the same name, it was last updated circa 1994 and uses
> non-portable long doubles. We've booted packages from Sage for much
> less felonious offences.

Cephes is in Sage because it's absolutely critical for the Windows port.

> Is there a modularity problem here? I always thought that given Sage
> sees itself as more of a distribution (like linux) than a single
> project,

Sage is not more of a distribution than a single project. Sage is:

1. A distribution
2. Unified interfaces to math software
3. A new library and math software system,

with the single guiding goal to "create a viable free open source
alternative to Magma, Maple, Mathematica, and MATLAB."

> then all but a very small number of core libs should be able
> to be added/removed from Sage at the flick of a switch.

This is not even true of Linux distributions. Even the most minimal
ubuntu install still has tons of packages... or at least a lot more
then "a very small number".

> It sure would've made a Windows port easier!

Programs that aren't Sage are easier to port to Windows :-).
You might try porting SPD to Windows instead, which would
be easier. http://github.com/certik/spd

> Is it too late to fix the spkg
> system to be more modular. Is there even an efficient method in Python
> of conditionally including code based on availability of prereqs? I
> know this exists at some levels, for example GMP vs MPIR or NTL vs
> FLINT or zn_poly vs FLINT vs Pari for Z/nZ[x], etc. But this only
> seems to be where one library can substitute for another, which is
> rare. Could this be extended and still efficient? For example I don't
> think you want to be checking in Python whether you have GMP loaded
> every time you go to add two integers together. But then maybe with
> the possibility of a choice between MPIR and GMP at the spkg level,
> GMP/MPIR should be viewed as essential to Sage and thus a core lib?
> Maybe ALL the libraries in Sage are considered core and only those
> packages which are optional spkgs are considered truly.... optional.

Correct.

> If so, I think the core is potentially unwieldy and

It's not unwieldy, but it is difficult to wield.

> may be
> contributing to some of the maintenance issues.....

Sage is definitely contributing to the maintenance issues of Sage.

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

François Bissey

unread,
Aug 16, 2010, 6:43:45 PM8/16/10
to sage-...@googlegroups.com
> Is there even a link from which Sympow can be downloaded so that one
> can look through the code to see if it is worth salvaging. I can't
> even find a webpage for Mark Watkins at the moment, let alone sympow.
I asked that question on the 17th of February on this very same mailing list.
No one could find a homepage for sympow at the time. There is a blurb about
Mark on the University of Sydney's web page.

sympow is a runtime dependency of "sage", that is you don't need it to build
sage itself. You only need it if you want the functionality.

Basically if it doesn't link to the sage extension library it almost always
can be installed later. Off those that can be installed afterwards a select few
will result in sage missing essential functionality or not working altogether
(maxima comes to mind). We have a precise (we hope) list of these in sage-on-
gentoo.

If you weren't using sympow you won't be missing it and nothing would break.
Nothing apart from lfunctions/sympow.py calls sympow and I don't think
anything in that python file is called from anywhere else - it just provides
the interface to talk to sympow. In short sympow could be made optional
overnight by dropping it from the deps file.

Francois

Bill Hart

unread,
Aug 16, 2010, 6:48:25 PM8/16/10
to sage-devel


On 16 Aug, 22:50, William Stein <wst...@gmail.com> wrote:
> On Mon, Aug 16, 2010 at 2:32 PM, Bill Hart <goodwillh...@googlemail.com> wrote:
> > Is there even a link from which Sympow can be downloaded so that one
> > can look through the code to see if it is worth salvaging. I can't
> > even find a webpage for Mark Watkins at the moment, let alone sympow.
>
> > I honestly suspect that Mark has just moved on to other projects. I
> > doubt very much if his work on Magma precludes him from maintaining
> > this package. I've personally worked with him on projects since he
> > joined the Magma group and we both made use of each other's code
> > without even asking the question.
>
> The above is just speculation without even asking him.

I'll email him, as I speak to him quasi-regularly by email. And I know
for a fact he has been working on other things. I personally worked on
them with him, and he discussed other projects he's done too. I don't
think I speculated in any of the three sentences here. But I will
explicitly ask him about the status of sympow.

>
> > But, we should be more selective when including packages.
>
> No new packages ever again :-)

Is that the official policy or were you joking. I actually don't know
what the current policy is, though I think I saw a not too old version
of it some time not too long ago, which significantly toughened the
restrictions for new packages.

>
> >  IMHO, from
> > the start, sympow should have been merged to become part of a larger
> > library of functionality, though to be honest, I'm not quite sure
> > where it exactly belongs. I don't think it belongs in flint as such.
> > Maybe Pari?
>
> And whose going to do this work? You?

John's original question was about whether a rewrite of the particular
functionality we want from sympow was warranted. Isn't this rather the
point of the thread?

And no, I am not personally planning to rewrite sympow. I have never
used it and it isn't really important for me at the present time. I
might become more interested over the next few years, but I'm not
currently interested.

>
> > I'm also really wondering why cephes is in Sage. Unless there are two
> > packages with the same name, it was last updated circa 1994 and uses
> > non-portable long doubles. We've booted packages from Sage for much
> > less felonious offences.
>
> Cephes is in Sage because it's absolutely critical for the Windows port.

You mean you used it in the Cygwin port?

Doesn't the inclusion of an old package like this with no maintainer
potentially cause more maintenance problems for people down the line.
Isn't this the lesson we are learning from sympow? Or am I missing
that it now has a more recent version maintained by someone and it no
longer uses long doubles.

>
> > Is there a modularity problem here? I always thought that given Sage
> > sees itself as more of a distribution (like linux) than a single
> > project,
>
> Sage is not more of a distribution than a single project.  Sage is:
>
>   1. A distribution
>   2. Unified interfaces to math software
>   3. A new library and math software system,
>
> with the single guiding goal to "create a viable free open source
> alternative to Magma, Maple, Mathematica, and MATLAB."

Ok.

>
> >  then all but a very small number of core libs should be able
> > to be added/removed from Sage at the flick of a switch.
>
> This is not even true of Linux distributions.    Even the most minimal
> ubuntu install still has tons of packages... or at least a lot more
> then "a very small number".

But I think packages in a linux distribution have prerequisites, and
if you install one, the prereqs get installed too (e.g. using apt-
get). I'm not seeing that happening with the sage spkgs. And I don't
know if that is possible or technically feasible. But I am asking the
question.

>
> > It sure would've made a Windows port easier!
>
> Programs that aren't Sage are easier to port to Windows :-).
> You might try porting SPD to Windows instead, which would
> be easier.http://github.com/certik/spd

The primary difficulty in porting Sage is in porting the C libraries
Sage relies on. SPD probably just doesn't include them.

Looking at the project now.......

....OK. I see, it is based on Sage, but has a more modular install
system, just like what I've been advocating, and the base install is
60mb and builds in 10 minutes. But unless I am mistaken, unlike Sage
it is geared more towards scientific computation than just general
mathematics. Good idea, but it misses on an important criteria for me.
Having said that, it might be a good place to *start* on a true Sage
port to Windows. At least it excises a functional component of Sage,
though IMHO it would be better if more such "components" were
"excised", which is what I guess I was advocating, i.e. modularity.

Hey someone "forked" Sage. Quick, call the lawyers! This is
outrageous. Those f**kers!!

>
> > Is it too late to fix the spkg
> > system to be more modular. Is there even an efficient method in Python
> > of conditionally including code based on availability of prereqs? I
> > know this exists at some levels, for example GMP vs MPIR or NTL vs
> > FLINT or zn_poly vs FLINT vs Pari for Z/nZ[x], etc. But this only
> > seems to be where one library can substitute for another, which is
> > rare. Could this be extended and still efficient? For example I don't
> > think you want to be checking in Python whether you have GMP loaded
> > every time you go to add two integers together. But then maybe with
> > the possibility of a choice between MPIR and GMP at the spkg level,
> > GMP/MPIR should be viewed as essential to Sage and thus a core lib?
> > Maybe ALL the libraries in Sage are considered core and only those
> > packages which are optional spkgs are considered truly.... optional.
>
> Correct.

I take it this refers to the last sentence, meaning that a minimal
Sage will always be all of Sage standard. IMHO this makes it harder
for Sage to grow in certain ways....

>
> > If so, I think the core is potentially unwieldy and
>
> It's not unwieldy, but it is difficult to wield.
>
> > may be
> > contributing to some of the maintenance issues.....
>
> Sage is definitely contributing to the maintenance issues of Sage.
>

I mean the lack of modularity is contributing. But I've made this
point before and it all just got sent to sage-flame. So I won't
continue making it here. Back to what I was doing....

Bill Hart

unread,
Aug 16, 2010, 6:55:44 PM8/16/10
to sage-devel
Hi Francois,

On 16 Aug, 23:43, François Bissey <f.r.bis...@massey.ac.nz> wrote:
> > Is there even a link from which Sympow can be downloaded so that one
> > can look through the code to see if it is worth salvaging. I can't
> > even find a webpage for Mark Watkins at the moment, let alone sympow.
>
> I asked that question on the 17th of February on this very same mailing list.
> No one could find a homepage for sympow at the time. There is a blurb about
> Mark on the University of Sydney's web page.
>
> sympow is a runtime dependency of "sage", that is you don't need it to build
> sage itself. You only need it if you want the functionality.

Thanks for that explanation!

>
> Basically if it doesn't link to the sage extension library it almost always
> can be installed later. Off those that can be installed afterwards a select few
> will result in sage missing essential functionality or not working altogether
> (maxima comes to mind). We have a precise (we hope) list of these in sage-on-
> gentoo.

Where do I find "sage-on-gentoo"? And where would I find this list?

>
> If you weren't using sympow you won't be missing it and nothing would break.
> Nothing apart from lfunctions/sympow.py calls sympow and I don't think
> anything in that python file is called from anywhere else - it just provides
> the interface to talk to sympow. In short sympow could be made optional
> overnight by dropping it from the deps file.
>
> Francois

OK, I was understanding it was used by modular forms and elliptic
curves stuff for computing modular degrees or something, and so
imagined it hopelessly intertwined with those. Also, I think this
issue was brought up elsewhere and William said categorically he
needed it and didn't want it removed from Sage. But I might've gotten
that mixed up. I read so many threads they all become a blur after a
while.

I suppose if *someone* needs it, then it still has to be maintained. I
don't personally need it, or I would offer to maintain it. But the day
may soon come...

Bill.

François Bissey

unread,
Aug 16, 2010, 7:05:05 PM8/16/10
to sage-...@googlegroups.com
> Where do I find "sage-on-gentoo"? And where would I find this list?
http://github.com/cschwan/sage-on-gentoo

As for the lists I am copying then here (beware Gentoo-speak):
*buildtime dependencies:
DEPEND="|| ( =dev-lang/python-2.6.4-r99
=dev-lang/python-2.6.5-r99 )
dev-libs/gmp
>=dev-libs/mpfr-2.4.2
>=dev-libs/ntl-5.5.2
>=dev-lisp/ecls-10.2.1[-unicode]
=dev-python/cython-0.12*
~dev-python/numpy-1.3.0[lapack]
>=sci-mathematics/eclib-20080310_p10
>=sci-mathematics/ecm-6.2.1
>=sci-libs/flint-1.5.0[ntl]
>=sci-libs/fplll-3.0.12
=sci-libs/givaro-3.2*
>=sci-libs/gsl-1.10
>=sci-libs/iml-1.0.1
>=sci-libs/libcliquer-1.2.5
>=sci-libs/linbox-1.1.6[ntl,sage]
>=sci-libs/m4ri-20100221
>=sci-libs/mpfi-1.4
>=sci-libs/pynac-0.2.0_p4
>=sci-libs/symmetrica-2.0
>=sci-libs/zn_poly-0.9
>=sci-mathematics/glpk-4.43[gmp]
>=sci-mathematics/lcalc-1.23[pari]
>=sci-mathematics/pari-2.3.5[data,gmp]
>=sci-mathematics/polybori-0.6.4[sage]
>=sci-mathematics/ratpoints-2.1.3
~sci-mathematics/sage-baselayout-${PV}[testsuite=]
~sci-mathematics/sage-clib-${PV}
!!sci-mathematics/sage-core
~sci-mathematics/singular-3.1.1.4[libsingular,sage]
media-libs/gd
media-libs/libpng
>=sys-libs/readline-6.0
sys-libs/zlib
virtual/cblas"
===========
*runtime (some overlap):
RDEPEND="
>=dev-lang/R-2.10.1[lapack,readline]
>=dev-python/cvxopt-0.9
>=dev-python/gdmodule-0.56
>=dev-python/ipython-0.9.1
>=dev-python/jinja-2.1.1
>=dev-python/matplotlib-0.99.3
~dev-python/mpmath-0.15
~dev-python/networkx-1.0.1
~dev-python/pexpect-2.0
>=dev-python/pycrypto-2.0.1
>=dev-python/python-gnutls-1.1.4
>=dev-python/rpy-2.0.6
>=dev-python/sphinx-0.6.3
~dev-python/sympy-0.6.6
>=media-gfx/tachyon-0.98
>=net-zope/zodb-3.7.0
>=sci-libs/cddlib-094f
=sci-libs/scipy-0.7*
>=sci-mathematics/flintqs-20070817_p4
>=sci-mathematics/gap-4.4.12
>=sci-mathematics/gap-guava-3.4
>=sci-mathematics/genus2reduction-0.3
>=sci-mathematics/gfan-0.4
>=sci-mathematics/cu2-20060223
>=sci-mathematics/cubex-20060128
>=sci-mathematics/dikcube-20070912_p12
~sci-mathematics/maxima-5.20.1[ecl]
>=sci-mathematics/mcube-20051209
>=sci-mathematics/optimal-20040603
>=sci-mathematics/palp-1.1
~sci-mathematics/sage-data-conway_polynomials-0.2
~sci-mathematics/sage-data-elliptic_curves-0.1
~sci-mathematics/sage-data-graphs-20070722_p1
~sci-mathematics/sage-data-polytopes_db-20100210
sci-mathematics/sage-doc
~sci-mathematics/sage-extcode-${PV}
~sci-mathematics/sage-notebook-0.8.1
>=sci-mathematics/sympow-1.018
examples? ( ~sci-mathematics/sage-examples-${PV} )
latex? ( ~sci-mathematics/sage-latex-2.2.5 )
testsuite? (
~sci-mathematics/sage-doc-${PV}[html]
~sci-mathematics/sage-examples-${PV}
)"
================
list is for sage-4.5.2 - we have already switched to singular-3.1.1.4.

Francois

Dr. David Kirkby

unread,
Aug 16, 2010, 7:13:15 PM8/16/10
to sage-...@googlegroups.com
On 08/16/10 11:48 PM, Bill Hart wrote:

>> Cephes is in Sage because it's absolutely critical for the Windows port.
>
> You mean you used it in the Cygwin port?

I suspect William does. Like you, I don't consider that a native Windows port.
But it might be the closest we get. (We know we differ on this topic. It's been
debated elsewhere, so lets forget that for now).

But Cephes is also needed for FreeBSD port too.

> Doesn't the inclusion of an old package like this with no maintainer
> potentially cause more maintenance problems for people down the line.

Agreed, though in the case of cephes, the basic functionality is somewhat easier
to understand. If a part of it is not working, there's a greater chance of a
Sage developer being able to fix it.

There's also more incentive for someone to fix it. It is used in the Cygwin port
and its used in the FreeBSD port. It is needed for those ports to work at all.

In contrast, it seems sympow is not used by many people. The fact there are no
bug reports of it not working on the public servers shows how little it is used.
(If someone actually tried to run many of the examples, they will find they
don't work). It has however been fixed in the Gentoo port.

Dave

Bill Hart

unread,
Aug 16, 2010, 7:19:03 PM8/16/10
to sage-devel
Ah, that's a very useful list, thank you.

What makes the difference between sci-libs and sci-mathematics? This
saves me some work actually, as I was wondering what the dependencies
were and which ones were scientific/mathematical and which weren't.

Bill.

William Stein

unread,
Aug 16, 2010, 7:24:24 PM8/16/10
to sage-devel
> I'll email him, as I speak to him quasi-regularly by email. And I know
> for a fact he has been working on other things. I personally worked on
> them with him, and he discussed other projects he's done too. I don't
> think I speculated in any of the three sentences here. But I will
> explicitly ask him about the status of sympow.

Thank you.

>
>>
>> > But, we should be more selective when including packages.
>>
>> No new packages ever again :-)
>
> Is that the official policy or were you joking.

The smiley face was supposed to indicate that I was joking.

> I actually don't know
> what the current policy is, though I think I saw a not too old version
> of it some time not too long ago, which significantly toughened the
> restrictions for new packages.

We've had a documented policy for a few years now about how to get new
packages into Sage. I think it hasn't changed since we made it.
It's basically:

(1) vote -- with many criterion for what should be taken into
account when voting,
(2) package has to be optional for a while no matter what
(3) maintainer for at least a while

Python itself also has a policy for getting new packages into Python,
which is much more stringent.

>>
>> > I'm also really wondering why cephes is in Sage. Unless there are two
>> > packages with the same name, it was last updated circa 1994 and uses
>> > non-portable long doubles. We've booted packages from Sage for much
>> > less felonious offences.
>>
>> Cephes is in Sage because it's absolutely critical for the Windows port.
>
> You mean you used it in the Cygwin port?

Yes.

> Doesn't the inclusion of an old package like this with no maintainer
> potentially cause more maintenance problems for people down the line.
> Isn't this the lesson we are learning from sympow? Or am I missing
> that it now has a more recent version maintained by someone and it no
> longer uses long doubles.

I would rather have a windows port than not having a Windows port.
Problems may be created as a result. But since Cephes is only used
on Windows, the problems will be less than the current problem which
is that Sage doesn't work at all on Windows.

>
>>
>> > Is there a modularity problem here? I always thought that given Sage
>> > sees itself as more of a distribution (like linux) than a single
>> > project,
>>
>> Sage is not more of a distribution than a single project.  Sage is:
>>

>> >  then all but a very small number of core libs should be able
>> > to be added/removed from Sage at the flick of a switch.
>>
>> This is not even true of Linux distributions.    Even the most minimal
>> ubuntu install still has tons of packages... or at least a lot more
>> then "a very small number".
>
> But I think packages in a linux distribution have prerequisites, and
> if you install one, the prereqs get installed too (e.g. using apt-
> get). I'm not seeing that happening with the sage spkgs. And I don't
> know if that is possible or technically feasible. But I am asking the
> question.

There is a dependency system in place now for standard packages. It
is defined by spkg/standard/deps

>
>>
>> > It sure would've made a Windows port easier!
>>
>> Programs that aren't Sage are easier to port to Windows :-).
>> You might try porting SPD to Windows instead, which would
>> be easier.http://github.com/certik/spd
>
> The primary difficulty in porting Sage is in porting the C libraries
> Sage relies on.

Probably. Eliminating all the use of pexpect from Sage will also be
a massive undertaking, which will likely be even more difficult than
porting the C libraries Sage depends upon. It's probably at least
possible though.

> SPD probably just doesn't include them.

You're right, SPD doesn't.

>> > packages which are optional spkgs are considered truly.... optional.
>>
>> Correct.
>
> I take it this refers to the last sentence, meaning that a minimal
> Sage will always be all of Sage standard. IMHO this makes it harder
> for Sage to grow in certain ways....

That's correct.

However, there could be a "Sage lite", which would be a lot smaller.
There has been very little interest by people in the Sage community in
actually working on this though. 99% of people who work on Sage, work
on Sage because they actually want Sage to do something useful for
their research or teaching. Working on a Sage lite does not
contribute at all to this motivation.
Outside the Sage community there is interest, and of course the result
is SPD (and the like), which has little to do with Sage.

>> > If so, I think the core is potentially unwieldy and
>>
>> It's not unwieldy, but it is difficult to wield.
>>
>> > may be
>> > contributing to some of the maintenance issues.....
>>
>> Sage is definitely contributing to the maintenance issues of Sage.
>>
>
> I mean the lack of modularity is contributing. But I've made this
> point before and it all just got sent to sage-flame. So I won't
> continue making it here. Back to what I was doing....

Sage is a lot more modular than Magma.

William

Dr. David Kirkby

unread,
Aug 16, 2010, 7:25:56 PM8/16/10
to sage-...@googlegroups.com
On 08/16/10 11:43 PM, Fran�ois Bissey wrote:

> If you weren't using sympow you won't be missing it and nothing would break.
> Nothing apart from lfunctions/sympow.py calls sympow and I don't think
> anything in that python file is called from anywhere else - it just provides
> the interface to talk to sympow.

Unfortunately, with sympow not working, several doctests fail.

I stated in

http://trac.sagemath.org/sage_trac/ticket/9703

that it was

devel/sage/sage/lfunctions/sympow.py

but in fact, these all fail with a message about sympow.

devel/sage/sage/modular/abvar/abvar.py
devel/sage/sage/modular/hecke/submodule.py
devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py

> In short sympow could be made optional
> overnight by dropping it from the deps file.

I think it needs a bit more than that.

Many people, myself included, feel optional should mean it works on all
supported platforms. In the case of sympow, it is not working on two platforms
that should soon be supported.

I'd like to see it moved to experimental myself. It would save a lot of headaches!

> Francois

Dave

William Stein

unread,
Aug 16, 2010, 7:40:21 PM8/16/10
to sage-devel, sage-windows
On Mon, Aug 16, 2010 at 4:13 PM, Dr. David Kirkby
<david....@onetel.net> wrote:
> On 08/16/10 11:48 PM, Bill Hart wrote:
>
>>> Cephes is in Sage because it's absolutely critical for the Windows port.
>>
>> You mean you used it in the Cygwin port?
>
> I suspect William does. Like you, I don't consider that a native Windows
> port. But it might be the closest we get. (We know we differ on this topic.
> It's been debated elsewhere, so lets forget that for now).

"Native" is not black and white. By reading the docs at
http://www.cygwin.com one can I hope understand what Cygwin is: "a DLL
(cygwin1.dll) that acts as an emulation layer providing substantial
POSIX (Portable Operating System Interface) system call functionality,
and a collection of tools, which provide a Linux look and feel". If
you were to rewrite everything in Sage that makes POSIX calls by
providing your own implementation of exactly those calls (say by
making calls into the Win32 API), then you would get something that
has little right to be called more "native" than Sage built using
Cygwin. It would be equivalently native, but likely be far less
modular (and far more buggy and slower) than Sage built on Cygwin.
Instead, to get something that is arguably more native, you'll have to
rewrite the Sage library in a completely different way, starting with
getting rid of all use of fork and pseduotty's (which are used by
pexpect).

> In contrast, it seems sympow is not used by many people. The fact there are
> no bug reports of it not working on the public servers shows how little it
> is used. (If someone actually tried to run many of the examples, they will
> find they don't work).

There is 1 "example" -- the modular degree function -- that is 1000
times more important than the rest. Here's me *using* sympow:

sage: E = EllipticCurve('5077a'); E
Elliptic Curve defined by y^2 + y = x^3 - 7*x + 6 over Rational Field
sage: E.modular_degree()
1984

This modular_degree function is the only function in sympow that
people are likely to care about. But it is not just a tiny trivial
function that uses only 1% of sympow (it's a major piece of
functionality).

--William

John H Palmieri

unread,
Aug 16, 2010, 7:56:41 PM8/16/10
to sage-devel
On Aug 16, 4:40 pm, William Stein <wst...@gmail.com> wrote:
> On Mon, Aug 16, 2010 at 4:13 PM, Dr. David Kirkby
> > In contrast, it seems sympow is not used by many people. The fact there are
> > no bug reports of it not working on the public servers shows how little it
> > is used. (If someone actually tried to run many of the examples, they will
> > find they don't work).
>
> There is 1 "example" -- the modular degree function --  that is 1000
> times more important than the rest.    Here's me *using* sympow:
>
> sage: E = EllipticCurve('5077a'); E
> Elliptic Curve defined by y^2 + y = x^3 - 7*x + 6 over Rational Field
> sage: E.modular_degree()
> 1984
>
> This modular_degree function is the only function in sympow that
> people are likely to care about.  But it is not just a tiny trivial
> function that uses only 1% of sympow (it's a major piece of
> functionality).

As far as I can tell, this is exactly the kind of thing that's broken
on Solaris on x86. See

<http://trac.sagemath.org/sage_trac/ticket/9703#comment:5>

--
John
Message has been deleted

William Stein

unread,
Aug 16, 2010, 8:31:55 PM8/16/10
to sage-devel
On Mon, Aug 16, 2010 at 5:08 PM, mda_ <donmo...@gmail.com> wrote:
>> system to be more modular. Is there even an efficient method in Python
>> of conditionally including code based on availability of prereqs? I

try:
import foo
except ImportError:
# do something to note that the module foo isn't available

>
> Since I'm a sage-noob, I don't know if my answer is relevant
> enough....However, the import statement in python, in general, has
> been misused in the past.
>
> An excerpt from "What's new in Python 3.1": http://docs.python.org/dev/py3k/whatsnew/3.1.html
>
> A new module, importlib was added. It provides a complete, portable,
> pure Python reference implementation of the import statement and its
> counterpart, the __import__() function. It represents a substantial
> step forward in documenting and defining the actions that take place
> during imports.
>
> (Contributed by Brett Cannon.)


>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

--

Dr. David Kirkby

unread,
Aug 16, 2010, 8:31:23 PM8/16/10
to sage-...@googlegroups.com
On 08/17/10 12:24 AM, William Stein wrote:

> I would rather have a windows port than not having a Windows port.
> Problems may be created as a result. But since Cephes is only used
> on Windows, the problems will be less than the current problem which
> is that Sage doesn't work at all on Windows.

I think Peter Jeremy is intending using Cephes on FreeBSD. I think there will be
more people willing and able to debug Cephes than SYMPOW.

To maintain SYMPOW, you need a skill set very few people process.

Dave

Bill Hart

unread,
Aug 16, 2010, 8:59:32 PM8/16/10
to sage-devel
Hmm, I see the argument between the researcher (William) who wants to
*use* sympow, and the code porter (David) who wants to *port* sympow.

I think I will remain on the fence on this one, pull up a chair, grab
a beer from the fridge..... :-)

Bill Hart

unread,
Aug 16, 2010, 9:12:22 PM8/16/10
to sage-devel
Hi Francois,

I looked through the list, and here are some spkg's that don't seem to
be on either list (as deps or deps of deps). Some I can see why you
didn't list, e.g. f95, mercurial, but many others seem to be part of
Sage and not listed. I guess sqlite is a dependency of python....

Here is a list of ones which are definitely spkgs in Sage but not on
the list:

boost
cephes
docutils
f2c
freetype
boehm-gc
gnutls
libgcrypt
libgpg_error
mercurial
opencdk
pil
sage-scripts
setuptools
sqlalchemy
sqlite
termcap

Here is a list of ones that are listed as being part of Sage on the
sagemath website (though I can't find spkgs - probably didn't look
hard enough)

bzip2
g95
mwrank

The following are not spkgs, not listed on the sagemath website, but
were listed elsewhere, for reasons unknown (possibly they used to be
in Sage but are no longer - or they might have been considered
important for a potential Windows port or something or are part of
sage but not spkgs).

dir
openopt
prereq

Bill.

On 17 Aug, 00:05, François Bissey <f.r.bis...@massey.ac.nz> wrote:

Bill Hart

unread,
Aug 16, 2010, 9:13:22 PM8/16/10
to sage-devel
Ah, and cephes is only required for Cygwin and FreeBSD, so that
explains that one....

Dr. David Kirkby

unread,
Aug 16, 2010, 9:26:49 PM8/16/10
to sage-...@googlegroups.com
On 08/17/10 01:59 AM, Bill Hart wrote:
> Hmm, I see the argument between the researcher (William) who wants to
> *use* sympow, and the code porter (David) who wants to *port* sympow.
>
> I think I will remain on the fence on this one, pull up a chair, grab
> a beer from the fridge..... :-)

Bill,

If it was a case of porting sympow, I would just get on and do it. I've got
countless other packages working on Solaris by sorting out portability issues.

But SYMPOW's problems extend far beyond portability issues. That's just the tip
of the iceberg. Take a look at the source code, and you will see what I mean.

Dave

Bill Hart

unread,
Aug 16, 2010, 9:29:53 PM8/16/10
to sage-devel
Where? I can't even download it, so I don't even get past go.

Bill Hart

unread,
Aug 16, 2010, 9:32:03 PM8/16/10
to sage-devel
OK, mwrank is part of eclib, g95 is fortran and only needed for blas,
which I see you use CBLAS for. bzip2 is a prereq of python and may
even only be required on some platforms, I don't know.

Sorry, this is all OT for the thread and only tangentially relevant to
how spkgs are required by sage.

Bill.

On 17 Aug, 02:12, Bill Hart <goodwillh...@googlemail.com> wrote:

William Stein

unread,
Aug 16, 2010, 9:32:28 PM8/16/10
to sage-devel
On Mon, Aug 16, 2010 at 6:29 PM, Bill Hart <goodwi...@googlemail.com> wrote:
> Where? I can't even download it, so I don't even get past go.

http://sagemath.org/packages/standard/sympow-1.018.1.p7.spkg

Dr. David Kirkby

unread,
Aug 16, 2010, 9:35:12 PM8/16/10
to sage-...@googlegroups.com
On 08/17/10 02:29 AM, Bill Hart wrote:
> Where? I can't even download it, so I don't even get past go.

There is no web site for SYMPOW - apparently it has been removed.

But SYMPOW is a standard package in Sage - see

spkg/standard/sympow-1.018.1.p7.spkg

Dave

Bill Hart

unread,
Aug 16, 2010, 10:31:59 PM8/16/10
to sage-devel
I have no sympathy for anyone who wants to include that code in Sage.
No documentation. It's virtually obfuscated. Test code is where?

At any rate I used my "special powers" to divine that the following
files are required to get the modular degree functions working:

moddeg.c QD.c eulerfactors.c conductors.c compute.c ec_ap.c util.c
factor.c ec_ap_bsgs.c ec_ap_large.c compute2.c analytic.c sympow.h

where I removed deflate and following from eulerfactors.c, everything
but ec_do and the first few lines from compute.c, everything from
wild_local_conductor and down from conductors.c. I did not bother
checking what was needed out of util.c or compute2.c, (though
presumably not much) because I was far to ***LAZY**.

Does that make your "porting" job any easier?

Bill.

On 17 Aug, 02:32, William Stein <wst...@gmail.com> wrote:
> On Mon, Aug 16, 2010 at 6:29 PM, Bill Hart <goodwillh...@googlemail.com> wrote:
> > Where? I can't even download it, so I don't even get past go.
>
> http://sagemath.org/packages/standard/sympow-1.018.1.p7.spkg
>
>
>
>
>
>
>
> > On 17 Aug, 02:26, "Dr. David Kirkby" <david.kir...@onetel.net> wrote:
> >> On 08/17/10 01:59 AM, Bill Hart wrote:
>
> >> > Hmm, I see the argument between the researcher (William) who wants to
> >> > *use* sympow, and the code porter (David) who wants to *port* sympow.
>
> >> > I think I will remain on the fence on this one, pull up a chair, grab
> >> > a beer from the fridge..... :-)
>
> >> Bill,
>
> >> If it was a case of porting sympow, I would just get on and do it. I've got
> >> countless other packages working on Solaris by sorting out portability issues.
>
> >> But SYMPOW's problems extend far beyond portability issues. That's just the tip
> >> of the iceberg. Take a look at the source code, and you will see what I mean.
>
> >> Dave
>
> > --
> > To post to this group, send an email to sage-...@googlegroups.com
> > To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> > For more options, visit this group athttp://groups.google.com/group/sage-devel

François Bissey

unread,
Aug 17, 2010, 12:23:47 AM8/17/10
to sage-...@googlegroups.com
> On 08/16/10 11:43 PM, François Bissey wrote:
> > If you weren't using sympow you won't be missing it and nothing would
> > break. Nothing apart from lfunctions/sympow.py calls sympow and I don't
> > think anything in that python file is called from anywhere else - it
> > just provides the interface to talk to sympow.
>
> Unfortunately, with sympow not working, several doctests fail.
>
> I stated in
>
> http://trac.sagemath.org/sage_trac/ticket/9703
>
> that it was
>
> devel/sage/sage/lfunctions/sympow.py
>
> but in fact, these all fail with a message about sympow.
>
> devel/sage/sage/modular/abvar/abvar.py
> devel/sage/sage/modular/hecke/submodule.py
> devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py
>
> > In short sympow could be made optional
> > overnight by dropping it from the deps file.
>
> I think it needs a bit more than that.
>
Right I missed the bits in sage/schemes/elliptic_curves/lseries_ell.py
which is probably why all these extra tests fails. Some stuff is defined there
and is used.
Too bad for the optional idea, my mistake.

Francois

John Cremona

unread,
Aug 17, 2010, 4:26:49 AM8/17/10
to sage-...@googlegroups.com
At last, a contributiuon to this thread which actually addresses my
original question!

>> In contrast, it seems sympow is not used by many people. The fact there are
>> no bug reports of it not working on the public servers shows how little it
>> is used. (If someone actually tried to run many of the examples, they will
>> find they don't work).
>
> There is 1 "example" -- the modular degree function --  that is 1000
> times more important than the rest.    Here's me *using* sympow:
>
> sage: E = EllipticCurve('5077a'); E
> Elliptic Curve defined by y^2 + y = x^3 - 7*x + 6 over Rational Field
> sage: E.modular_degree()
> 1984
>

This is what I thought. I could provide modular degree from eclib but
it would be *vastly* slower. I myself stopped using my own modular
degree function (despite publishing a paper describing the algorithm,
which I was rather proud of at the time) and started to use sympow
(which was then called ec) for the elliptic curve database, since my
own version took much too long. In fact it is fair to say that for
any elliptic curve not in the database (whose modular degrees are
known and in the database), my implementation would not finish in
reasonable time, which makes my "offer" worth little.

It would be a student project to reimplement Mark W's algorithm (Here:
http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)
.

John

Peter Jeremy

unread,
Aug 17, 2010, 4:09:09 PM8/17/10
to sage-...@googlegroups.com
On 2010-Aug-16 14:32:14 -0700, Bill Hart <goodwi...@googlemail.com> wrote:
>I'm also really wondering why cephes is in Sage. Unless there are two
>packages with the same name, it was last updated circa 1994 and uses
>non-portable long doubles. We've booted packages from Sage for much
>less felonious offences.

cephes is used to provide C99 math functions for systems that do not
provide complete C99 math functionality natively. Currently, this
is cygwin and FreeBSD, though other proprietary Unix systems may also
need it.

Whilst it was _written_ in the early 90's, it has been updated since
then, with patches dated up to around 2005. In any case, age of code
is not inherently an indication of bugginess. Assuming code has no
known bugs and achieves its stated functionality, there is no need to
touch it (though I don't know this is true of cephes).

As for "long doubles" - whilst they may be non-portable, AFAIK, they
are part of C99. They are also extensively used in Sage. If you
want to get rid of long doubles completely, feel free. It's being
unreasonable to have long doubles in Sage and then remove code that
is needed to support them.

On 2010-Aug-16 16:24:24 -0700, William Stein <wst...@gmail.com> wrote:
>I would rather have a windows port than not having a Windows port.
>Problems may be created as a result. But since Cephes is only used
>on Windows, the problems will be less than the current problem which
>is that Sage doesn't work at all on Windows.

See http://trac.sagemath.org/sage_trac/ticket/9543
Is there no longer any interest in a FreeBSD port?

--
Peter Jeremy

Dr. David Kirkby

unread,
Aug 17, 2010, 5:59:22 PM8/17/10
to sage-...@googlegroups.com
On 08/17/10 03:31 AM, Bill Hart wrote:
> I have no sympathy for anyone who wants to include that code in Sage.

I have very little myself.

I know John Cremona has made the point SYMPOW is much faster than anything he
has written at computing the modular degree of an elliptic curve, but I'd
personally not want to trust this code.

It obviously works in some cases, and works faster than anything else available.
But how reliable is it? That's what concerns me.

Of course we can never 100% trust software, but there are indicators here to
make me *much* more mistrustful than usual.

> No documentation.

Well the README file has some (very minimal) documentation.

Comments in the code are far from adequate.

> It's virtually obfuscated.

It can't however be entered in to The International Obfuscated C Code Contest,

http://www.ioccc.org/

since it's not legal C! Using the return value of functions declared as void is
not permitted in C, thought gcc rather stupidly accepts it.

> Test code is where?

Exactly. How well has it been tested?

It seems to be that it difficult to test it, as there's nothing else able to
compute some of these things quickly.

> At any rate I used my "special powers" to divine that the following
> files are required to get the modular degree functions working:
>
> moddeg.c QD.c eulerfactors.c conductors.c compute.c ec_ap.c util.c
> factor.c ec_ap_bsgs.c ec_ap_large.c compute2.c analytic.c sympow.h

> Does that make your "porting" job any easier?

Thanks for your efforts Bill, but you need to take a step back and try to
convince me *why* I should bother "porting" this.

IMHO, this is not a portability problem, but a quality control problem.

Perhaps William will look at this for the Cygwin port, as it is also a package
breaking on Cygwin.

SYMPOW is not something I feel willing or able to do.

> Bill.

Dave

Dr. David Kirkby

unread,
Aug 17, 2010, 6:11:28 PM8/17/10
to sage-...@googlegroups.com
On 08/17/10 09:26 AM, John Cremona wrote:
> At last, a contributiuon to this thread which actually addresses my
> original question!
>
>>> In contrast, it seems sympow is not used by many people. The fact there are
>>> no bug reports of it not working on the public servers shows how little it
>>> is used. (If someone actually tried to run many of the examples, they will
>>> find they don't work).
>>
>> There is 1 "example" -- the modular degree function -- that is 1000
>> times more important than the rest. Here's me *using* sympow:
>>
>> sage: E = EllipticCurve('5077a'); E
>> Elliptic Curve defined by y^2 + y = x^3 - 7*x + 6 over Rational Field
>> sage: E.modular_degree()
>> 1984
>>
>
> This is what I thought. I could provide modular degree from eclib but
> it would be *vastly* slower. I myself stopped using my own modular
> degree function (despite publishing a paper describing the algorithm,
> which I was rather proud of at the time) and started to use sympow
> (which was then called ec) for the elliptic curve database, since my
> own version took much too long. In fact it is fair to say that for
> any elliptic curve not in the database (whose modular degrees are
> known and in the database), my implementation would not finish in
> reasonable time, which makes my "offer" worth little.

John,

I think if you are able to compute this with your package, it would be useful
addition to Sage. Having two independent algorithms, implemented by two
different people, could be useful, despite one being much slower than the other.

I think it's particularly so in this case, given the concerns some people have
about the code quality.

> It would be a student project to reimplement Mark W's algorithm (Here:
> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)

Yes, it would be useful. But even if someone did that and got it working fast,
having the choice of a second algorithms is useful IMHO.


>
> John

William Stein

unread,
Aug 17, 2010, 6:31:56 PM8/17/10
to sage-devel

The difference in complexity is very significant. John's is probably
O(N^3), whereas Mark's is O(sqrt(N)). This is a *dramatic* difference
in practice. It means hundreds of years versus a few seconds...

Also, something that is close to John's algorithm is already in Sage,
which I implemented. Well, this is technically a third distinct
algorithm (which is much more general in some ways):

sage: A = J0(54).new_subvariety()[1]
sage: A.modular_degree()
2
sage: EllipticCurve('54b').modular_degree()
2

> I think it's particularly so in this case, given the concerns some people
> have about the code quality.
>
>> It would be a student project to reimplement Mark W's algorithm (Here:
>> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)

This is what should happen. After somebody implements his algorithm
in Sage, then
we can (re)move Sympow. Here's a trac ticket:

http://trac.sagemath.org/sage_trac/ticket/9758

I wish Robert Bradshaw would finish
http://trac.sagemath.org/sage_trac/ticket/9193, by the way.

William

John Cremona

unread,
Aug 18, 2010, 6:33:25 AM8/18/10
to sage-...@googlegroups.com

That's quite true. The set of curves on which my algorithm will
deliver in reasonable time is a strict subset of those for which we
know the answer already and have it in the database. (The numbers in
the database were computed either by both my algorithm and Mark's, and
compared, or -- for the larger conductors -- just by his). That
makes it quite redundant to revive my implementation (which would be
nontrivial since I commented out that code in eclib years ago and have
made many other changes to the rest of the code since).

> Also, something that is close to John's algorithm is already in Sage,
> which I implemented.  Well, this is technically a third distinct
> algorithm (which is much more general in some ways):
>

I guess this is what I think of as the Merel linear algebra method.
This was also once in eclib.....(but only for elliptic curves, yours
is more genera as you say).

> sage: A = J0(54).new_subvariety()[1]
> sage: A.modular_degree()
> 2
> sage: EllipticCurve('54b').modular_degree()
> 2
>
>> I think it's particularly so in this case, given the concerns some people
>> have about the code quality.
>>
>>> It would be a student project to reimplement Mark W's algorithm (Here:
>>> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)
>
> This is what should happen.  After somebody implements his algorithm
> in Sage, then
> we can (re)move Sympow.  Here's a trac ticket:
>
>   http://trac.sagemath.org/sage_trac/ticket/9758
>

Excellent. Delaunay replied to me saying that he was away from home
but when he returns he will try to locate his own implementation,
which is in a GP script. That might well help the implementer. Also,
in one of his replies on this discussion (not copied to the list so I
hope he will not mind me quoting hime here, Mark says

"My understanding was that Christophe had more details about the analytic
questions (like inverse Mellin transforms and round-off error -- neither
referee insisted that I say anything there), while I spent more time with
the bad Euler factors (having tracked down the Coates-Schmidt paper where
they essentially appear, though with a few accounting errors, and with a
method of computation not explicitly given). I also took an "experimental"
view of the question, computing the modular degree for a large dataset of
curves, whilst his focus was more "French" (if I may), in that it gives a
theoretical description and then (almost grudgingly) appends a few examples."

This difference of emphasis is significant. Mark wanted to collect a
lot of data, so wanted his program to be fast, while Christophe just
wanted some examples to show that the method was feasible. Hie script
may still be useful though.

John

> I wish Robert Bradshaw would finish
> http://trac.sagemath.org/sage_trac/ticket/9193, by the way.
>
>
>
> William
>

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com

David Kirkby

unread,
Aug 18, 2010, 7:18:55 AM8/18/10
to sage-...@googlegroups.com
On 18 August 2010 11:33, John Cremona <john.c...@gmail.com> wrote:
> On 17 August 2010 23:31, William Stein <wst...@gmail.com> wrote:

>>>> It would be a student project to reimplement Mark W's algorithm (Here:
>>>> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)
>>


>> This is what should happen.  After somebody implements his algor

>> in Sage, then
>> we can (re)move Sympow.  Here's a trac ticket:
>>
>>   http://trac.sagemath.org/sage_trac/ticket/9758
>>

It would be good to state on the ticket what level of student project
this is (undergrad, postgrad), and the skill set needed (C, Python
etc). I expect you would prefer it in Python, though I think
personally a C implementation like Marks, which you can easily call
from Python, would be more beneficial to the scientific community in
general - not everyone is using Sage/Python.

I know if I were a student, I'd prefer a bit more information on that
trac ticket.

Is there a Sage web page where potential student projects could be
listed? A student is unlikely to find it on a trac ticket? I believe
some Sage projects have been done in collaboration with other
universities - IIRC, there was student at a uni in London who did a
Sage related project.

I'm basically trying to see how you could maximise the chances of
getting a student to take it on. SYMPOW is the *only* significant
barrier to Sage passing all tests on 32-bit Solaris x86 and
OpenSolaris - the others are numerical noise. But I'm reluctant to
even look at SYMPOW.

> Excellent.  Delaunay replied to me saying that he was away from home
> but when he returns he will try to locate his own implementation,
> which is in a GP script.  That might well help the implementer.  Also,
> in one of his replies on this discussion (not copied to the list so I
> hope he will not mind me quoting hime here, Mark says
>
> "My understanding was that Christophe had more details about the analytic
> questions (like inverse Mellin transforms and round-off error -- neither
> referee insisted that I say anything there), while I spent more time with
> the bad Euler factors (having tracked down the Coates-Schmidt paper where
> they essentially appear, though with a few accounting errors, and with a
> method of computation not explicitly given). I also took an "experimental"
> view of the question, computing the modular degree for a large dataset of
> curves, whilst his focus was more "French" (if I may), in that it gives a
> theoretical description and then (almost grudgingly) appends a few examples."
>
> This difference of emphasis is significant.  Mark wanted to collect a
> lot of data, so wanted his program to be fast, while Christophe just
> wanted some examples to show that the method was feasible.  Hie script
> may still be useful though.
>
> John

Again, that could usefully be added on the trac ticket.

>
>> I wish Robert Bradshaw would finish
>> http://trac.sagemath.org/sage_trac/ticket/9193, by the way.

>> William

Dave

William Stein

unread,
Aug 18, 2010, 8:13:28 PM8/18/10
to sage-...@googlegroups.com
On Wed, Aug 18, 2010 at 4:18 AM, David Kirkby <david....@onetel.net> wrote:
> On 18 August 2010 11:33, John Cremona <john.c...@gmail.com> wrote:
>> On 17 August 2010 23:31, William Stein <wst...@gmail.com> wrote:
>
>>>>> It would be a student project to reimplement Mark W's algorithm (Here:
>>>>> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)
>>>
>
>
>>> This is what should happen.  After somebody implements his algor
>>> in Sage, then
>>> we can (re)move Sympow.  Here's a trac ticket:
>>>
>>>   http://trac.sagemath.org/sage_trac/ticket/9758
>>>
>
> It would be good to state on the ticket what level of student project
> this is (undergrad, postgrad), and the skill set needed (C, Python
> etc). I expect you would prefer it in Python, though I think
> personally a C implementation like Marks, which you can easily call
> from Python, would be more beneficial to the scientific community in
> general - not everyone is using Sage/Python.
>
> I know if I were a student, I'd prefer a bit more information on that
> trac ticket.

Good idea -- I've posted some stuff there.

> Is there a Sage web page where potential student projects could be
> listed? A student is unlikely to find it on a trac ticket? I believe
> some Sage projects have been done in collaboration with other
> universities - IIRC, there was student at a uni in London who did a
> Sage related project.

I think every open ticket on trac (all >2000 of them) has the
potential to be a student project. However, during the last year when
I've told prospective students: "try to find a trac ticket to work on"
they almost always disappear and never return. So something more
organized would be of value. However, who has the time...

> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

--

Dr. David Kirkby

unread,
Aug 18, 2010, 10:23:26 PM8/18/10
to sage-...@googlegroups.com
On 08/19/10 01:13 AM, William Stein wrote:
> On Wed, Aug 18, 2010 at 4:18 AM, David Kirkby<david....@onetel.net> wrote:

>>>> This is what should happen. After somebody implements his algor
>>>> in Sage, then
>>>> we can (re)move Sympow. Here's a trac ticket:
>>>>
>>>> http://trac.sagemath.org/sage_trac/ticket/9758
>>>>
>>
>> It would be good to state on the ticket what level of student project
>> this is (undergrad, postgrad), and the skill set needed (C, Python
>> etc). I expect you would prefer it in Python, though I think
>> personally a C implementation like Marks, which you can easily call
>> from Python, would be more beneficial to the scientific community in
>> general - not everyone is using Sage/Python.
>>
>> I know if I were a student, I'd prefer a bit more information on that
>> trac ticket.
>
> Good idea -- I've posted some stuff there.

Thank you.

>> Is there a Sage web page where potential student projects could be
>> listed? A student is unlikely to find it on a trac ticket? I believe
>> some Sage projects have been done in collaboration with other
>> universities - IIRC, there was student at a uni in London who did a
>> Sage related project.
>
> I think every open ticket on trac (all>2000 of them) has the
> potential to be a student project.

I'd have to disagree with that. Many are trivial issues and could not form the
basis of a project.

Some, such as fixing bugs in code written by others, are not very attractive to
students.

http://trac.sagemath.org/sage_trac/ticket/9758

seems a nice self-contained project. A student could get on with it, and not be
too relient on others for getting it reviewed. In contrast, if you took a ticket
like "Port Sage to the Blackberry", then a student would be very reliant on
getting others to review their changes. That would be very frustrating,
especially as the deadline for the project approaches, but you can't find
reviewers.

> However, during the last year when
> I've told prospective students: "try to find a trac ticket to work on"
> they almost always disappear and never return.

To be honest, I'm amazed any do! There are too many tickets that are unsuitable,
so students probably lose interest quite quickly.

> So something more
> organized would be of value. However, who has the time...

It might be sensible to add something on trac to make it easier.

* The keyword "student_project" could be used, but it seems nobody really uses
the keywords, so I don't believe that would work.

* Another possibility might be to add to "Milestone" the term "student
project". But that might tend to get missed by people. Milestone is normally set
to be the next release.

* More work, but probably best, is a tick-box/radio button where tickets that
could form the basis of a student project were marked as "potential student
project". Then you could give the students a URL which lists all the tickets
which could be potential projects. Anyone chosing to mark a ticket as suitable
for a student project should be encouraged to provide provide information such
as the skill level, relevant references etc.

Dave

Mitesh Patel

unread,
Aug 19, 2010, 4:50:53 AM8/19/10
to sage-...@googlegroups.com
On 08/18/2010 07:13 PM, William Stein wrote:
> On Wed, Aug 18, 2010 at 4:18 AM, David Kirkby <david....@onetel.net> wrote:
>> On 18 August 2010 11:33, John Cremona <john.c...@gmail.com> wrote:
>>> On 17 August 2010 23:31, William Stein <wst...@gmail.com> wrote:
>>
>>>>>> It would be a student project to reimplement Mark W's algorithm (Here:
>>>>>> http://www.emis.ams.org/journals/EM/expmath/volumes/11/11.4/pp487_502.pdf)
>>>>
>>>> This is what should happen. After somebody implements his algor
>>>> in Sage, then
>>>> we can (re)move Sympow. Here's a trac ticket:
>>>>
>>>> http://trac.sagemath.org/sage_trac/ticket/9758
>>>>
>>
>> It would be good to state on the ticket what level of student project
>> this is (undergrad, postgrad), and the skill set needed (C, Python
>> etc). I expect you would prefer it in Python, though I think
>> personally a C implementation like Marks, which you can easily call
>> from Python, would be more beneficial to the scientific community in
>> general - not everyone is using Sage/Python.
>>
>> I know if I were a student, I'd prefer a bit more information on that
>> trac ticket.
>
> Good idea -- I've posted some stuff there.

Would a project to rewrite genus2reduction.c [1] as part of the Sage
library be at a similar level? Is there a paper(s) on the algorithm?

[1] This is part of the genus2reduction spkg, which does not have the
same build problems as SYMPOW, as far as I'm aware. The motivation here
would be, perhaps, to get some experience with PARI, Cython, and
(hyper)elliptic curves. We'd also reduce very slightly our spkg
maintenance burden.

John Cremona

unread,
Aug 19, 2010, 5:12:58 AM8/19/10
to sage-...@googlegroups.com

I think the relavant paper from Liu's web page
(http://www.math.u-bordeaux.fr/~liu/) is this:

Qing Liu : Conducteur et discriminant minimal de courbes de genre 2
Compositio Math., 94 (1994), 51-79

Unfortunately, unless I am mistaken (and I hope so!) there are no
genus 2 experts currently working on (or with) Sage. The credit for
his program goes to various people (Cohen, Poonen, Schaefer, Stoll and
more) as well as himself. I think that this would be considerably
harder than reimplementing the modular degree part of sympow.

Note, in particular, that Liu's algorithm only gives the odd part of
the conductor of a genus 2 curve. Even after 15 years or so, no-one
has ever worked out the details for p=2. I regularly ask genus 2
people (if you see what I mean; it sounds a very uncomfortable state
to be in) about this and they usually throw up their hands saying
there are hundreds of cases to be considered. I don't think I have
the expertise to do that, or supervise someone else to do it.

John

David Kirkby

unread,
Aug 20, 2010, 12:28:40 PM8/20/10
to sage-...@googlegroups.com
On 16 August 2010 23:48, Bill Hart <goodwi...@googlemail.com> wrote:
>
>
> On 16 Aug, 22:50, William Stein <wst...@gmail.com> wrote:
>> On Mon, Aug 16, 2010 at 2:32 PM, Bill Hart <goodwillh...@googlemail.com> wrote:
>> > Is there even a link from which Sympow can be downloaded so that one
>> > can look through the code to see if it is worth salvaging. I can't
>> > even find a webpage for Mark Watkins at the moment, let alone sympow.
>>
>> > I honestly suspect that Mark has just moved on to other projects. I
>> > doubt very much if his work on Magma precludes him from maintaining
>> > this package. I've personally worked with him on projects since he
>> > joined the Magma group and we both made use of each other's code
>> > without even asking the question.
>>
>> The above is just speculation without even asking him.
>
> I'll email him, as I speak to him quasi-regularly by email. And I know
> for a fact he has been working on other things. I personally worked on
> them with him, and he discussed other projects he's done too. I don't
> think I speculated in any of the three sentences here. But I will
> explicitly ask him about the status of sympow.

Did you do this Bill? Did you get any response?

Dave

Bill Hart

unread,
Aug 23, 2010, 8:38:24 AM8/23/10
to sage-devel


On 17 Aug, 21:09, Peter Jeremy <peterjer...@acm.org> wrote:
> On 2010-Aug-16 14:32:14 -0700, Bill Hart <goodwillh...@googlemail.com> wrote:
>
> >I'm also really wondering why cephes is in Sage. Unless there are two
> >packages with the same name, it was last updated circa 1994 and uses
> >non-portable long doubles. We've booted packages from Sage for much
> >less felonious offences.
>
> cephes is used to provide C99 math functions for systems that do not
> provide complete C99 math functionality natively.  Currently, this
> is cygwin and FreeBSD, though other proprietary Unix systems may also
> need it.
>

I see.

> Whilst it was _written_ in the early 90's, it has been updated since
> then, with patches dated up to around 2005.  In any case, age of code
> is not inherently an indication of bugginess.  Assuming code has no
> known bugs and achieves its stated functionality, there is no need to
> touch it (though I don't know this is true of cephes).
>

The issue is not whether it has no known bugs, but who will maintain
it when bugs are found.

> As for "long doubles" - whilst they may be non-portable, AFAIK, they
> are part of C99.  They are also extensively used in Sage.  If you
> want to get rid of long doubles completely, feel free.  It's being
> unreasonable to have long doubles in Sage and then remove code that
> is needed to support them.
>

They are part of the C99 standard. However, unlike IEEE doubles, they
are not standardised. This makes them totally worthless for
mathematical computations where one wants to rely on any properties of
the result. They should not be used in Sage. They are essentially a
hack.

If someone wrote a long doubles library which was standardised across
all systems supported by Sage and used software emulation on platforms
where native hardware support is not provided (pretty much all of
them), then I would support the inclusion of that in Sage.

Bill.

> On 2010-Aug-16 16:24:24 -0700, William Stein <wst...@gmail.com> wrote:
>
> >I would rather have a windows port than not having a Windows port.
> >Problems may be created as a result.   But since Cephes is only used
> >on Windows, the problems will be less than the current problem which
> >is that Sage doesn't work at all on Windows.
>
> Seehttp://trac.sagemath.org/sage_trac/ticket/9543
> Is there no longer any interest in a FreeBSD port?
>
> --
> Peter Jeremy
>
>  application_pgp-signature_part
> < 1KViewDownload

Bill Hart

unread,
Aug 23, 2010, 8:46:30 AM8/23/10
to sage-devel
Yes, I wrote to Mark. I asked him for permission to post his response
in toto here. But he did not reply to that email. His response was
*very* helpful and I suspect anyone wanting to reimplement Sympow
would do well to contact Mark directly and also John Cremona.

The two important points I got from Mark's reply were:

1) Mark is definitely *not* intending to "maintain" Sympow as a
software project. He "maintains" it primarily for his own use, and if
it runs on his machine that is enough for him.

2) The modular degree stuff, in the range that it is feasible, would
probably work just fine if we ditched the QD stuff and just replaced
it with doubles. If this is all we really require from Sympow, it
makes good sense to do this. Long doubles are evil!

Bill.

On 20 Aug, 17:28, David Kirkby <david.kir...@onetel.net> wrote:

Dr. David Kirkby

unread,
Aug 23, 2010, 9:03:07 AM8/23/10
to sage-...@googlegroups.com
On 08/23/10 01:46 PM, Bill Hart wrote:
> Yes, I wrote to Mark. I asked him for permission to post his response
> in toto here. But he did not reply to that email. His response was
> *very* helpful and I suspect anyone wanting to reimplement Sympow
> would do well to contact Mark directly and also John Cremona.
>
> The two important points I got from Mark's reply were:
>
> 1) Mark is definitely *not* intending to "maintain" Sympow as a
> software project. He "maintains" it primarily for his own use, and if
> it runs on his machine that is enough for him.
>
> 2) The modular degree stuff, in the range that it is feasible, would
> probably work just fine if we ditched the QD stuff and just replaced
> it with doubles. If this is all we really require from Sympow, it
> makes good sense to do this. Long doubles are evil!
>
> Bill.

From my own perspective, the code is so convoluted that I would not want to try
to replace anything.

In any case, it now works. Mike has tested it on Cygwin, and I've tested it on
Solaris, OpenSolaris, Linux and OS X.

I'm still amazed the code works at all. It's hard to know what one is supposed
to do with code which has

void foobar()

main() {
return foobar()
}

I think its a bug in gcc in compiling this, but my attempts to produce a subset
of it which I can report ot the gcc developers has failed.

The Sun C compiler naturally rejects this code, as it's not C.

When you see things like a configure script that starts

#! /bin/sh

then has code to test if 'sh' exists or not, I think you can understand why I am
reluctant to do any more with SYMPOW than I absolutely need to. Replacing it
with double, long double, mpfr is not something I want to do.

$ rm -rf sympow

would suite me better, but I think the patch I have works.

Dave

Robert Bradshaw

unread,
Aug 23, 2010, 12:19:10 PM8/23/10
to sage-...@googlegroups.com

I was just thinking about that this morning :). I plan to attack it
later this week.

- Robert

Reply all
Reply to author
Forward
0 new messages