proposal to make zn_poly a standard spkg

2 views
Skip to first unread message

David Harvey

unread,
Mar 18, 2008, 9:29:55 PM3/18/08
to sage-...@googlegroups.com
Hi all,

I propose making the zn_poly library a standard spkg in Sage, and hereby request a vote on the matter.

Disclaimer: I am the author of zn_poly.

Here is the ticket:

This spkg has been in the optional repository for about three months.

You can try downloading and building it via

   sage -i zn_poly-0.4.1.spkg

So far there have been positive build reports from:

* mike hansen: builds and installs fine for me on 64-bit Ubuntu 7.10
* tom boothby: Compiled fine on my server (debian / xeon)
* joel mohler: Builds, tunes, and installs fine for me with Gentoo on Intel P4 and Intel T2060
* michael abshoff: Solaris/Sparc
* myself: mac os 10.4.11, intel core 2 duo and ppc G5; and on sage.math

LICENSE: GPL v2 or GPL v3.

FILE SIZE: 38 KB.

BENEFITS:

zn_poly is a new C library for polynomial arithmetic in (Z/nZ)[x] where 3 <= n <= ULONG_MAX (i.e. any machine-word-sized modulus).

The main benefit is speed. Three examples on sage.math, from my current development code (this code is *not* yet in the spkg):

* Multiplying length 200 polynomials over Z/nZ where n has 10 bits:
    * NTL (zz_pX): 113 µs
    * Magma: 44 µs
    * zn_poly: 13 µs

* Multiplying length 10^6 polynomials over Z/nZ where n has 40 bits and is odd:
    * NTL (zz_pX): 9.1s
    * Magma: 8.3s
    * zn_poly: 2.06s

* Reciprocal of a length 10^6 power series over Z/nZ where n has 40 bits and is odd:
    * NTL (zz_pX): 25.4s
    * Magma: ludicrously slow, maybe I'm doing something wrong
    * zn_poly: 3.62s

Other benefits:

* zn_poly is threadsafe (unlike the next-best alternative in Sage, namely NTL --- no nasty global modulus objects).
* zn_poly is a prerequisite for http://sagetrac.org/sage_trac/ticket/1568 (faster zeta functions for hyperelliptic curves)
* zn_poly gives me a warm fuzzy feeling inside.

My long-term plan for zn_poly in Sage is to:

* make it the default implementation of PolynomialRing(Integers(n)) for 3 <= n <= ULONG_MAX
* provide a cython wrapper so that it can be easily used in other cython code.

The version in the spkg (release 0.4.1) is a bit old now; zn_poly has grown a lot since then, see my webpage


for the latest release. Once the spkg is standard in Sage, I will be supplying upgraded spkgs for each new zn_poly release (usually about once every 2-4 weeks).

david

mabshoff

unread,
Mar 18, 2008, 9:36:27 PM3/18/08
to sage-devel
> * zn_poly is a prerequisite forhttp://sagetrac.org/sage_trac/ticket/
> 1568 (faster zeta functions for hyperelliptic curves)
> * zn_poly gives me a warm fuzzy feeling inside.
>
> My long-term plan for zn_poly in Sage is to:
>
> * make it the default implementation of PolynomialRing(Integers(n))  
> for 3 <= n <= ULONG_MAX
> * provide a cython wrapper so that it can be easily used in other  
> cython code.
>
> The version in the spkg (release 0.4.1) is a bit old now; zn_poly has  
> grown a lot since then, see my webpage
>
>    http://math.harvard.edu/~dmharvey/zn_poly/
>
> for the latest release. Once the spkg is standard in Sage, I will be  
> supplying upgraded spkgs for each new zn_poly release (usually about  
> once every 2-4 weeks).
>
> david

+1 on the spkg, but I would greatly prefer to merge the 0.7 release
instead of the currently optional 0.4.1 release.

I can do some more build testing in case you need it for the 0.7
release.

Cheers,

Michael

David Harvey

unread,
Mar 18, 2008, 9:43:52 PM3/18/08
to sage-...@googlegroups.com

On Mar 18, 2008, at 9:36 PM, mabshoff wrote:

> +1 on the spkg, but I would greatly prefer to merge the 0.7 release
> instead of the currently optional 0.4.1 release.

The main problem with 0.7 is that the tuning process is quite slow,
and needs more work to make it feasible during the sage build. I will
be addressing this in 0.8.

Also there was a slight interface change somewhere between 0.4.1 and
0.7 which will make hypellfrob (ticket #1568) break. I will need to
supply updated code for that too.

david

mabshoff

unread,
Mar 18, 2008, 10:08:49 PM3/18/08
to sage-devel
No problem. Assuming the 0.4.1.spkg will be merged you can then update
to 0.8 once it is ready.

Cheers,

Michael

William Stein

unread,
Mar 18, 2008, 11:31:19 PM3/18/08
to sage-...@googlegroups.com
On Tue, Mar 18, 2008 at 6:29 PM, David Harvey <dmha...@math.harvard.edu> wrote:
>
> Hi all,
>
> I propose making the zn_poly library a standard spkg in Sage, and hereby
> request a vote on the matter.

Enthusiastic +1

The package is tiny, currently builds in 4 seconds on my laptop, and the
code is quite pretty. Also, the functionality is clearly very relevant to
Sage.

-- William

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

Craig Citro

unread,
Mar 19, 2008, 12:29:27 AM3/19/08
to sage-...@googlegroups.com
> > I propose making the zn_poly library a standard spkg in Sage, and hereby
> > request a vote on the matter.
>

Another very enthusiastic +1.

-cc

Martin Albrecht

unread,
Mar 19, 2008, 5:45:36 AM3/19/08
to sage-...@googlegroups.com

Robert Bradshaw

unread,
Mar 19, 2008, 6:27:25 AM3/19/08
to sage-...@googlegroups.com
On Mar 18, 2008, at 6:29 PM, David Harvey wrote:

> Hi all,
>
> I propose making the zn_poly library a standard spkg in Sage, and
> hereby request a vote on the matter.
>
> Disclaimer: I am the author of zn_poly.
>
> Here is the ticket:
>
> http://sagetrac.org/sage_trac/ticket/1567
>
> This spkg has been in the optional repository for about three months.

+1 from me too. I would like to see a .pxd file for easy use from
Sage, but I'm sure that's on your todo list.

- Robert

David Roe

unread,
Mar 19, 2008, 1:42:17 AM3/19/08
to sage-...@googlegroups.com
+1 from me as well.
David

David Harvey

unread,
Mar 19, 2008, 7:44:07 AM3/19/08
to sage-...@googlegroups.com

On Mar 19, 2008, at 6:27 AM, Robert Bradshaw wrote:

> +1 from me too. I would like to see a .pxd file for easy use from
> Sage, but I'm sure that's on your todo list.

It is indeed on my list.

david

boo...@u.washington.edu

unread,
Mar 19, 2008, 3:08:36 PM3/19/08
to sage-...@googlegroups.com

+1 pending easy use from Sage.


William Stein

unread,
Mar 19, 2008, 3:23:48 PM3/19/08
to sage-...@googlegroups.com

Hey, that's an *extremely* interesting point!  I think you're saying
that znpoly's spkg should be standard in Sage until it is possible
to actually use it from Sage.  If the author hasn't at least done that
in the last few months, perhaps making it a requirement for inclusion
in Sage will motivate him to do so!

Indeed -- David Harvey -- I think Tom has a very good point.
Perhaps znpoly shouldn't be in sage until I can at least do
   sage: from sage.libs.znpoly.all import znpoly
   sage: znpoly([...])

William

boo...@u.washington.edu

unread,
Mar 19, 2008, 3:44:48 PM3/19/08
to sage-...@googlegroups.com

Independent of the discussion of znpoly, (which honestly, I'm rather excited about) I think this should be one of the requirements for including any (standard) spkg in Sage. Otherwise, what's the point?

David Harvey

unread,
Mar 19, 2008, 3:46:54 PM3/19/08
to sage-...@googlegroups.com

On Mar 19, 2008, at 3:23 PM, William Stein wrote:

> >> +1 from me too. I would like to see a .pxd file for easy use from
> >> Sage, but I'm sure that's on your todo list.
> >
> > It is indeed on my list.
>
> +1 pending easy use from Sage.
>
> Hey, that's an *extremely* interesting point! I think you're saying
> that znpoly's spkg should be standard in Sage until it is possible
> to actually use it from Sage. If the author hasn't at least done that
> in the last few months, perhaps making it a requirement for inclusion
> in Sage will motivate him to do so!
>
> Indeed -- David Harvey -- I think Tom has a very good point.
> Perhaps znpoly shouldn't be in sage until I can at least do
> sage: from sage.libs.znpoly.all import znpoly
> sage: znpoly([...])

Well, I'll leave it to the list to decide whether this should be a
requirement for inclusion as a standard spkg, but let me say a few
relevant things.

First, regardless of whether there is "direct access" to zn_poly from
Sage, it simply won't be possible to incorporate my faster
hyperelliptic curve zeta function code (http://trac.sagemath.org/
sage_trac/ticket/1568) until zn_poly is included. So if that counts
as "using zn_poly from Sage", then I think that answers the question.
I mean, for the moment you could just think of it as an auxiliary
library that's required to make the zeta function code work.

Second, the interface is still not complete enough to be worth
wrapping. I don't want to tie myself down too much yet, in terms of
making hard-to-reverse design decisions about the interface. When I
am more comfortable about how the interface looks, then I will write
a wrapper.

If people feel more strongly that a wrapper is necessary, I would
prefer to withdraw both patches and come back with more code in a few
months, rather than put in a wrapper now.

david

boo...@u.washington.edu

unread,
Mar 19, 2008, 3:56:05 PM3/19/08
to sage-...@googlegroups.com


On Wed, 19 Mar 2008, David Harvey wrote:

Good point. I retract my last statement. Being useful for an application is generally more important than having a wrapper.


William Stein

unread,
Mar 19, 2008, 4:14:43 PM3/19/08
to sage-...@googlegroups.com

Actually, I think zn_poly being used in the zeta function code reasonably
satisfies the constraint that the code be "usable from Sage", i.e,. provide
some clear added value to Sage.

William

Nick Alexander

unread,
Mar 19, 2008, 4:32:13 PM3/19/08
to sage-...@googlegroups.com
> Actually, I think zn_poly being used in the zeta function code
> reasonably
> satisfies the constraint that the code be "usable from Sage", i.e,.
> provide
> some clear added value to Sage.

I have immediate uses for the new zeta function code and would
appreciate it being included. It sounds like I should install
zn_poly and the new patch and referee.

Nick

Robert Bradshaw

unread,
Mar 19, 2008, 4:32:39 PM3/19/08
to sage-...@googlegroups.com

I don't think a full-blown wrapper is a requirement (this can be a
fair amount of work)--but given that you have to write a C program to
try it out how many people have actually tried to use it?

I'd be happy to provide a .pxd myself in the next couple days. My
only concern is how stable the API is.

- Robert


David Harvey

unread,
Mar 19, 2008, 4:57:12 PM3/19/08
to sage-...@googlegroups.com

On Mar 19, 2008, at 4:32 PM, Robert Bradshaw wrote:

>> If people feel more strongly that a wrapper is necessary, I would
>> prefer to withdraw both patches and come back with more code in a few
>> months, rather than put in a wrapper now.
>
> I don't think a full-blown wrapper is a requirement (this can be a
> fair amount of work)--but given that you have to write a C program to
> try it out how many people have actually tried to use it?

If you mean "how many people have tried to use the spkg", well no-one
has really used it except for me, via the zeta function code. Mike
Hansen also commented in december on ticket #1568 that he installed
the zn_poly spkg, and applied the zeta function patch, and all
doctests passed successfully. Therefore at least one person other
than me has computed at least one zeta function using this code, from
within Sage. The zeta function code #includes a zn_poly header file
and calls at least one zn_poly function to multiply polynomials, so
at that most basic level it seems to be working.

> I'd be happy to provide a .pxd myself in the next couple days. My
> only concern is how stable the API is.

The API is insufficiently stable to provide a .pxd file at this point.

david

Robert Bradshaw

unread,
Mar 19, 2008, 5:50:04 PM3/19/08
to sage-...@googlegroups.com

OK, that's good to know.

I still say an enthusiastic +1, it's being used in the zeta function
code and the code looks good to me.

- Robert

Kiran Kedlaya

unread,
Mar 19, 2008, 6:54:58 PM3/19/08
to sage-devel
Belated +1 from me. For me, the zeta function stuff is a killer app,
so this is important.

Besides tickets 1567 and 1568, David at some point opened ticket #793,
which requests a zeta_function method for HyperellipticCurves (at
least over F_p). This should include:

-- Whatever we provide in genus 1 (e.g., the SEA from PARI)
-- Drew's fast point-enumeration and generic-groups code (for small
genus and not-too-large p)
-- David's fast Frobenius code (for moderately large p)
-- Existing Monsky-Washnitzer code (for large genus and small p)
-- Compiled version(s) of the M-W code, which does not yet exist. In
particular, this should use zn_poly whenever possible.

That last one could be pretty useful in some applications. For
instance, I recently got an inquiry from Zeev Rudnick (via Mark
Watkins) about computing zeta functions of hyperelliptic curves of
genus ~50 over F_3; such computations should fit in 64-bit words, so
we should be able to clean Magma's clock.

Kiran

On Mar 19, 5:50 pm, Robert Bradshaw <rober...@math.washington.edu>
Reply all
Reply to author
Forward
0 new messages