singular fails to build with the latest ntl (10+)

[François]

Victor Shoup has released ntl-10.0.0 and then 10.1.0. We discovered on sage-devel that singular (4.0.3p3 but it probably go all the way back to 3.x) cannot build against that version of ntl:

NTLconvert.cc: In function ‘CanonicalForm convertZZ2CF(const NTL::ZZ&)’:

NTLconvert.cc:509:38: error: invalid static_cast from type ‘const raw_ptr {aka _ntl_gbigint_body* const}’ to type ‘long int*’

      static_cast<long *>( a.rep.rep ); // what about NTL7?

                                     ^

Makefile:1205: recipe for target 'NTLconvert.lo' failed

It is part of a bigger block that is already separated by version of ntl

   const long * rep =

#if NTL_MAJOR_VERSION <= 6

     static_cast<long *>( a.rep );

#else

     static_cast<long *>( a.rep.rep ); // what about NTL7?

#endif

According to Victor, singular is using an undocumented/internal interface. Victor has expressed the willingness to provide a documented and stable interface for singular to use https://groups.google.com/d/msg/sage-devel/nCXt1cjbTJ8/U6jB7lq-BQAJ provided of course, he understand the exact needs.

[han...@mathematik.uni-kl.de]

On Tue, Oct 18, 2016 at 06:51:25PM -0700, François wrote:
> Victor Shoup has released ntl-10.0.0 and then 10.1.0. We discovered on
> sage-devel that singular (4.0.3p3 but it probably go all the way back to
> 3.x) cannot build against that version of ntl:

fixed with 861b7899904e3bbe3e8fe49fc0dd11ef9f5f238e
(as suggested in sage-devel)
...

>
> According to Victor, singular is using an undocumented/internal interface.
> Victor has expressed the willingness to provide a documented and stable
> interface for singular to

> use https://groups.google.com/d/msg/sage-devel/nCXt1cjbTJ8/U6jB7lq-BQAJ

> provided of course, he understand the exact needs.

The problem is a fast/convinient conversion between long integers in
factory of type CanonicalForm (which is in principle a mpz_t number)
and long integers in NTL of type ZZ (which is, if I understood it
correctly, analogue to the internal of a mpz_t).
(There are routines to access the mpz_t representation of a CanonicalForm in
factory and to construct a CanonicalForm from a mpz_t)

The current implementation uses a string representation in basis
16 (NTL->factory) resp. 10 (factory->NTL),
which is neither nice nor fast, but was easy to implement and works:

mpn_get_str converts the NTL-ZZ-number to a string
and mpz_init_set_str creates then the GMP-mpz_t-number (and then the
CanonicalForm number)

The reverse uses a string with basis 10: mpz_get_str from factory to
string and conv from string to NTL ZZ

[Jean-Pierre Flori]

This means that once Victor provides ZZ to mpz_t routines it will be trivial to get a fast conversion btw Singular (and GMP) and NTL.

Do you plan on making a 4.0.3p4 or 4.0.4 release?
That would be great to simplify what Sages ships, currently we backport some critical commits to 4.0.3p3.

Best,
JP

[han...@mathematik.uni-kl.de]

On Wed, Oct 19, 2016 at 04:33:47AM -0700, Jean-Pierre Flori wrote:
>
> This means that once Victor provides ZZ to mpz_t routines it will be
> trivial to get a fast conversion btw Singular (and GMP) and NTL.

yes.

>
> Do you plan on making a 4.0.3p4 or 4.0.4 release?
> That would be great to simplify what Sages ships, currently we backport
> some critical commits to 4.0.3p3.

There is already a 4.0.3p4 (which includes everything except THIS fix for
NTL 10). It is not broadly announced because it is the test for the
upcoming 4.0.4 release which should be released really soon.
See
ftp://www.mathematik.uni-kl.de/pub/Math/Singular/src/4-0-3/singular-4.0.3p4.tar.gz
and for the news:
http://www.mathematik.uni-kl.de/~hannes/s/sing_2499.htm

[Jean-Pierre Flori]

Great!
Last time I visited the tarball page it was not ther.

[Victor Shoup]

Hi all,

I was off the grid for a few days.

Let me say a couple of things.

First, I think a quick fix to the problem at hand is to replace

static_cast<long*> to (long*) at lines 507 and 509 of NTLconvert.cc.

I didn't really change that much, the void* type got changed to a different

pointer type that helped catch some bugs in the modifications I made

elsewhere.

Second, I will update NTL soon to provide better "low level"  conversion routines.

I have been hesitant to do this previously, because in general, NTL header files

do not include <gmp.h>, and I prefer to keep things that way, for a variety

of reasons.

However, one needs <gmp.h> just to get the mp_limb_t typedef.

But what I will do is make a special header file just for these conversion

routines, which will include <gmp.h> (assuming NTL_GMP_LIP=on).

This will require that client code using this header needs to make sure

that the environment is set up to find <gmp.h>.

The signature for these functions will be something like

   void ZZRawConvertFrom(const ZZ& x, zzigit_t *p);

   // writes the x.size() zzigits of x to location p, starting with the low-order zzigit

   void ZZRawConvertTo(ZZ& x, const zzigit_t *p, long n);

   // initialize x with p[0..n-1], where p[0] is the low-oder zzigit

Here, zzigit_t will be a typedef to mp_limb_t (if NTL_GMP_LIP=on)

or to unsigned long (if NTL_GMP_LIP=off).

I'll also define a macro NTL_BITS_PER_ZZIGIT_T (which may

be less than NTL_ZZ_NBITS).

Granted, mp_limb_t is usually set to unsigned long, but I guess this is

not always the case for some platforms.  Also, this interface will work

even if NTL is not built with GMP.

[Victor Shoup]

Actually, the signature for ZZRawConvertFrom will mpre likely be

   const zzigit *ZZRawConvertFrom(const ZZ& x)

which will simply return a pointer to the limbs of x.

You can use x.size() to see how many limbs there are.

If x.size() == 0, then the return value may or may not be null.

This is more general and efficient than the other interface I proposed.

[Jean-Pierre Flori]

Hi Hans,

Just to let you know that Victor released a new NTL version including a public interface to get limbs of a multiprecision integer.
See details at https://groups.google.com/d/msg/sage-devel/Df4hL8fPTj0/TG6-k6CKBwAJ

Best,
JP