FLINT and ARB using Visual Studio

[Brian Gladman]

Hi Bill and Fredrik,

Being in CORVID lockdown, I decided to have a look at building Arb with
Visual Studio. This has gone pretty well but I have a few issues on
which I would appreciate your advice.

I can compile and build Arb as a static library without any issues but
building Arb as a DLL fails because it needs two flint export ssymbols
that are not present in the FLINT DLL.

The first of these is '__flint_clz_tab' which I find can be switched on
in FLINT by defining NEED_CLZ_TAB when building FLINT. This should not
be a problem since I can modify my FLINT build to do this.

The second missing symbol is 'partitions_lookup', which is defined in
the FLINT file 'number_of_partitions.c' where it does have an export
definition. But this is not visible to Arb because it this symbol is not
declared in flint.h or, as far as I can tell, in any other exported
FLINT header.

So is this a symbol that FLINT should export and, if so, where should
the export declaration be placed?

Another issue I am keen to tidy up is library location and naming. I am
using simple names for the mpir dependent Windows builds: mpir, mpfr,
pthreads, ... and adding extensions 'lib and .dll accordingly.

For example, MPIR is in a root directory callled 'mpir' and within this
directory I have lib, dll, and exe sub-directories where I place the
mpir.lb, mpir.dll and mpir.exe build outputs. The same structure applies
to mpfr, pthreads but for FLINT I have been using the directory 'flint2'
rather than 'flint' and 'lib_flint.lib' and 'dll_flint.dll' for the
libraries (I am not sure why!)

In order to get a consistent approach now that I am adding Arb, I would
like to use the directory 'flint' rather than 'flint2' as the root
directory for FLINT and use the names 'flint.lib' and 'flint.dll' for
the libraries. But before I do this I would like to know if this change
is going to cause any issues for others (I am aware the the CMake FLINT
build depends on my FLINT build, which is why I am asking).

with my regards,

Brian

[Bill Hart]

Hi Brian,

I have fixed the three issues you pointed out to do with building an Arb dll. Thanks very much for pointing these out.

As for the flint dll, I don't mind what name is used. It's not a problem for me. This is technically flint2, not flint. But it is very many years since anyone has used the latter. I think it is fine to simply call it flint now.

Bill.

--
You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/b0d4c8ab-8f2e-b1a3-0327-72f8d9e0557a%40gmail.com.

[Bill Hart]

Hi Brian,

After making these changes, our cmake continuous integration fails. I think the problem is with this line:

FLINT_DLL const unsigned int partitions_lookup[128];

Is this only relevant for MSVC?

Bill.

[Cactus]

On Tuesday, 21 April 2020 00:28:28 UTC+1, Bill Hart wrote:

Hi Brian,

After making these changes, our cmake continuous integration fails. I think the problem is with this line:

FLINT_DLL const unsigned int partitions_lookup[128];

Is this only relevant for MSVC?

Bill.

Good morning Bill,

Unless something strange is going on, an Arb DLL, when built to import from a FLINT DLL, will require that this symbol is exported when the flint DLL is built.  

So the need for this symbol is not unique to MSVC but the mechanism for exporting it using the FLINT_DLL defines in source code is MSVC specific.

I believe that CMake builds flint on Windows using my flint MSVC build files.  But my guess is that it doesn't use my build system to regenerate these files when the build is updated (as in this case).  

If I am right about this, CMake may simply be failing because it is now using the old MSVC build files.   

Regenerating these build files or importing them from my flint repository may hence fix this issue (assuming that my repository is still in sync with the master repository).

    Brian

To unsubscribe from this group and stop receiving emails from it, send an email to mpir-...@googlegroups.com.

[Bill Hart]

Hi Brian,

Isuru Fernando believes he may have found the issue. I'll try reinserting the export as soon as we fix some other issues and see if it passes again.

For example, the fmpz_poly_factor test hangs at present. We haven't been able to figure out why.

We are using MSVC files built by CMake I think. So we don't need the ones from your repository for our CI (I think). (To be honest I am a bit lost with the current Windows maintenance, as others have largely been doing it for me. For example I just learned that they had disabled some of the tests in the CI because they were hanging or crashing. We are now down to one of each.)

Best Wishes,

Bill.

To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/8e593b67-2c8d-4285-beef-03795e087f6a%40googlegroups.com.

[Bill Hart]

Isuru reinserted the necessary line FLINT_DLL const unsigned int partitions_lookup[128];

Perhaps you could check if this works for you now.

Bill.

[Brian Gladman]

On 21/04/2020 16:50, 'Bill Hart' via mpir-devel wrote:
> Isuru reinserted the necessary line FLINT_DLL const unsigned int
> partitions_lookup[128];
>
> Perhaps you could check if this works for you now.

It builds fine. The tests give one error:

----------------------------------------
1523 tests:
1522 ran correctly
1 failed
fmpq_mat: t-minpoly ERROR 3 minpoly....
----------------------------------------

I have not yet retested Arb integration though. But it should be OK

Brian

[Bill Hart]

That's interesting. You have different errors with MSVC than we do.

We don't see this but we do see a different error.

--
You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/aecec632-87af-95b8-efc4-231d6c669387%40gmail.com.

[Brian Gladman]

On 28/04/2020 00:06, Bill Hart wrote:
> Dear Brian,
>
> MSVC 32 and 64 bit are now completely passing in out continuous
> integration with our trunk branch.
>
> Could you possibly test this and see if it works for you. We are of
> course using CMake to generate the build files and I know you have your
> own solution files.
>
> I'd be interesting to hear whether you still have the t-minpoly failure.
> Can you tell me what ERROR 3 is? That is not printed by our test code
> and it should in fact print a failure message with some diagnostic
> information if the test fails.
>
> I've opened a ticket for this issue until it is resolved, but there's
> nothing more I can do for now without further information as we are
> unable to reproduce it.

THe name change of config.h to flint-config.h was an initial issue for
me but this is fortunately easy to fix. I have built and tested the
trunk branch of FLINT on Windows x64 and all tests pass.

I am not running win32 builds these days as I consider them obsolescent
but there is no reason to believe that they wont build if anyone wants
to run them.

The fmpq_mat/t-minpoly test passes and is not a FLINT code issue. I have
tracked this down to a failure somewhere in my automted code to build
and run the tests (why it fails one just one of well over 1000 tests is
a puzzle but I suspect a naming anomaly of some kind). Anyway, I have
double checked by running this this test manually and it passes so the
ticcket can be closed.

Brian

[Bill Hart]

That's great news, thanks Brian.

Bill.

--
You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/cc2bae2e-fa20-e5a0-c500-fe17fa2b09ed%40gmail.com.

[Brian Gladman]

On 28/04/2020 15:44, 'Bill Hart' via mpir-devel wrote:

> That's great news, thanks Brian.

I fear that I have reported a complete pass too early as I missed the
fact that my test code had to abort two tests after running for too long
(over 120 seconds). These are:

fmpq_mpoly/t-gcd.exe
fmpq_mpoly/t-gcd_cofactors.exe

I have just run them both manually and neither finsihed within the five
minutes I gave them. Should I expect to take longer then this?

Brian

[Bill Hart]

They both take about 0.6s with our continuous integration. I don't know if @tthsqe12 would expect them to randomly hit a really hard case?

https://ci.appveyor.com/project/wbhart/flint2/builds/32491737/job/1os2ex9o3pe53bhe 

 

Bill.

--
You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/4c21c68f-510f-dc35-5723-e0480c10f038%40gmail.com.

[Cactus]

I am afraid that the situation on the Visual Studio build is a lot worse than I first thought.  It turns out that quite a few tests are hanging and timing out.

This is new so I have  I have revamped my test code to extend and fully record the timeouts and I am now in the process of running the tests again with a 5 minute per test time out.   

I'll report as soon as the test run has finished but I can already see a significant number of tests that are hanging and two (so far) are failing with access violations.

    Brian

[Brian Gladman]

On 28/04/2020 22:48, 'Bill Hart' via mpir-devel wrote:
> They both take about 0.6s with our continuous integration. I don't know
> if @tthsqe12 would expect them to randomly hit a really hard case?
>
> https://ci.appveyor.com/project/wbhart/flint2/builds/32491737/job/1os2ex9o3pe53bhe

I was under the impression that the Flint MSVC Windows build was no
longer dependent on my build system. But I am now suspicious that you
might still be relying on my build as a basis for the CMake build.

If I am right, I am afraid that the build will be out of date and will
not be correct unless the test build generator has been run to keep it
aligned with the Flint source code.

Otherwise its hard to explain why my results are different.

Brian

[Cactus]

Here are my results using a 5 minute timeout for each test:

Failed:

fmpz_mat: t-scalar_smod ... ERROR 3221225477

nmod_poly_mat: t-concat_vertical ... ERROR 3221225477

Timed out:

fmpq_mpoly: t-gcd

fmpq_mpoly: t-gcd_cofactors

fmpz_mod_poly: t-compose_mod_brent_kung_vec_preinv_threaded

fmpz_mpoly: t-divides

fmpz_mpoly: t-divides_heap_threaded

fmpz_mpoly: t-gcd

fmpz_mpoly: t-gcd_berlekamp_massey

fmpz_mpoly: t-gcd_berlekamp_massey_threaded

fmpz_mpoly: t-gcd_brown_threaded

fmpz_mpoly: t-gcd_cofactors

fmpz_mpoly: t-gcd_zippel

fmpz_mpoly: t-mpolyuu_divides_threaded

fmpz_mpoly: t-mul

fmpz_mpoly: t-mul_array_threaded

fmpz_mpoly: t-mul_heap_threaded

fmpz_poly: t-taylor_shift_divconquer

nmod_mpoly: t-divides

nmod_mpoly: t-divides_heap_threaded

nmod_mpoly: t-gcd

nmod_mpoly: t-gcd_brown_threaded

nmod_mpoly: t-gcd_cofactors

nmod_mpoly: t-gcd_zippel

nmod_mpoly: t-mpolyn_divides_threaded

nmod_mpoly: t-mul

nmod_mpoly: t-mul_array_threaded

nmod_mpoly: t-mul_heap_threaded

nmod_vec: t-discrete_log_pohlig_hellman

qsieve: t-factor

thread_pool: t-thread_pool

ulong_extras: t-n_urandintulong_extras: t-n_urandint

The two failures are both access violations.  

     Brian

 

[Bill Hart]

You may be right, but I don't see where your files are in our repo:

https://github.com/wbhart/flint2 

This would be better discussed on the flint list as Isuru Fernando could chime in. He understands CMake much better than I do. I don't want to make assertions about it, as they often prove to be wrong.

Perhaps you could open a ticket with your observations on my Flint repo?

Bill.

--

You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mpir-devel/8ae7122f-049d-4099-8e4f-239693bf6fe6%40googlegroups.com.

[Brian Gladman]

On 28/04/2020 23:28, 'Bill Hart' via mpir-devel wrote:
> You may be right, but I don't see where your files are in our repo:
>
> https://github.com/wbhart/flint2 
>
> This would be better discussed on the flint list as Isuru Fernando could
> chime in. He understands CMake much better than I do. I don't want to
> make assertions about it, as they often prove to be wrong.
>
> Perhaps you could open a ticket with your observations on my Flint repo?

I don't know a lot about CMake but I haave taken a look and, as far as I
can see, it is no longer dependent on my build files.

But the Appveyor script in the Flint root directory DOES run my build
system and is not hence testing the output of the CMake build.

So if you are relying on Appveyor you will be running my out of date
build and judging its output rather than the CMake build output.

I'll ask about this on your repo.

Brian

[Cactus]

I have updated my build options to match those used in the CMake build and the two earlier access violation failures no longer occur.  

But all the hanging tests are the same. I used a one minute timeout, which I think is reasonable.

 

[Samuel Lelièvre]

Tue 2020-04-28 22:49:09 UTC, Cactus:
>
> On 28/04/2020 23:28, 'Bill Hart' via mpir-devel:

> > You may be right, but I don't see where your files are in our repo:
> >
> > https://github.com/wbhart/flint2  
> >

> > [...]

> >
> > Perhaps you could open a ticket with your observations on my Flint repo?
>

> [...]

>
> I'll ask about this on your repo.
>
>    Brian

Link to the ticket opened by Brian:

  https://github.com/wbhart/flint2/issues/675