Re: [linbox-use] unable to install latest stable release

[Clement Pernet]

Dear Robert,

From your log file, I would suggest that you add all directories containing the dynamic libraries of
givaro and gmp to the LD_LIBRARY_PATH environment:

export
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/schabr/lbinst2/Deps/Givaro/lib:/home/schabr/lbinst2/Deps/GMP/lib

As for the problems of documentation, it is indeed currently in a terrible state, and as we are
progressively changing the design of the library, it is not up to date.
To answer your precise question: fflas-ffpack was included in v1.1.7 hence it was not a mandatory
pacakge. Since then , especially in the lastest developper release, it has been put aside as a
self-standing package, and became a mandatory dependency.

linbox-maple interface has not been maintained since a while ago. It will very likely cause problems.
Sorry for the inconvenience, do not hesitate to contact us on this mailing list for further questions.

Regards.
Cl�ment

Le 25/09/2012 16:09, Robert Schabinger a �crit :
> Hello all,
>
> First let me make it clear that I have read all of the documentation that I could find, both on the
> linbox website and in the doc folder that comes with the v.1.1.7 download (and some on this forum
> which doesn't appear to exist elsewhere). Also, I was able to install ATLAS (latest version), GMP
> (latest version), and Givaro-3.3.6 in a folder in my home directory (I am running the latest version
> of Kubuntu on a 2.5 GHz Pentium Core 2 Duo laptop) with no obvious problems (100% of 'make check'
> tests passed).
>
> Unfortunately, my numerous attempts to install linbox in a variety of different ways have all
> failed. Let me start by listing the issues that I have had with the documentation. For one thing, as
> far as I can tell, this forum is the only place that makes any mention of the fact that Givaro-3.3.6
> is the only version currently available that can be detected by linbox-1.1.7. Furthermore, the
> documentation on the webpage for v.1.1.7 and the documentation that comes with the v.1.1.7 distro
> are not only very different from one another but, as far as I can tell, both are incorrect. The
> documentation that appears on the webpage for v.1.1.7 is, I guess, intended for the latest developer
> release; it says that the installation of fflas-ffpack is mandatory but, in reality, the v.1.1.7
> configuration script does not even recognize the option --with-fflas-ffpack=<pathofinstallation>.
> The installation guide that comes with the distro (under doc/install-dist.html) actually appears to
> be outdated; it lists Givaro as an optional package which appears not to be the case for linbox-1.1.7.
>
> Eventually, I did get the configuration script to run. However, when I ran 'make' the program
> promptly quit with an error. Actually, I think that the error, whatever it is, probably already
> shows up in the configuration step since using the --enable-optimization option causes the script to
> output: 'checking best threshold for Strassen-Winograd matrix multiplication . . . problem'. I've
> attached the configuration log file so that you can have a look. I've also attached a text file with
> the explicit outputs of the configuration and make steps just in case that makes it easier to tell
> what I did wrong.
>
> I should probably also tell you what my end goal is so that you can tell me if what I'm trying to do
> is not what I should be doing. At the moment, I have several very large very sparse linear systems
> with integer coefficients which I need to solve. I have their matrix representations stored in Maple
> and was planning to use linbox via its Maple interface to solve them. However, two things that I
> have noticed are making me nervous. It seems that, in the experimental auto-install shell script
> there is no option which builds a linbox-Maple interface. Does this mean the the linbox-Maple
> interface doesn't actually work (I have Maple v.13) or is no longer maintained? Also, I can't help
> but wonder: are the other developer releases going to be much faster for my intended application
> than the latest stable release? If I need to use Sage instead of Maple or if a different version of
> linbox would give me a substantial boost in speed, please let me know and I'll try to install
> accordingly. Also, it would be good to know which extra packages are actually worth installing for
> my intended application.
>
> Finally, let me say that I just started using Linux this year and have almost no knowledge of c,
> fortran, or scripting. So it is very likely that my failures are the result of a simple mistake that
> a more advanced user would never make. I am at my wits end and would welcome any suggestions as to
> how I might proceed because I really, really, want to use linbox. Obviously, I would be more than
> happy to provide additional information or perform additional tests if it would help.
>
> Sincerely,
>
> Rob Schabinger
>
> --
> You received this message because you are subscribed to the Google Groups "linbox-use" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/linbox-use/-/h7_4xRwmehUJ.
> To post to this group, send email to linbo...@googlegroups.com.
> To unsubscribe from this group, send email to linbox-use+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/linbox-use?hl=en.

[Robert Schabinger]

Hi Clement,

Thank you very much for your response. I will try reinstalling the code as soon as I get the chance. It is particularly helpful to know that the Maple-LinBox interface is no longer maintained. Is the situation any better for Sage-LinBox interface? It would also be very helpful to know what sort of build I should attempt for my intended application. Would you recommend installing the latest developer version (as opposed to the latest stable version)? Which of the optional external packages are important for good LinBox performance? I have managed to successfully install all of the ones I could find. I couldn't find LiDiA because the webpage seems to be down but maybe this is not important... 

Best,

Rob

On Monday, October 8, 2012 3:48:09 PM UTC+2, Clement Pernet wrote:

Dear Robert,

From your log file, I would suggest that you add all directories containing the dynamic libraries of
givaro and gmp to the LD_LIBRARY_PATH environment:

export
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/schabr/lbinst2/Deps/Givaro/lib:/home/schabr/lbinst2/Deps/GMP/lib

As for the problems of documentation, it is indeed currently in a terrible state, and as we are
progressively changing the design of the library, it is not up to date.
To answer your precise question: fflas-ffpack was included in v1.1.7 hence it was not a mandatory
pacakge. Since then , especially in the lastest developper release, it has been put aside as a
self-standing package, and became a mandatory dependency.

linbox-maple interface has not been maintained since a while ago. It will very likely cause problems.
Sorry for the inconvenience, do not hesitate to contact us on this mailing list for further questions.

Regards.

Cl�ment

Le 25/09/2012 16:09, Robert Schabinger a �crit :

[Volker Braun]

Linbox is definitely alive and well in Sage. If you are having troubles getting the prerequisites together I'd definitely suggest you install Sage, which will compile a functional linbox for you.

[B Saunders]

Hi, Robert, here's a bit more input on this thread:

You can ignore LiDiA.  That optional dependency was for a few experiments and is now dormant.
The only optional dependencies that add meaningfully are NTL and lapack.  NTL contributes to some algorithms that use the FFT and lapack contributes
to one of the methods for solving a rational linear system (integer matrix and right hand side, rational solution vector).

As Volker said, getting LinBox installed via Sage and using Sage for access to some of LinBox is a good way to go.

Alternatively, if you have the gnu autotools and use the developer experimental release (script on website for that), then if your work induces any changes/additions/corrections to LinBox, you easily get them by a svn update.

Best,
-dave

--
Prof. B. David Saunders, 302-831-6238, CIS Department, University of Delaware

To view this discussion on the web visit https://groups.google.com/d/msg/linbox-use/-/2vA6fa8fkYAJ.

[Robert Schabinger]

Hi All,

First of all, thank you all so much for your responses. I started to take a look at Sage but, before I even really got started, a colleague of mine was able to help me install LinBox 1.3.2 as a c++ program. Evidently, LinBox does not install correctly (without additional human interference) if you try to install ATLAS yourself by downloading the latest distro from their webpage; in order to get LinBox to install in a straightforward fashion, I had to first download and install a pre-built version of ATLAS using my package manager.

Although most of the people who posted on this thread seem to be either seasoned users or developers, I want to first explain what I had to do to get the basic LinBox tutorial program (suggested in the thread "linbox newbie with configure and gcc compile problem"):

#include <iostream> 
#include <linbox/field/modular.h> 
#include <linbox/matrix/blas-matrix.h> 
#include <linbox/solutions/det.h> 

using namespace LinBox; 

// Usage: Assuming that you are in the directory where the executable sits, simply type

// ./<executable_name>  <  <name_of_input_file_in_one_of_several_standard_formats> at the command line.

main() 
{ 
    typedef Modular<double> Field; 

    typedef BlasMatrix<Field> Matrix; 
    Field F(65521); 
    Matrix A(F); A.read(std::cin); 
    Field::Element d; 
    Method::BlasElimination M; 
    det(d, A, M); 
    F.write(std::cout << "the determinant is ", d) << std::endl; 
}

 

to compile/link and run correctly. I found the process highly non-obvious and I'm guessing that other c++/LinBox beginners will be similarly confused. If you aren't interested in the details or are already a LinBox expert (or both I guess) please skip to the question at the end of this post.

For me, the above program does compile if one first exports the paths to libraries installed in non-standard locations  (see Clement's post above for details) and includes the following statements after g++ -o <executable_name> <source_filename.cpp>: -I<path_of_non-standard_Givaro_installation>/include -L<path_of_non-standard_Givaro_installation>/lib -lgivaro -I<path_of_non-standard_GMP_installation>/include -L<path_of_non-standard_GMP_installation>/lib -lgmp -lgmpxx -I<path_of_non-standard_LinBox_installation>/include -L<path_of_non-standard_LinBox_installation>/lib -llinbox -I/usr/include -L/usr/lib  -lblas

HOWEVER, at least for me, it does not run correctly. There are two problems with the code related to the form in which my matrices are exported from Maple and Mathematica. I used a format called the rectangular matrix market format. Just to be clear, this format was not chosen at random. On the contrary, it seems to be quite standard and it is supported by both Maple and Mathematica. After quite a bit of frustration I finally came up a hack which seems to run correctly and produce the desired output:

#include <iostream>

#include <string>

#include <fstream>

#include <stdio.h>

#include <linbox/field/modular.h> 

#include <linbox/matrix/blas-matrix.h> 

#include <linbox/solutions/det.h> 

using namespace LinBox;

main() 

{   

    int fieldchar = 65521;  

  

    typedef Modular<double> Field; 

    typedef BlasMatrix<Field> Matrix; 

    Field F(fieldchar);

    Matrix A(F);

    Field::Element d; 

    Method::BlasElimination M;

    

    char buffer[10];

    char* tmpf;

    tmpf = std::tmpnam(buffer);

    std::ofstream firstfile;

    std::ifstream secondfile;

    std::string firstline;

    std::string restlines;

    

    firstfile.open(tmpf);

    std::cin.ignore(255,'\n');

       

    std::getline(std::cin,firstline);

    firstfile << firstline << " 0" << std::endl;

    

    while(!std::cin.eof())

    {

      getline(std::cin,restlines);

      firstfile << restlines << std::endl;

    }

    

    secondfile.open(tmpf);

    std::istream& finalfile = secondfile;

    

    A.read(finalfile);

    det(d, A, M);

    F.write(std::cout << A << std::endl << "has determinant ", d) << " in the finite field F(" << fieldchar << ")." << std::endl;

} 

For those of you  who are unfamiliar, here is a sample rectangular matrix market file (from Maple):

%%MatrixMarket matrix array integer general

3 3

0

0

1

2

0

3

0

1

0

The first problem with the original tutorial program is that the comment line at the top of matrix market files needs to be ignored (using the ignore member function of cin is how I did it in this case). The second problem is less obvious but can be immediately diagnosed if one prints the values stored in A (as I did in my modification of the tutorial program's code). Basically, the function read (as in A.read()) expects another character after the two integers specifying the size of the matrix. To solve this problem, I had to write the non-trivial content of the input rectangular matrix market file to a temporary file, changing in the process the first line of the file (e.g. 3 3 becomes 3 3 0). In short, the read function is expecting an input that looks something like

3 3 0

0

0

1

2

0

3

0

1

0

where, I believe, one could replace the line 3 3 0 with 3 3 x. It is worth pointing out that, if one is using Mathematica, there is an extra comment line at the top of rectangular matrix market files which also needs to be ignored. This can be achieved simply by inserting  the line std::cin.ignore(255,'\n'); between the lines std::cin.ignore(255,'\n'); and std::getline(std::cin,firstline); .

Now that I've probably bored 99% of the people who clicked on this post to death, I have a question of my own. Has the code in the "examples" section of the webpage actually been tested? If so, on exactly what sort(s) of input? If not, would it be possible to obtain versions of the example programs that compile, run, and produce the desired output (for some given precisely specified input matrix format or formats)?

Best Regards,

Rob

[B Saunders]

You've uncovered an embarrassing lapse in LinBox.  The BlasMatrix read() function is not up to date so that it will read matrix market format among others.

The det and rank examples in the examples directory use a different matrix representation (and hence reader) and do fine with MatrixMarket formats.  For instance, I just checked the examples

det ../tests/data/matrix-market-array.matrix
det ../tests/data/matrix-market-coordinate.matrix
rank ../tests/data/matrix-market-array.matrix
rank ../tests/data/matrix-market-coordinate.matrix

which work fine.

I'll update the BlasMatrix read() function.

Best,
-dave

--
Prof. B. David Saunders, 302-831-6238, CIS Department, University of Delaware

To view this discussion on the web visit https://groups.google.com/d/msg/linbox-use/-/i48M4wU-7MsJ.

[Robert Schabinger]

Hi Dave,

Thanks for the rapid response. Unfortunately, it seems that there may be further problems with the examples. I just copied Solve.C (the example that I am most interested in studying and understanding) and tried to compile it in exactly the same way that I compiled the tutorial program. In this case, the code doesn't even compile. I get the following error message:

solve.cpp: In function ‘int main(int, char**)’:

solve.cpp:15:16: error: ‘LinBox::commentator’ does not have class type

solve.cpp:16:16: error: ‘LinBox::commentator’ does not have class type

solve.cpp:17:16: error: ‘LinBox::commentator’ does not have class type

Does this just mean that I have to #include an additional header file or something like that? Any tips you can give me would be greatly appreciated.

Sincerely,

Rob

[Jean-Guillaume Dumas]

Robert Schabinger wrote, on 24/10/2012 21:02:
> Hi Dave,
>
> Thanks for the rapid response. Unfortunately, it seems that there may
> be further problems with the examples. I just copied Solve.C (the
> example that I am most interested in studying and understanding) and
> tried to compile it in exactly the same way that I compiled the
> tutorial program. In this case, the code doesn't even compile. I get
> the following error message:
>

> solve.cpp: In function �int main(int, char**)�:
> solve.cpp:15:16: error: �LinBox::commentator� does not have class type
> solve.cpp:16:16: error: �LinBox::commentator� does not have class type
> solve.cpp:17:16: error: �LinBox::commentator� does not have class type

>
> Does this just mean that I have to #include an additional header file
> or something like that? Any tips you can give me would be greatly
> appreciated.
>
> Sincerely,
>
> Rob

Hello,
this one is a change of version for commentator,
with one version you need to use e.g.
"LinBox::commentator().setMaxDetailLevel (-1);" where in preceeding
versions you would use "LinBox::commentator.setMaxDetailLevel (-1);"
without the parenthesis after commentator.

I will investigate this version mismatch ...
Best,


--
Jean-Guillaume Dumas.
____________________________________________________________________
Jean-Guill...@imag.fr T�l.: +33 476 514 866
Universit� Joseph Fourier, Grenoble I. Fax.: +33 476 631 263
Laboratoire Jean Kuntzmann, Math�matiques Appliqu�es et Informatique
51, avenue des Math�matiques. LJK/IMAG - BP53. 38041 Grenoble FRANCE
http://ljk.imag.fr/membres/Jean-Guillaume.Dumas
____________________________________________________________________

[Robert Schabinger]

Hi All,

I implemented Jean-Guillaume's suggestion but the compiler is still unhappy. Now I get:

/tmp/cc7hBlpM.o: In function `LinBox::solveIML(std::vector<Givaro::Integer, std::allocator<Givaro::Integer> >&, Givaro::Integer&, LinBox::BlasMatrix<LinBox::PID_integer> const&, std::vector<Givaro::Integer, std::allocator<Givaro::Integer> > const&, LinBox::IMLTraits const&)':

solve.cpp:(.text+0x4d62): undefined reference to `nonsingSolvLlhsMM'

solve.cpp:(.text+0x4db6): undefined reference to `maxMagnMP'

solve.cpp:(.text+0x4df4): undefined reference to `RNSbound'

solve.cpp:(.text+0x4e15): undefined reference to `findRNS'

solve.cpp:(.text+0x4fe5): undefined reference to `nonsingSolvRNSMM'

solve.cpp:(.text+0x5191): undefined reference to `certSolveMP'

solve.cpp:(.text+0x5228): undefined reference to `certSolveRedMP'

/tmp/cc7hBlpM.o: In function `clapack_dgetrf':

solve.cpp:(.text.clapack_dgetrf[clapack_dgetrf]+0x3f): undefined reference to `dgetrf_'

/tmp/cc7hBlpM.o: In function `clapack_dgetri':

solve.cpp:(.text.clapack_dgetri[clapack_dgetri]+0x52): undefined reference to `dgetri_'

collect2: ld returned 1 exit status

Ideas? If it helps, for the moment I am just interested in trying out the fastest implementation you have for sparse integer matrices and the fastest implementation you have for dense integer matrices. In both cases, it is possible that the matrices involved are rank deficient.

Best,

Rob

[Jean-Guillaume Dumas]

Hello,
this looks like missing/misordered libraries at link time,
can you post the command line that produced the error (or the ouput of
/bin/linbox-config --prefix --cflags --libs),
and possibly the config.log ?

Best,

--
Jean-Guillaume Dumas.
____________________________________________________________________

Jean-Guill...@imag.fr T�l.: +33 476 514 866
Universit� Joseph Fourier, Grenoble I. Fax.: +33 476 631 263
Laboratoire Jean Kuntzmann, Math�matiques Appliqu�es et Informatique
51, avenue des Math�matiques. LJK/IMAG - BP53. 38041 Grenoble FRANCE
http://ljk.imag.fr/membres/Jean-Guillaume.Dumas
____________________________________________________________________

[Robert Schabinger]

Absolutely. I actually tried compiling two different ways. First I used the same series of commands that worked for the tutorial program:

 g++ -o solve solve.cpp -I/home/schabr/lbinstall/Givaro/include -L/home/schabr/lbinstall/Givaro/lib -lgivaro -I/home/schabr/lbinstall/GMP/include -L/home/schabr/lbinstall/GMP/lib -lgmp -lgmpxx -I/home/schabr/linbox/include -L/home/schabr/linbox/lib -llinbox -I/usr/include -L/usr/lib -lblas

This is what produced the output that I included in my last post. After seeing that the output of /bin/linbox-config --prefix --cflags --libs is 

-I/home/schabr/linbox/include -I/home/schabr/lbinstall/GMP/include -I/home/schabr/lbinstall/NTL/include -I/home/schabr/lbinstall/Givaro/include   -L/home/schabr/linbox/lib -llinbox   -L/home/schabr/lbinstall/NTL/lib -lntl -L/home/schabr/lbinstall/Givaro/lib -lgivaro  -L/home/schabr/lbinstall/GMP/lib -lgmpxx -lgmp -lblas -llapack

I deduced that this sequence of commands is probably what I should actually be using to link. I tried to compile once again and this is what happens:

schabr@Hal9000:~/linbox$ g++ -o solve solve.cpp -I/home/schabr/lbinstall/Givaro/include -L/home/schabr/lbinstall/Givaro/lib -lgivaro -I/home/schabr/lbinstall/GMP/include -L/home/schabr/lbinstall/GMP/lib -lgmp -lgmpxx -I/home/schabr/linbox/include -L/home/schabr/linbox/lib -llinbox -I/home/schabr/lbinstall/NTL/include -L/home/schabr/lbinstall/NTL/lib -lntl -lblas -llapack

/tmp/ccsP9Fql.o: In function `LinBox::solveIML(std::vector<Givaro::Integer, std::allocator<Givaro::Integer> >&, Givaro::Integer&, LinBox::BlasMatrix<LinBox::PID_integer> const&, std::vector<Givaro::Integer, std::allocator<Givaro::Integer> > const&, LinBox::IMLTraits const&)':

solve.cpp:(.text+0x4d62): undefined reference to `nonsingSolvLlhsMM'

solve.cpp:(.text+0x4db6): undefined reference to `maxMagnMP'

solve.cpp:(.text+0x4df4): undefined reference to `RNSbound'

solve.cpp:(.text+0x4e15): undefined reference to `findRNS'

solve.cpp:(.text+0x4fe5): undefined reference to `nonsingSolvRNSMM'

solve.cpp:(.text+0x5191): undefined reference to `certSolveMP'

solve.cpp:(.text+0x5228): undefined reference to `certSolveRedMP'

collect2: ld returned 1 exit status

As far as I can tell, the output is exactly the same as it was before. I've attached the LinBox config.log file so that you can have a look.

Best,

Rob

Jean-Guill...@imag.fr                   T�l.: +33 476 514 866
Universit� Joseph Fourier, Grenoble I.         Fax.: +33 476 631 263
Laboratoire Jean Kuntzmann, Math�matiques Appliqu�es et Informatique
51, avenue des Math�matiques. LJK/IMAG - BP53. 38041 Grenoble FRANCE
http://ljk.imag.fr/membres/Jean-Guillaume.Dumas
____________________________________________________________________

[Robert Schabinger]

Google groups will not let me attach the log file...I guess I'll send it to you by email.

--Rob

[Jean-Guillaume Dumas]

Well,
apparently you don't have the lapack error anymore. (maybe make solve
would be correct for this too),
and apparently your are using "solveIML" but not linking with it (from
the config log I guess just adding "-liml" to your command line might do
it ...
let me know if it works.

Best,

--
Jean-Guillaume Dumas.
____________________________________________________________________

Jean-Guill...@imag.fr T�l.: +33 476 514 866
Universit� Joseph Fourier, Grenoble I. Fax.: +33 476 631 263
Laboratoire Jean Kuntzmann, Math�matiques Appliqu�es et Informatique
51, avenue des Math�matiques. LJK/IMAG - BP53. 38041 Grenoble FRANCE
http://ljk.imag.fr/membres/Jean-Guillaume.Dumas
____________________________________________________________________

[Robert Schabinger]

Yep, that was the problem. So I guess that IML is a mandatory dependency after all? Or is it just that it effectively becomes a mandatory dependency if you configure LinBox with it enabled? Anyway, when I get some time, I'll try to actually do something non-trivial with your code now that everything seems to be in its right place. Thank you all again for the extremely helpful posts.

Best,

Rob

Jean-Guill...@imag.fr                   T�l.: +33 476 514 866
Universit� Joseph Fourier, Grenoble I.         Fax.: +33 476 631 263
Laboratoire Jean Kuntzmann, Math�matiques Appliqu�es et Informatique
51, avenue des Math�matiques. LJK/IMAG - BP53. 38041 Grenoble FRANCE
http://ljk.imag.fr/membres/Jean-Guillaume.Dumas
____________________________________________________________________

[B Saunders]

Attached is a patch to blas-matrix.inl to fix so that it's read function handles matrix market formats among others.

Best,
-dave

--
You received this message because you are subscribed to the Google Groups "linbox-use" group.

To view this discussion on the web visit https://groups.google.com/d/msg/linbox-use/-/sr_J2rinI9QJ.