Re: Perl and Stats

86 views
Skip to first unread message

Jonathan Leto

unread,
Oct 15, 2008, 2:58:18 AM10/15/08
to Michael Roberge, math-g...@googlegroups.com
Hello Michael,

Thanks for your interest in Math::GSL. The short answer is that Math::GSL *will* require a compiler (but is it known to work on Linux/BSD/Solaris/Darwin/Windows) and that Statistics::Descriptive does seem to be a bit stagnant.

I think the most important factor is, what tasks do you want to accomplish? What Statistics::Descriptive does is give you a simple-to-use OO interface to calculating basic descriptive statistics. And that's it.

So if you think that all you'll ever want to calculate is min/mode/mean/variance/deviation, then it will be fine. If you find that you need skew, kurtosis, autocorrelation or weighted averages then you are out of luck. Math::GSL::Statistics has these and plenty more. And they are written in C, so they are
quite a bit faster. This allows you to use Perl to handle computational problems that were previously out of reach because they would be too damn slow.

Also, Math::GSL::CDF is where many Cumulative Distribution Functions live and if you find that you want to generate random numbers according to different
distributions, there is Math::GSL::RNG. Hundreds of special functions (like Bessel, Airy, Gamma, Legendre) are available from Math::GSL::SF. And you
can do numerical integration and derivatives of Perl subroutines, but I digress...

So the long answer is that Math::GSL is under active development and you are welcome to join the math-gsl-dev google group [0] and ask questions! I will be representing the The Perl Foundation and Math::GSL at the Google Summer of Code 2008 Mentor Summit this month,
where I hope to brainstorm with others about Math::GSL and open-source scientific computing in general. 

Cheers,



--
[---------------------]
 Jonathan Leto
jal...@gmail.com


miker

unread,
Oct 15, 2008, 9:47:04 AM10/15/08
to math-gsl-dev
Thanks Jonathan for getting back with me. I will see about getting
into the google group as soon as possible, until then I hope you can
continue to help via email.

I think your assessment of the situation is dead on. In other words
Statistics::Descriptive does what it does and that's it. I was able to
make use of it immediately so it does have some value to me. However
looking forward I think I am interested in getting the Math::GSL
working since it appears to have more function and a better future
beyond just simple stats. To that end I down loaded the 0.12 version
yesterday, stored the extracted tar ball into my personal afs space. I
copied the Math-GSL-0.12/examples/correlation over to a new working
location and added the following lines so that perl would know where
to find my locally installed package libs. I have done similar things
before successfully with other packages(ie Statistics::Descriptive).
At this point I have not done any compiles, are the compiles required
because of all the .i & .c files in the top directory? Are those file
the actual GSL functions in C?

use lib "/..my_AFS_path.../bin/PerlModules/Math-GSL-0.12/lib"; <==
this prepends this path to @INC var so that 'use' cmd can find what it
needs.

But perl keeps saying it failed to find the loadable object
math::GSL::Statistics in @INC.
I have tried all sorts of variations on the above 'use lib' to no
avail by asserting additional statements in hopes that I just didn't
have the path correct.
It's kind of strange to me because I can see the Statistics.pm down in
lib/Math/GSL.

use lib "/..my_AFS_path.../bin/PerlModules/Math-GSL-0.12/"
use lib "/..my_AFS_path.../bin/PerlModules/Math-GSL-0.12/lib"
use lib "/..my_AFS_path.../bin/PerlModules/Math-GSL-0.12/lib/Math"
use lib "/..my_AFS_path.../bin/PerlModules/Math-GSL-0.12/lib/Math/GSL"

I also tried different combo of.... and still no luck
use Math::GSL::Statistics
use Math
use Math::GSL
use Statistics

To be honest, I am not the most advanced user when it comes to
installing packages. And I have very little experience doing compiles.
Do you have a primer/readme on the steps required to get the Math::GSL
packages installed and compiled? I assume the reason things are not
working currently is because I have not actually done the required C
compiles? But maybe there is more to it than just that. At this point
I am kind of lost.

On Oct 15, 2:58 am, "Jonathan Leto" <jal...@gmail.com> wrote:
> Hello Michael,
>
> Thanks for your interest in Math::GSL. The short answer is that Math::GSL
> *will* require a compiler (but is it known to work on
> Linux/BSD/Solaris/Darwin/Windows) and that Statistics::Descriptive does seem
> to be a bit stagnant.
>
> I think the most important factor is, what tasks do you want to accomplish?
> What Statistics::Descriptive does is give you a simple-to-use OO interface
> to calculating basic descriptive statistics. And that's it.
>
> So if you think that all you'll ever want to calculate is
> min/mode/mean/variance/deviation, then it will be fine. If you find that you
> need skew, kurtosis, autocorrelation or weighted averages then you are out
> of luck. Math::GSL::Statistics has these and plenty more. And they are
> written in C, so they arequite a bit faster. This allows you to use Perl to
> handle computational problems that were previously out of reach because they
> would be too damn slow.
>
> Also, Math::GSL::CDF is where many Cumulative Distribution Functions live
> and if you find that you want to generate random numbers according to
> different
> distributions, there is Math::GSL::RNG. Hundreds of special functions (like
> Bessel, Airy, Gamma, Legendre) are available from Math::GSL::SF. And you
> can do numerical integration and derivatives of Perl subroutines, but I
> digress...
>
> So the long answer is that Math::GSL is under active development and you are
> welcome to join the math-gsl-dev google group [0] and ask questions! I will
> be representing the The Perl Foundation and Math::GSL at the Google Summer
> of Code 2008 Mentor Summit this month,
> where I hope to brainstorm with others about Math::GSL and open-source
> scientific computing in general.
>
> Cheers,
>
> [0] -http://groups.google.com/group/math-gsl-dev

Jonathan Leto

unread,
Oct 15, 2008, 12:35:01 PM10/15/08
to Michael Roberge, math-g...@googlegroups.com
Howdy,

Since you were a first time user I had to moderate your messages (you should be seeing them real soon), now you can post to the list and they will go right through without needing moderation. 

It will help me out a lot to help you if you can tell me which Operating System/hardware architecture  you are using. If it is a unix-like system,
if you could send me the output of

uname -a  
perl -V
gcc --version

it would really facilitate me helping you get Math::GSL working.

The first step that you need to accomplish is getting and installing GSL (the C Library) which can be downloaded here:


For example, on a unix system you would do:

tar zxpvf gsl-1.11.tar.gz
cd gsl-1.11
./configure
make
make check > log 2>&1  # this is optional and takes a while
make install     # you need to be root

The INSTALL file has all this and lots more of system-dependent stuff. If you do not have root on your system
then you set the --prefix=/some/dir on the ./configure command like

./configure --prefix=/home/user/lib

At this point you should have GSL installed, you can check by running

gsl-config --version
gsl-config --prefix

This will tell you the version and the place that it is installed on your system. 

At this point I suggest the easiest possible thing that could work:

As root, run "cpan" on the command line. It will ask you some questions
because it is the first time you will be using it, the defaults are fine. Just
put in your correct continent so that it uses a good set of mirrors.
You should finally see a cpan shell commandline like:

cpan[1]>

which means this is the first command in this session.

Now just type

install Math::GSL

and hopefully everything will work :) 

You may run into snags, please mail the list if you get stuck! If you used --prefix in the first step
then the cpan shell has to be told about it and I forget how off the top of my head...

Good luck,

PS: I will see about making Math::GSL fetch and build GSL automatically, I realize now that the
complexity of compiling GSL is a barrier to entry for people to use Math::GSL. There are some binary
packages of Math::GSL available for Windows, Sisyphus would know much more about those...

Your suggestions on tearing down barriers to entry are much appreciated!


On Wed, Oct 15, 2008 at 7:48 AM, Michael Roberge <mrob...@us.ibm.com> wrote:

Hi...
Not knowing much about the GSL library, I did a little poking around. I have come to realize that the GSL lib has been around since 1996 and is written in C. It appears that there are a number of pre-compiled binary available already for various systems. So the question is.... why does your perl-GSL download include the GSL source code which requires compile? In other words ... can a user download the precompiled GSL lib for there particular system and then simply use the *.pm files in the perl-GSL which you supply?



Regards...
_______________________________________
Michael Roberge
embedded DRAM Design
1000 River Rd, Essex Jct VT 05452, Bldg 861-1 B5
Internal email: Michael Roberge/Burlington/IBM
External email: mrob...@us.ibm.com
(802)769-6272 T/L 446-6272

miker

unread,
Oct 15, 2008, 4:25:08 PM10/15/08
to math-gsl-dev


Hi...
Since I used --prefix=path for the GSL install I need to know what
'cpan' cmd will need in order to install MATH::GSL into the same file
server locations.

miker

unread,
Oct 15, 2008, 3:55:58 PM10/15/08
to math-gsl-dev

Hi...

Downloaded the GSL and placed it our filesystem. Did the ./configure --
prefix=to_our_file_system and did the 'make'. 'make check' returned
two fails.

FAIL: gsl_vector_complex_swap_elements exchanges elements correctly
FAIL: test
FAIL: gsl_vector_complex_swap_elements (static) exchanges elements
correctly
FAIL: test_static
make[2]: *** [check-TESTS] Error 1
make[1]: *** [check-am] Error 2
make: *** [check-recursive] Error 1

What should I do with that? I will go ahead and do the 'make install'
and 'make clean' in hopes that these two fails are not releavent.

miker

unread,
Oct 15, 2008, 5:45:24 PM10/15/08
to math-gsl-dev
Now I am stuck.
I downloaded and extracted the MATH::GSL and tried to do perl
Makefile.PL PREFIX=/afs/install_place. I got errors because of
dependencies and such. It suggested I do it as root. The problem is
that as root I can't seem to do anything in the /afs/filesystem I
suspect due to permission issues. Some how I have to get the perl
Makefile.PL install code updated with the latest stuff as root and
then go back in as a regular user and try the to issue the cmd again.
Not sure what to do.

miker

unread,
Oct 16, 2008, 10:40:27 AM10/16/08
to math-gsl-dev
I was able to make the GSL test errors go away. It turned out that had
downloaded v1.1 instead of v1.11. Once I downloaded and compiled v1.11
the assoicated 'make check' fails no longer exist.

On Oct 15, 3:55 pm, miker <mrobe...@aol.com> wrote:

Jonathan Leto

unread,
Oct 16, 2008, 10:52:17 AM10/16/08
to math-g...@googlegroups.com
Howdy,

Ok, so I think you have GSL installed and at this point, it is easier to get stuff to work
manually (instead of using the cpan shell). You need to find where your GSL is installed,
so type

gsl_config --prefix

Let's say this is "/opt/local/" (which is what it is on my system). Then a "package 
config" file for GSL exists here:

/opt/local/lib/pkgconfig/gsl.pc

What you need to do to tell Module::Build about libs outside the normal /lib, /usr/lib, etc
is set 

export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig

This is because Math::GSL uses ExtUtils::PkgConfig to find GSL and figure out
the proper build flags for the current system. I don't think it is necessary but
if something doesn't work you may also want to set

export LD_LIBRARY_PATH=/opt/local/lib

if you see that your compiler can't find the GSL libraries.

SO, the instructions are:

cd Math-GSL-0.13_02
export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig
perl Build.PL --prefix=/home/user
./Build 
./Build test
./Build install

If any tests fail, please send the entire output of:

DEBUG=1 ./Build test --verbose 1

as a text attachment to the mailing list with a subject like "Math::GSL bug report for version 0.13_02" .

Thanks and good luck!

PS: Good thing you got 1.11, Math::GSL doesn't play nice with older versions....


From Module::Build manpage in regards to --prefix:

NOTE: Ensure you have at least version 0.28 of Module::Build installed and 6.31 of ExtUtils::MakeMaker. Prior versions have differing (and in some cases quite strange) installation behaviors.


miker

unread,
Oct 16, 2008, 2:18:47 PM10/16/08
to math-gsl-dev
Stuck again.
For what ever reason the Build step and/or Makefile.PL can not find
Module::Build. So I logged in as root and did a >cpan Module::Build
and it all looked like it went ok. However perl still doesnt think it
exist. I have even tried the one liner such as "perl -MModule::Build -
e 'print $Module::Build::VERSION' and that does find it either. What
the heck... where is this stuff?

miker

unread,
Oct 16, 2008, 3:37:21 PM10/16/08
to math-gsl-dev
This is very strange...

perl -MModule::Build -e 'print $Module::Build::VERSION'
returns

Can't locate Module/Build.pm in @INC (@INC contains: /usr/lib64/
perl5/5.8.5/x86_64-linux-thread-multi /usr/lib/perl5/5.8.5 /usr/lib64/
perl5/site_perl/5.8.5/x86_64-linux-thread-multi /usr/lib64/perl5/
site_perl/5.8.4/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/
5.8.3/x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.2/
x86_64-linux-thread-multi /usr/lib64/perl5/site_perl/5.8.1/x86_64-
linux-thread-multi /usr/lib64/perl5/site_perl/5.8.0/x86_64-linux-
thread-multi /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/
5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl/5.8.2 /
usr/lib/perl5/site_perl/5.8.1 /usr/lib/perl5/site_perl/5.8.0 /usr/lib/
perl5/site_perl /usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-
multi /usr/lib64/perl5/vendor_perl/5.8.4/x86_64-linux-thread-multi /
usr/lib64/perl5/vendor_perl/5.8.3/x86_64-linux-thread-multi /usr/lib64/
perl5/vendor_perl/5.8.2/x86_64-linux-thread-multi /usr/lib64/perl5/
vendor_perl/5.8.1/x86_64-linux-thread-multi /usr/lib64/perl5/
vendor_perl/5.8.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/
5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/
5.8.3 /usr/lib/perl5/vendor_perl/5.8.2 /usr/lib/perl5/vendor_perl/
5.8.1 /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .).
BEGIN failed--compilation aborted

And I confirmed the contents of @INC from with in perl to be....

/usr/lib64/perl5/5.8.5/x86_64-linux-thread-multi
/usr/lib/perl5/5.8.5
/usr/lib64/perl5/site_perl/5.8.5/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.4/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.3/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.2/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.1/x86_64-linux-thread-multi
/usr/lib64/perl5/site_perl/5.8.0/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.5
/usr/lib/perl5/site_perl/5.8.4
/usr/lib/perl5/site_perl/5.8.3
/usr/lib/perl5/site_perl/5.8.2
/usr/lib/perl5/site_perl/5.8.1
/usr/lib/perl5/site_perl/5.8.0
/usr/lib/perl5/site_perl
/usr/lib64/perl5/vendor_perl/5.8.5/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.4/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.3/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.2/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.1/x86_64-linux-thread-multi
/usr/lib64/perl5/vendor_perl/5.8.0/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.5
/usr/lib/perl5/vendor_perl/5.8.4
/usr/lib/perl5/vendor_perl/5.8.3
/usr/lib/perl5/vendor_perl/5.8.2
/usr/lib/perl5/vendor_perl/5.8.1
/usr/lib/perl5/vendor_perl/5.8.0
/usr/lib/perl5/vendor_perl
.

And using 'find' as root I was able to locate Build.pm in, which is
presumably where cpan stuck it...
/usr/lib/perl5/site_perl/5.8.5/Module/Build.pm

So why the heck cant perl find Module::Build ??????????????????

Why... I'll tell you why... Because for what ever reasons my normal
afs acct userid only has access down into /usr/lib/perl5/site_perl/
5.8.5
I am locked out of Module/ even for read. What the heck, is this a
cpan issue or is this some sort of other security issue related to my
general system?

miker

unread,
Oct 16, 2008, 4:39:02 PM10/16/08
to math-gsl-dev
Ok ... I got the permissions for the Module::Build locations corrected
so that my normal afs user ID has access.

Here is what I got when I did "perl Build.PL --prefix=/home/user"
where /home/user is my targeted install path.

I need you to debug the error...

-------------------------------------------------------------------
Checking for GSL..Found GSL version 1.11
Checking if gcc supports "-Wno-strict-aliasing"...yes
Checking if gcc supports "-Wno-unused-function"...yes
Checking if gcc supports "-Wno-unused-value"...yes
Checking if gcc supports "-Wno-unused-function"...yes
Checking if gcc supports "-Wno-unused-variable"...yes
- ERROR: Test::Exception is not installed
- ERROR: Test::Class is not installed

ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the
versions
of the modules indicated above before proceeding with this
installation

Checking whether your kit is complete...
Looks good

Checking prerequisites...
Deleting Build
Removed previous script 'Build'

Creating new 'Build' script for 'Math-GSL' version '0.13_02'
Have a great day!



miker

unread,
Oct 16, 2008, 5:02:31 PM10/16/08
to math-gsl-dev
I got cpan to install the Test::xxx required stuff. Now getting this
with ./Build

gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so BLAS_wrap.o -shared -L/afs/
btv/data/a20/usrlocal/lib -lgsl -lgslcblas -lm -gsl
/usr/bin/ld: BLAS_wrap.o: relocation R_X86_64_32S against `a local
symbol' can not be used when making a shared object; recompile with -
fPIC
BLAS_wrap.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
error building blib/arch/auto/Math/GSL/BLAS/BLAS.so file from
'BLAS_wrap.o' at inc/GSLBuilder.pm line 99.


Sisyphus

unread,
Oct 16, 2008, 5:56:39 PM10/16/08
to math-g...@googlegroups.com

You could now run 'perl Build test', but you'd probably get errors because
of the 2 missing modules. So ... first install Test::Exception and
Test::Class, then run 'perl Build test', followed by 'perl Build install'.

Cheers,
Rob

Jonathan Leto

unread,
Oct 17, 2008, 12:21:47 AM10/17/08
to math-g...@googlegroups.com
Howdy,

I ran into this once before, and it turns out that it is an issue that pops up in AMD64. After reading this:


I think you can get it to compile if you do

./Build clean
export CFLAGS=-fPIC
./Build install --prefix=/blah

I hope this works. If not, the sledge hammer is to change line 195 of Build.PL that looks like

    extra_linker_flags  => '-shared ' . $ldflags,

to look like this instead:

    extra_linker_flags  => '-shared -fPIC  ' . $ldflags,


Now I will go make the Build.PL script more amd64-aware...

Cheers,

Jonathan Leto

unread,
Oct 17, 2008, 4:32:24 AM10/17/08
to math-g...@googlegroups.com
Howdy,

I fixed this in the latest bleed branch of Math::GSL :



The next version (real soon now) will have it. 

Note: If you had to use the second way, it actually required adding -fPIC to the $ccflags also on the line below. If you get Math::GSL to successfully compile and you run the tests, you will probably see failures in these subsystems:

t/Chebyshev   
t/Deriv      
t/Integration  
t/Min        
t/Roots   


They all require "callbacks" which currently do not work with the way your current Perl is compiled. This can be easily fixed by compiling your own Perl (recommended, so that it has only the features that you want and not all the default bloat) and compiling without multiplicity (which is the default the last time I checked.)

If you don't need the above subsystems, then you don't really need to go through with making your own Perl, but you will definitely feel much more comfortable in your understanding of how Perl works if you compile it, at least just once.

Let us know if you need help recompiling Perl more to Math::GSL's liking...

Cheers,

miker

unread,
Oct 17, 2008, 8:56:45 AM10/17/08
to math-gsl-dev
Hi Jonathan....
I tried both of your solutions(ie the export and the Build.PL line 195
mod), one at a time and both togehter. Still doesn't work. I am
getting the same error. You are correct in that I think my machine is
AMD, not Intel.

Also note that after a "./Build clean" it wanted me to run 'perl
Build.PL --prefix=' again before it would allow "./Build" or "./Build
install"

What next... Not sure I need the functions you said would require a
new compile for Perl. it might however be an interesting learning
step. But I need to get beyond this mess first before I even consider
that.

P.s. Is there any way in this google forum for me to alter one of my
previous post? I noticed a slight typo that would be nice to fix.

On Oct 17, 12:21 am, "Jonathan Leto" <jal...@gmail.com> wrote:
> Howdy,
> I ran into this once before, and it turns out that it is an issue that pops
> up in AMD64. After reading this:
>
> http://www.gentoo.org/proj/en/base/amd64/howtos/index.xml?part=1&chap=3
>
> I think you can get it to compile if you do
>
> ./Build clean
> export CFLAGS=-fPIC
*** Note I had to "perl Build.PL --prefix=/home/user" before it would
let me ./Build again
> ./Build install --prefix=/blah
>
> I hope this works. If not, the sledge hammer is to change line 195 of
> Build.PL that looks like
>
>     extra_linker_flags  => '-shared ' . $ldflags,
>
> to look like this instead:
>
>     extra_linker_flags  => '-shared -fPIC  ' . $ldflags,
>
> Now I will go make the Build.PL script more amd64-aware...
>
> Cheers,
>
>
>
>
>
> On Thu, Oct 16, 2008 at 2:02 PM, miker <mrobe...@aol.com> wrote:
>
> > I got cpan to install the Test::xxx required stuff. Now getting this
> > with ./Build
>
> > gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so BLAS_wrap.o -shared -L/afs/
> > btv/data/a20/usrlocal/lib -lgsl -lgslcblas -lm -gsl
> > /usr/bin/ld: BLAS_wrap.o: relocation R_X86_64_32S against `a local
> > symbol' can not be used when making a shared object; recompile with -
> > fPIC
> > BLAS_wrap.o: could not read symbols: Bad value
> > collect2: ld returned 1 exit status
> > error building blib/arch/auto/Math/GSL/BLAS/BLAS.so file from
> > 'BLAS_wrap.o' at inc/GSLBuilder.pm line 99.
>
> --
> [---------------------]
>  Jonathan Leto
> jal...@gmail.com- Hide quoted text -
>
> - Show quoted text -

miker

unread,
Oct 20, 2008, 8:39:33 AM10/20/08
to math-gsl-dev
Jonathan....
Do you have any suggestions? Per the last append I still can not get ./
Build to compile.
> > - Show quoted text -- Hide quoted text -

Sisyphus

unread,
Oct 20, 2008, 8:57:50 AM10/20/08
to math-g...@googlegroups.com

----- Original Message -----
From: "miker" <mrob...@aol.com>
To: "math-gsl-dev" <math-g...@googlegroups.com>
Sent: Monday, October 20, 2008 11:39 PM
Subject: Re: Perl and Stats


>
> Jonathan....
> Do you have any suggestions? Per the last append I still can not get ./
> Build to compile.

I've got a bit lost here ... what is the output you now get when you run
./Build ?

>>
>> Also note that after a "./Build clean" it wanted me to run 'perl
>> Build.PL --prefix=' again before it would allow "./Build" or "./Build
>> install"

Yes - that's normal. After a "./Build clean" you have to start again from
the beginning with a "perl Build.PL" (or a "./Build.PL", if that works for
you).

Cheers,
Rob

miker

unread,
Oct 20, 2008, 9:36:13 AM10/20/08
to math-gsl-dev
Here is the error from ./Build, even after I did 'export CFLAGS=-
fPIC'. I also tried
'extra_linker_flags => '-shared -fPIC ' . $ldflags,' in Build.PL. In
other words the -fPIC flags did not make the suspected AMD error go
away.

# Here is error from ./Build.....
gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so BLAS_wrap.o -shared -L/
afs/
btv/data/a20/usrlocal/lib -lgsl -lgslcblas -lm -gsl
/usr/bin/ld: BLAS_wrap.o: relocation R_X86_64_32S against `a local
symbol' can not be used when making a shared object; recompile with -
fPIC
BLAS_wrap.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
error building blib/arch/auto/Math/GSL/BLAS/BLAS.so file from
'BLAS_wrap.o' at inc/GSLBuilder.pm line 99.




Sisyphus

unread,
Oct 20, 2008, 9:56:16 AM10/20/08
to math-g...@googlegroups.com

----- Original Message -----
From: "miker" <mrob...@aol.com>
To: "math-gsl-dev" <math-g...@googlegroups.com>
Sent: Tuesday, October 21, 2008 12:36 AM
Subject: Re: Perl and Stats


>


> Here is the error from ./Build, even after I did 'export CFLAGS=-
> fPIC'. I also tried
> 'extra_linker_flags => '-shared -fPIC ' . $ldflags,' in Build.PL. In
> other words the -fPIC flags did not make the suspected AMD error go
> away.
>
> # Here is error from ./Build.....
> gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so BLAS_wrap.o -shared -L/
> afs/
> btv/data/a20/usrlocal/lib -lgsl -lgslcblas -lm -gsl
> /usr/bin/ld: BLAS_wrap.o: relocation R_X86_64_32S against `a local
> symbol' can not be used when making a shared object; recompile with -
> fPIC

It looks like the '-fPIC' flag has not been applied (despite your attempt to
apply it).
The "-shared" bit is there, so there's no need to worry about it ... it's
probably just a question of how to get that "-fPIC" applied using
Module::Build - and I don't know the answer.

If, having got to that stage, you then enter:

gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so
BLAS_wrap.o -shared -fPIC -L/afs/btv/data/a20/usrlocal/lib -lgsl -lgslcblas
-lm -gsl

(that's all one line), does BLAS_wrap.o then get built ok ? (That would at
least confirm that probably the only problem to work out is how to get that
damned -fPIC switch applied :-)

Cheers,
Rob

miker

unread,
Oct 20, 2008, 10:35:57 AM10/20/08
to math-gsl-dev
Hi Rob...
Unfortunatly this is all new to me and as such I really don't know how
to proceed. I have not been able to focus what I have read on the
internet into a directed solution. You maybe correct in that the -fPIC
isn't being picked up. But how can that be even after I altered the
file itself? Jonathan seems to have a pretty good nose for this and
maybe he will be able to come up with something.

Jonathan Leto

unread,
Oct 20, 2008, 11:06:01 AM10/20/08
to math-g...@googlegroups.com
Howdy,

The -fPIC AMD64 problem *should* be fixed in 0.13_03:


Try that and let me know what happens. Good luck!

Cheers,


miker

unread,
Oct 20, 2008, 3:23:01 PM10/20/08
to math-gsl-dev
Hum... 0.13_03 "./Build test" looks like every thing failed, its
reporting a missing Test/Builder/Module.pm Is this a new module that I
did need in previous releases? Or am I having permission issues all
over again? Or maybe the ./Build failed?

On Oct 20, 11:06 am, "Jonathan Leto" <jal...@gmail.com> wrote:
> Howdy,
> The -fPIC AMD64 problem *should* be fixed in 0.13_03:
>
> http://search.cpan.org/~leto/Math-GSL-0.13_03/
>
> Try that and let me know what happens. Good luck!
>
> Cheers,
>
Message has been deleted

Sisyphus

unread,
Oct 20, 2008, 9:15:21 PM10/20/08
to math-g...@googlegroups.com

----- Original Message -----
From: <mrob...@aol.com>
To: <math-g...@googlegroups.com>
Sent: Tuesday, October 21, 2008 6:57 AM
Subject: Re: Perl and Stats


> Please do not post this file to the google forum since it?contains some
> machine specific paths.

Jonathan didn't post it to the google forum ... but *you* did :-)

Anyway - the ./Build output looks fine. (Congratulations - you've just built
Math::GSL !!)

So it's just a matter of working out what's going wrong at the test stage.
What's the actual error you're getting ?

What does the following output for you:
perl -MTest::Builder::Module -le 'print $Test::Builder::Module::VERSION'

and also, the following:
perl -MTest::Simple -le "print $Test::Simple::VERSION"

Cheers,
Rob

Sisyphus

unread,
Oct 20, 2008, 9:53:25 PM10/20/08
to math-g...@googlegroups.com

Oops ... that should be:
perl -MTest::Simple -le 'print $Test::Simple::VERSION'

(Single quotes instead of double quotes.)

Cheers,
Rob

miker

unread,
Oct 21, 2008, 9:21:20 AM10/21/08
to math-gsl-dev
Ya I realized after I sent a response to Jonathan that it got posted.
I went back and deleted the post. Nothing all that sensitive any way.
I am glad to hear that the ./Build looks good. So lets see if we can
figure out the ./Build test issue.

> perl -MTest::Builder::Module -le 'print $Test::Builder::Module::VERSION'
says it cant be found in @INC.

> perl -MTest::Simple -le 'print $Test::Simple::VERSION'
says it cant be found in @INC

Kind of strange because at 1st blush it looks to me mike @INC contains
a path to the required stuff. In addition previous './Build test'
actions seemed to be finding the required test modules. Let me dig
into this a little more. Johnathan also suggested to investigate
PERL5LIB env var which I am also aware of and do make use of. I will
also be looking at that.

Sisyphus

unread,
Oct 21, 2008, 6:57:20 PM10/21/08
to math-g...@googlegroups.com

----- Original Message -----
From: "miker" <mrob...@aol.com>
.
.
>
>> perl -MTest::Simple -le 'print $Test::Simple::VERSION'
> says it cant be found in @INC
>
> Kind of strange because at 1st blush it looks to me mike @INC contains
> a path to the required stuff.

If it's telling you that Test/Simple.pm can't be found in @INC, and yet
Test/Simple.pm *is* already located in one of the @INC directories, then
that means there's a problem with the Test/Simple.pm permissions - or a
problem with the relevant directory permissions. (Afaik there's no other
explanation.)

There are, however, some subtle variations in the "can't be found" errors,
all of which mean slightly different things - so make sure you take note of
exactly what it is that can't be found .... and that it's saying it can't be
found (not that it can't be loaded).

Since Test::Simple is needed to run the test suite for just about every
module, you should be getting the same error for other modules whenever you
run 'make test' or './Build test'.

Cheers,
Rob

miker

unread,
Oct 22, 2008, 9:22:01 AM10/22/08
to math-gsl-dev
Hi...
I think I fixed the in ability to locate Test::Builder, I had to again
change the permissions of the files located in /usr/lib/perl5/5.8.5/
Test/Builder from 'r' to 'rx'. Looks like Test/Builder.pm is in usr/
lib/perl5/5.8.5 and Test/Class.pm and Exception.pm are in site_perl.

I have been attempting to install these perl stnd support modules in
the stnd system locations by aquiring a temp root pw which is only god
for 10min or so. However I am installing all the higher modules such
as the GSL stuff in a different more common user location on our afs
file server.

I am not sure why I keep running into permission issues with the stuff
stored in these stnd machine/system locations. I will have to keep my
eye on this. It could be that even though I change them our install
server is forcing them back after a couple of days?

But even after correcting access issues related to locating the Test
modules I still have "./Build test" problems, but at least different
problems than before. I have sent the log from ./Build test to
Johnathan. It looks like ./Build test is having problems finding the
xxx.o files which as near as I can tell are located in the current
directory from which I am running build.
> Rob- Hide quoted text -

miker

unread,
Oct 22, 2008, 9:32:06 AM10/22/08
to math-gsl-dev
Does anyone know how to get more than one path prepended into the @INC
via the PERL5LIB env var?
In my .profile I have....
export PERL5LIB="$HOME/usrlocal/lib/perl5"
Which works for prepending all the perl5 5.8.x libs located in $HOME/
usrlocal into the @INC. However what if I wanted/needed that stem
path, plus another at say $COMMON/usrlocal/lib/perl5. How can I get
the PERL5LIB to prepend two paths? I tried space delimited list...
export PERL5LIB="$HOME/usrlocal/lib/perl5 $COMMON/usrlocal/lib/
perl5"
Which I dont think is correct because it caused the 1st element in
@INC to have two space delimited names instead of creating multiple
new entries for each unique path.
> > Rob- Hide quoted text -

miker

unread,
Oct 22, 2008, 9:48:47 AM10/22/08
to math-gsl-dev
Can you please expand on the comments related to the inability of a
stnd perl install to use several of the GSL functions. I was really
hoping that by going through this effort on Math::GSL that I and
others in my area would be getting access to the entire GSL suite. Now
it seems like I will have to generate a custom perl install. What will
be the side effects of that? Why doesn't stnd perl support the
callbacks? Will a new custom perl compile loose compatibility with any
existing usages?


On Oct 17, 4:32 am, "Jonathan Leto" <jal...@gmail.com> wrote:
> Howdy,
> I fixed this in the latest bleed branch of Math::GSL :
>
> http://leto.net/gitweb/?p=Math-GSL.git;a=commitdiff;h=295c7bfacce0b58...
> > On Thu, Oct 16, 2008 at 2:02 PM, miker <mrobe...@aol.com> wrote:
>
> >> I got cpan to install the Test::xxx required stuff. Now getting this
> >> with ./Build
>
> >> gcc -o blib/arch/auto/Math/GSL/BLAS/BLAS.so BLAS_wrap.o -shared -L/afs/
> >> btv/data/a20/usrlocal/lib -lgsl -lgslcblas -lm -gsl
> >> /usr/bin/ld: BLAS_wrap.o: relocation R_X86_64_32S against `a local
> >> symbol' can not be used when making a shared object; recompile with -
> >> fPIC
> >> BLAS_wrap.o: could not read symbols: Bad value
> >> collect2: ld returned 1 exit status
> >> error building blib/arch/auto/Math/GSL/BLAS/BLAS.so file from
> >> 'BLAS_wrap.o' at inc/GSLBuilder.pm line 99.
>
> > --
> > [---------------------]
> >  Jonathan Leto
> > jal...@gmail.com
>
> --
> [---------------------]
>  Jonathan Leto
> jal...@gmail.com- Hide quoted text -

Jonathan Leto

unread,
Oct 22, 2008, 11:03:24 AM10/22/08
to math-g...@googlegroups.com
Howdy,

You should be able to separate multiple directories in PERL5LIB with a colon like:

export PERL5LIB=/home/foo/lib:/home/boo/lib

Cheers,

miker

unread,
Oct 23, 2008, 9:58:46 AM10/23/08
to math-gsl-dev
I could and maybe I should kick my self...how could I have overlooked
using the stnd linux ':' for path concat? At any rate I have just
confirmed that using a ':' does indeed allow you to add multiple paths
into the @INC var.
> jal...@gmail.com- Hide quoted text -

miker

unread,
Oct 23, 2008, 10:15:17 AM10/23/08
to math-gsl-dev
Good news.... I have solved the problem with './Build test' not
finding the xxx.o files for GSL. In addition to exporting
PKG_CONFIG_PATH=/opt/local/lib/pkgconfig I also had had to export
LD_LIBRARY_PATH=/opt/local/lib. I am not sure why this wasn't needed
for Math::GSL 0.12, but is for 0.13? It is possible that that at one
point or another during the 0.12 build I had tossed the kitchen sink
at it with both exports and just forgot that LD_ was used when I got
to 0.13.

It says 7/44 failed test programs and 26/3313 subtest failed. Can I
assume this correct given the callback issue?
> > - Show quoted text -- Hide quoted text -

miker

unread,
Oct 23, 2008, 1:55:53 PM10/23/08
to math-gsl-dev
Getting the following error when I try to run the Math: modules from
within a perl script.

Can't load '..../xx/usrlocal/lib64/perl5/site_perl/5.8.5/x86_64-linux-
thread-multi/auto/Math/GSL/Statistics/Statistics.so' for module
Math::GSL::Statistics: libgsl.so.0: cannot open shared object file: No
such file or directory at /usr/lib64/perl5/5.8.5/x86_64-linux-thread-
multi/DynaLoader.pm line 230.
at /xxx/xxx/data/xxx/usrlocal/lib64/perl5/site_perl/5.8.5/x86_64-
linux-thread-multi/Math/GSL/Statistics.pm line 11
Compilation failed in require at debug_ststGSL.pl line 16.
BEGIN failed--compilation aborted at debug_ststGSL.pl line 16.

I got PERL5LIB and 'use lib' all pumped up with all these paths and
more. Do I also need to have the PKG_CONFIG_PATH=/opt/local/lib/
pkgconfig and LD_LIBRARY_PATH=/opt/local/lib exported as well?

miker

unread,
Oct 23, 2008, 3:11:45 PM10/23/08
to math-gsl-dev
Arrrrr. the LD_LIBRARY_PATH env var has to be exported in order to see
the GSL shared libs. The problem is a catch22... The compiler pragma
'use lib' and 'use' get evaluated before anything else, which means
that my attempts to export LD_LIBRARY_PATH inside the perl script
proper don't happen early enough. So what do I do...?
1) Could export the LD_ var in .profile. But that means all my users
would have to do the same.
2) Could put a wrapper around all my scripts so that export is done
1st. Not very clean.
3) Or one of you wizards have a better way?

Jonathan Leto

unread,
Oct 23, 2008, 3:46:50 PM10/23/08
to math-g...@googlegroups.com
Howdy,

Glad to see that got things working. You are correct that 7 subsystems (those which take Perl subs as function arguments) will not work with a Perl binary that was compiled with MULTIPLICITY.

If you need to access these subsystems we can help you compile a Perl binary for Math::GSL.

As for exporting LD_LIBRARY_PATH from your Perl script, try putting it into a BEGIN {} block above
any "use" lines. BEGIN blocks get executed at compile-time, the same time as "use" statements, so put it above
them.

What about putting it into /etc/profile, which is loaded by default for all users? Also, you could have a perl module somewhere in PERL5LIB like "SiteConfig.pm", which sets LD_LIBRARY_PATH and any other necessary environment variables by doing

use SiteConfig;

at the top of a perl script.

I think the /etc/profile solution is the least hassle as long as you don't mind it affecting all users on that machine.

Cheers,

miker

unread,
Oct 24, 2008, 9:42:27 AM10/24/08
to math-gsl-dev
See my comments below...

On Oct 23, 3:46 pm, "Jonathan Leto" <jal...@gmail.com> wrote:
> Howdy,
>
> Glad to see that got things working. You are correct that 7 subsystems
> (those which take Perl subs as function arguments) will not work with a Perl
> binary that was compiled with MULTIPLICITY.
>
> If you need to access these subsystems we can help you compile a Perl binary
> for Math::GSL.

Lets plan on doing that once I get this existing effort in the bag.

>
> As for exporting LD_LIBRARY_PATH from your Perl script, try putting it into
> a BEGIN {} block above
> any "use" lines. BEGIN blocks get executed at compile-time, the same time as
> "use" statements, so put it above
> them.

I had already tried the BEGIN {} block trick. I though for sure this
would work, but no it doesn't. It does get the LD_LIBRARY_PATH
exported to the current shell env. But I still get the error indcating
that it can't locate the shared lib. I suspect that LD_LIBRARY_PATH
has to also exist in the scope of the parent shell env? Can a child
process add an env var to the parent process? If so I don't know how
to do that.

>
> What about putting it into /etc/profile, which is loaded by default for all
> users?

Hum... I thought /etc/profile was only for users of that machine.
Doesn't each installed machines local hardware/drive have a /etc/
profile? We have many machines and thus we want/need all machines to
have the same /etc/profile. No way I am going to be allowed to change /
etc/profile.

Also, you could have a perl module somewhere in PERL5LIB like
> "SiteConfig.pm", which sets LD_LIBRARY_PATH and any other necessary
> environment variables by doing
>
> use SiteConfig;
>
> at the top of a perl script.

Yes I also thought about this concept. This may in fact be the best
solution assuming it can be made to work. I will test.

>
> I think the /etc/profile solution is the least hassle as long as you don't
> mind it affecting all users on that machine.

Question... Why can't the application that needs the shared lib
(Math:: ) be compiled in such a way that it knows the location of the
shared lib and/or exports the location of the shared lib at run time?
Doesn't the make and gcc have such features? The location of the GSL
lib is known and static in my env, why can't Math:: apps know about
this without needing a runtime env var?
> jal...@gmail.com- Hide quoted text -
Reply all
Reply to author
Forward
0 new messages