Windows-capable version (?) or Marpa::R2 on CPAN

[Jeffrey Kegler]

I took Jean-Damien's Windows port upstream, and it is now on CPAN as
Marpa::R2 2.025_000. Cpantesters troubles continue (though there is
progress) and I'd appreciate seeing test results from list members. I
made some serious changes to the build procedure on POSIX, so there may
be regression issues on UNIX and Linux.

Notes about the port:

1.) I realized there was no good reason for me to be untar'ing files
during the build and so eliminated all use of tar files. This removes
the need for Archive::Tar. It does mean a considerable rearrangement of
the way the build works, even on POSIX/UNIX/Linux.

2.) On Windows, Build.PL tries to load Config::Autoconf and die's if it
cannot. Config::Autoconf is not actually needed until Build is run, but
the way cpantesters works, it is best to signal a failure from Build.PL,
before generating a Build file. Since Config::Autoconf is only
recommended, this is a somewhat annoying way to tell a Windows user that
he needs Config::Autoconf, but this seems to be the least bad option.

3.) I kept Jean-Damien's autoconf logic almost 100% intact. One change
was the elimination of explicit checking of a couple of the open calls
-- the convention in my build is that Fatal catches open() errors.
Arguably, in substituting a catch-all for a hand-crafted error message,
I replaced Jean-Damien's better code with my worse code. But it's best
to have consistency in error handling.

4.) The environment variable to force use of Jean-Damien's install path,
even on POSIX, is MARPA_USE_PERL_AUTOCONF. It's an interesting alternative.

I tested as best I can without access to Windows. It won't surprise me
if this takes several iterations to get right.

-- jeffrey kegler

[Ruslan Shvedov]

windows (win xp sp3, ms visual studio 2003): fail, cygwin: pass (my tests pass too). Logs attached. 

On windows, libmarpa is built ok, but lib\Marpa\R2.c isn't (problem with ccs file and includes).

On cygwin, cpan complains about Config::AutoConf not found in prerequisites, but builds ok.

[Durand Jean-Damien]

Jeffrey,

- OK on Windows, strawberry perl-5.14, gcc/dmake based.
- OK on Debian/Unstable, perl 5.14, gcc/make based
- OK on Debian/Unstable, perl 5.14, gcc/make based, with MARPA_USE_PERL_AUTOCONF environment variable

Few comments:
- Config::AutoConf, not Config::Autoconf
- Timestamps hooks do not work, recompile/relink happens at all steps
- Strange gcc warning about \M characters

Will try later on a home-make nmake/cl based perl, Ruslan report is puzzling - was this using an activestate distro?

Nice work,

Cheers, JD.

/sorry if this appears twice on the list, google groups did not like the first try -;/

[Jeffrey Kegler]

@Jean-Damien: Thanks, that is great news.

With respect to timestamps, the behavior on your logs looks fine -- or
are you talking about an issue that would not show up in the log?

I do two things with timestamps. First, there is a stamp-h1 in the dist
and build directories, intended to prevent unnecessary remakes. If
libmarpa_build/stamp-h1 is up to date with respect to
libmarpa_dist/stamp-h1, nothing should happen in the libmarpa_build
directory.

Second, there is also timestamp hackery to prevent the autotools being
called to recreate configure from configure.ac -- a disaster when the
target system does not have the autotools installed. This second bit of
hackery is not intended to have any effect on your Config::AutoConf
builds -- it's to work around the over-aggressiveness of the default
GNU-autotool-generated Makefile's, which is only an issue with the POSIX
builds.

The '\M' problem is reported for R2.c, which is generated at Build time
from my R2.xs. There are line numbers given, and it'd be interesting to
see what's on those lines.

I wouldn't mind seeing your email twice. It was very good news. :-)

-- jeffrey

[Jeffrey Kegler]

@rns: Thanks for running this test.

I don't know if it is relevant here, but perhaps it is worth mentioning that, at install time, Marpa::R2 uses two different sets of flags, etc. for compiling C.  One is for libmarpa itself, and the other is for the XS code and building the XS library.  There can even be two different C compilers involved.

The two build systems -- ExtUtils::CBuilder and GNU autotools -- often come to different conclusions about the best way to do things.  ExtUtils::CBuilder has a strong preference for the way the current Perl was built, while GNU autotools doesn't know (and I suspect would not much care) how your Perl was built.  There are ways to change the behavior of the two, so in theory I could have synced them up.  But both XS and ExtUtils::CBuilder are very hard to second-guess and very good at their particular stage of the process.  Unless you're an expert in a particular aspect (and often even if you are), it's best to let them do the thinking.  So I let each of them do their thing.  Cpantesters turned into a sea of green, and I never looked back.

And after all, libmarpa *is* a library -- it's not unusual for a library to be compiled and differently than the executable that loads it.

-- jeffrey

[Jeffrey Kegler]

> - Timestamps hooks do not work, recompile/relink happens at all steps

OK. I discovered in Config::AutoConf mode that my stamp-h1 file was not
being created -- I was relying on the Makefile to do it. I am accepting
this as a bug in the Marpa::R2 build, and am not treating it as a
Config::AutoConf bug. I have just fixed it in commit
89d715cb4d609d89d6db637e76d68c5f41d8e164.

I've also fixed the spelling error so, from the list below, that leaves
the mysterious '\M'.

-- jeffrey

[Andrew Rodland]

I have a good guess what the \M warning is about: GCC is complaining about XS-generated

#line 123 "lib\Marpa\somefile.ext"

directives. I'm not sure what's to blame, or how to fix it, but if it's possible to deal with forward-slash paths everywhere, it would probably work around the problem.

Andrew

[Ruslan Shvedov]

On Wed, Nov 14, 2012 at 12:26 AM, Durand Jean-Damien <jeandami...@gmail.com> wrote:

Will try later on a home-make nmake/cl based perl, Ruslan report is puzzling - was this using an activestate distro?

Nope, this was the same cl (MS VS 2003) that compiled and build Marpa-R2-2.024000.tar.gz that you posted for the impatient, with perl 5.14.2 downloaded from perl.org.

 

[Ruslan Shvedov]

That's probably something with (my particular) cpan install or cpan in general. 

I've just successfully build and installed Marpa-R2-2.025_001.tar.gz with perl Makefile.pl; nmake; nmake test; nmake install. Ditto for Marpa-R2-2.025_000.tar.gz. :)

Sorry for what could well be a false positive.

 

 

[Jeffrey Kegler]

@rns: No need to apologize.  I desperately need the test data at this point, and false datapoints (if that is what it was) come with the territory.  You are doing us a favor. -- jeffrey

[Durand Jean-Damien]

Jeffrey,
Great news!
Thanks for the ''\M" explamantion, this is something that is libtool/gcc specific and not relevant to Marpa after all.
Cheers, JD.

[Durand Jean-Damien]

> Thanks for the ''\M" explamantion, this is something that is libtool/gcc specific and not relevant to Marpa after all.

Or specific to perl's in this distrib. Anyway I suggest to forget about this issue.
Cheers, JD.

[Ruslan Shvedov]

On Wed, Nov 14, 2012 at 8:00 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

@rns: No need to apologize.  I desperately need the test data at this point, and false datapoints (if that is what it was) come with the territory.  You are doing us a favor. -- jeffrey

Great to hear that, really.

[Durand Jean-Damien]

Jeffrey,
FYI the RT ticket #81064 on Config::AutoConf being solved, you might want to give a recommended dependency on version 0.20 on Config::AutoConf, if that's possible and replace the awful line in Build_Me.pm that contained stdint.h by a single line $ac->check_default_headers(). Later, at least let's have a full working release on Windows at least one -;
Cheers, JD.

[Jeffrey Kegler]

Sounds good.  Could you send me a patch?  It's the best way to avoid confusion. -- jeffrey

[Ruslan Shvedov]

Looks like in my case Config::AutoConf is somehow not invoked under cpan JKEGL/Marpa-R2-2.025_000.tar.gz, unlike in perl Makefile.PL; nmake.

Where perl Makefile.PL; nmake does

Using Config::AutoConf

Building Marpa-R2

Writing version files

Generating script 'lib\Marpa\R2.ccs'

cl -nologo -c @"lib\Marpa\R2.ccs" -Fo"lib\Marpa\R2.obj" "lib\Marpa\R2.c"

R2.c

ExtUtils::Mkbootstrap::Mkbootstrap('blib\arch\auto\Marpa\R2\R2.bs')

Generating script 'lib\Marpa\R2.lds'

link @"lib\Marpa\R2.lds" -out:"blib\arch\auto\Marpa\R2\R2.dll"

   Creating library lib\Marpa\R2.lib and object lib\Marpa\R2.exp

if exist -manifest "lib\Marpa\R2.dll.manifest" mt -nologo -manifest "lib\Marpa\R

2.dll.manifest" -outputresource:"blib\arch\auto\Marpa\R2\R2.dll";2

cpan JKEGL/Marpa-R2-2.025_000.tar.gz does

        C:\perl-5.14.2\bin\perl.exe -MExtUtils::Command -e "rm_rf" -- blib\arch\

auto\libmarpa\libmarpa.lib

        lib -out:blib\arch\auto\libmarpa\libmarpa.lib marpa.obj marpa_obs.obj avl.obj

Microsoft (R) Library Manager Version 7.10.3077

Copyright (C) Microsoft Corporation.  All rights reserved.

        C:\perl-5.14.2\bin\perl.exe -MExtUtils::Command -e "chmod" -- 755 blib\arch\auto\libmarpa\libmarpa.lib

Generating script 'lib\Marpa\R2.ccs'

cl -nologo -c @"lib\Marpa\R2.ccs" -Fo"lib\Marpa\R2.obj" "lib\Marpa\R2.c"

cl : Command line warning D4024 : unrecognized source file type 'and', object file assumed

cl : Command line warning D4027 : source file 'and' ignored

cl : Command line warning D4024 : unrecognized source file type 'Settings\Ruslan\.cpan\build\Marpa-R2-2.025_000-lqhFb3\libmarpa_build', object file assumed

cl : Command line warning D4027 : source file 'Settings\Ruslan\.cpan\build\Marpa-R2-2.025_000-lqhFb3\libmarpa_build' ignored

R2.c

lib\Marpa\R2.xs(24) : fatal error C1083: Cannot open include file: 'marpa.h': No such file or directory error building dll file from 'lib\Marpa\R2.c' at c:/perl-5.14.2/lib/ExtUtils/CBuilder/Platform/Windows.pm line 130.

and no lib\Marpa\R2.ccs file is in fact created, even when perl Makefile.PL; nmake is issued in, e.g., .cpan/build/ . that results in 

        c:\perl-5.14.2\bin\perl.exe Build --makefile_env_macros 1

Using Config::AutoConf

Building Marpa-R2

Writing version files

Generating script 'lib\Marpa\R2.ccs'

cl -nologo -c @"lib\Marpa\R2.ccs" -Fo"lib\Marpa\R2.obj" "lib\Marpa\R2.c"

cl : Command line warning D4024 : unrecognized source file type 'and', object file assumed

cl : Command line warning D4027 : source file 'and' ignored

cl : Command line warning D4024 : unrecognized source file type 'Settings\Ruslan\.cpan\build\Marpa-R2-2.025_001-Jk8mht\libmarpa_build', object file assumed

cl : Command line warning D4027 : source file 'Settings\Ruslan\.cpan\build\Marpa-R2-2.025_0

01-Jk8mht\libmarpa_build' ignored

R2.c

lib\Marpa\R2.xs(24) : fatal error C1083: Cannot open include file: 'marpa.h': No such file or directory 

error building dll file from 'lib\Marpa\R2.c' at c:/perl-5.14.2/lib/ExtUtils/CBuilder/Platform/Windows.pm line 130.

NMAKE : fatal error U1077: 'c:\perl-5.14.2\bin\perl.exe' : return code '0x19'

Stop.

and, again, no lib\Marpa\R2.ccs file created.

Looks like ExtUtils is invoked where Config::AutoConf should be. 

Hope this helps, --rns.

On Wed, Nov 14, 2012 at 8:00 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote: