NYTProf failure on Windows
Tim Bunce
It's a pity that there are so few test reports for NYTProf on Windows.
It's a pity that failures on Windows so so inscrutable:
http://www.cpantesters.org/cpan/report/6900799
[Sigh.]
Patches welcome from anyone who cares about Windows.
Tim.
Nicholas Clark
Would you like me to hold off merging my IO refactoring until this one is
better understood?
Nicholas Clark
Tim Bunce
No. Go for it. (Thanks!)
I'd like someone to champion windows portability for NYTProf,
but I don't want to delay progress for it.
Tim.
Jan Dubois
I tried to have a look, but I got the same errors as
https://rt.cpan.org/Public/Bug/Display.html?id=53288 when building with
ActivePerl on Linux. (I could also reproduce that bug on Windows). Then
I build a plain perl-5.10.1 build with all default options on Linux
(except for prefix), and got a different set of test failures.
I also get build failures on OS X, but those are probably due
to my setup, building universal binaries on 10.6 using the
10.4u SDK.
I've sent CPAN::Reporter reports for the 2 set of failures from
the Linux builds. I don't really want to look at the Windows
problem as long as I can't even get a clean baseline on Linux
either though.
I've reproduced the failures with both 3.02 and the latest
version from SVN, the test failures don't change.
Let me know if you get the CPAN::Reporter reports, and/or
if there is anything else you want me to try.
Cheers,
-Jan
Tim Bunce
> On Sat, 06 Mar 2010, Tim Bunce wrote:
> >
> > It's a pity that so few people are CPAN Testers on Windows.
> > It's a pity that there are so few test reports for NYTProf on Windows.
> > It's a pity that failures on Windows so so inscrutable:
> > http://www.cpantesters.org/cpan/report/6900799
> >
> > [Sigh.]
> >
> > Patches welcome from anyone who cares about Windows.
>
> I tried to have a look, but I got the same errors as
> https://rt.cpan.org/Public/Bug/Display.html?id=53288 when building with
> ActivePerl on Linux. (I could also reproduce that bug on Windows).
http://www.cpantesters.org/cpan/report/6913897
FYI the similar failure on strawberry perl was
http://www.cpantesters.org/cpan/report/6883938
That's a very odd bug. There seems to be an extra BEGIN called with a
line number of 0. Has anyone any idea whay that might be?
(One thing that strikes me looking at the perl configs is that the gcc
use is a little old, 3.[34], in both. A long shot I know.)
Anyway, could you email me the output of:
NYTPROF=trace=9:stmts=0 perl -d:NYTProf -x t/42-global.t
Hopefully that'll show me where the BEGIN@0 is coming from.
Thanks!
> Then I build a plain perl-5.10.1 build with all default options on
> Linux (except for prefix), and got a different set of test failures.
http://www.cpantesters.org/cpan/report/6913576
I've not seen that failure before. What's the @INC on that perl?
> I also get build failures on OS X, but those are probably due
> to my setup, building universal binaries on 10.6 using the
> 10.4u SDK.
Ok. I've not seen any failures on OS X (which is what I build on).
> I've sent CPAN::Reporter reports for the 2 set of failures from
> the Linux builds. I don't really want to look at the Windows
> problem as long as I can't even get a clean baseline on Linux
> either though.
Understood. I wish I could find out what the magic config is to
reproduce the extra BEGIN call.
I'll try some experimental perl builds at some point, but it might not
be soon.
Tim.
Jan Dubois
> That's a very odd bug. There seems to be an extra BEGIN called with a
> line number of 0. Has anyone any idea whay that might be?
I've found it now: the BEGIN block comes from the -DUSE_SITECUSTOMIZE
code. I've committed r1158 to account for the additional entry, but
it is actually still brittle, as the sitecustomize.pl file can compile
arbitrary additional Perl code.
So maybe it would be better to add a "no_sitecustomize" option to
Devel::NYTProf::Run::profile_this() and run the profiling with "perl -f"
in that case, which will skip the sitecustomize.pl processing.
This should of course only be done for the test script, as real world
code may depend on the execution of sitecustomize.pl (typically just
@INC modifications, but really can be anything).
Let me know if I should revert r1158 and implement the no_sitecustomize
option instead!
Cheers,
-Jan
Jan Dubois
> http://www.cpantesters.org/cpan/report/6900799
I cannot reproduce that error, even with the same version of MinGW on 5.10.1.
Similarly, I cannot reproduce the problem in bug
https://rt.cpan.org/Public/Bug/Display.html?id=55049
Although in that case I only tested with VC6 and MinGW and not the VC version
that the reporter used. I did however test against the 5.8.4 build he is
using as well as 5.8.8, 5.8.9 and 5.10.1.
All builds were at revision r1159, and all finished with *no* test
failures and no crashes.
So from my point of view Devel-NYTProf is working just fine on Windows
(at least 32-bit; haven't tried it with 64-bit in a while).
Cheers,
-Jan
Jan Dubois
>
> So from my point of view Devel-NYTProf is working just fine on Windows
> (at least 32-bit; haven't tried it with 64-bit in a while).
Ok, so I did try to build it on Win64 again, and it all compiles and the
test suite doesn't throw any errors.
I went in and cleaned up various type mismatches though, and noticed that
the XS code uses "long" and "unsigned long" types in various places.
This doesn't really make much sense; the type should be either just "int"
and "unsigned int" if 32 bits will always be big enough, or "UV" and "IV"
when they need to be able to hold a size_t. There is no guarantee that
sizeof(long) > sizeof(int), and there is no guarantee either that
sizeof(long) >= sizeof(size_t).
For example in 64-bit Perl on Windows these types are 8 bytes:
IV, UV, STRLEN, size_t, char*
but all these are just 4 bytes:
int, long, unsigned, unsigned long, I32
So whenever I need to cast e.g. a UV to (unsigned long) I'm wondering if there
isn't a subtle bug hiding somewhere.
So ideally someone should do an audit and make an explicit decision for
each "long" type if they need to be big enough to hold a pointer, or not.
I don't have time for that right now though.
Devel-NYTProf passes all tests on 32 and 64-bit Windows for all Perl versions
and compilers that I tested. I'm not sure if it is completely functional
though, as I see some noise about "ignored #line directives for autosplit
files" and warnings about negative time values. But again, I don't have
any more time to spend on this right now.
Cheers,
-Jan
Tim Bunce
> On Wed, 10 Mar 2010, Tim Bunce wrote:
> > That's a very odd bug. There seems to be an extra BEGIN called with a
> > line number of 0. Has anyone any idea whay that might be?
>
> I've found it now: the BEGIN block comes from the -DUSE_SITECUSTOMIZE code.
Excellent! Many thanks Jan.
> I've committed r1158 to account for the additional entry, but
> it is actually still brittle, as the sitecustomize.pl file can compile
> arbitrary additional Perl code.
>
> So maybe it would be better to add a "no_sitecustomize" option to
> Devel::NYTProf::Run::profile_this() and run the profiling with "perl -f"
> in that case, which will skip the sitecustomize.pl processing.
Yes, probably best.
> This should of course only be done for the test script, as real world
> code may depend on the execution of sitecustomize.pl (typically just
> @INC modifications, but really can be anything).
*nod*
> Let me know if I should revert r1158 and implement the no_sitecustomize
> option instead!
I'm on it. Thanks.
Tim.
Tim Bunce
> On Wed, 10 Mar 2010, Jan Dubois wrote:
> >
> > So from my point of view Devel-NYTProf is working just fine on Windows
> > (at least 32-bit; haven't tried it with 64-bit in a while).
>
> Ok, so I did try to build it on Win64 again, and it all compiles and the
> test suite doesn't throw any errors.
>
> I went in and cleaned up various type mismatches though, and noticed that
> the XS code uses "long" and "unsigned long" types in various places.
>
> This doesn't really make much sense; the type should be either just "int"
> and "unsigned int" if 32 bits will always be big enough, or "UV" and "IV"
> when they need to be able to hold a size_t. There is no guarantee that
> sizeof(long) > sizeof(int), and there is no guarantee either that
> sizeof(long) >= sizeof(size_t).
>
> For example in 64-bit Perl on Windows these types are 8 bytes:
>
> IV, UV, STRLEN, size_t, char*
>
> but all these are just 4 bytes:
>
> int, long, unsigned, unsigned long, I32
>
> So whenever I need to cast e.g. a UV to (unsigned long) I'm wondering if there
> isn't a subtle bug hiding somewhere.
I'll take a look.
> So ideally someone should do an audit and make an explicit decision for
> each "long" type if they need to be big enough to hold a pointer, or not.
> I don't have time for that right now though.
> Devel-NYTProf passes all tests on 32 and 64-bit Windows for all Perl versions
> and compilers that I tested. I'm not sure if it is completely functional
> though, as I see some noise about "ignored #line directives for autosplit
> files"
Those are harmless. We should probably make it smarter when it
recognises that the #line is from an AutoSplit as those get handled
quite well now.
> and warnings about negative time values.
That might be more serious.
> But again, I don't have any more time to spend on this right now.
Understood. Thanks for your time Jan. You've been very helpful.
Tim.