[perl #60312] [BUG] OpenBSD Smolder test failures

[James Keenan]

# New Ticket Created by James Keenan
# Please include the string: [perl #60312]
# in the subject line of all future correspondence about this issue.
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=60312 >

Thanks to some automated test reporting setups (which I think are
still coming from magnachef), we are getting a steady stream of test
reports on the Smolder site from FreeBSD, NetBSD and OpenBSD. Of the
three, the first two generally pass 100% of tests, but OpenBSD's
performance has often been much poorer. Of late, OpenBSD has been
passing 99.95% of its tests, which is a clear improvement over just a
month ago. So I figured it would probably be good to put all the
failures we're still getting on OpenBSD in one place to see if
patterns emerge.

I want to stress that I have neither experience with, nor access to,
OpenBSD. So I have no particular insight into these failures, nor
can I directly test patches.

As of today, there are 6 tests failing on OpenBSD, as reported on
Smolder.

1. t/compilers/imcc/syn/veracity.t - Test #5:

pir_output_is( <<'CODE', <<'OUT', "Float NaN" );
.sub test :main
$N0 = 'NaN'
unless $N0 goto not_nan
say "NaN is true"

not_nan:
end
.end
CODE
NaN is true
OUT

not ok 5 - Float NaN

# Failed test 'Float NaN'
# at t/compilers/imcc/syn/veracity.t line 113.
# got: ''
# expected: 'NaN is true
# '

2. t/op/arithmetics.t - Test #7:

not ok 7 - turn a native number into its negative

pasm_output_is( <<'CODE', <<OUTPUT, "turn a native number into its
negative" );
set N0, 0
neg N0
print N0
print "\n"
set N0, -0.0
neg N0
print N0
print "\n"
set N0, 123.4567890
neg N0
print N0
print "\n"
set N0, -123.4567890
neg N0
print N0
print "\n"
set N0, 0
set N1, 1
neg N1, N0
print N1
print "\n"
set N0, -0.0
neg N1, N0
print N1
print "\n"
set N0, 123.4567890
neg N1, N0
print N1
print "\n"
set N0, -123.4567890
neg N1, N0
print N1
print "\n"
end
CODE
-0
0
-123.456789
123.456789
-0
0
-123.456789
123.456789
OUTPUT

# Failed test 'turn a native number into its negative'
# at t/op/arithmetics.t line 175.
# got: '0
# 0
# -123.456789
# 123.456789
# 0
# 0
# -123.456789
# 123.456789
# '
# expected: '-0
# 0
# -123.456789
# 123.456789
# -0
# 0
# -123.456789
# 123.456789
# '

3,4,5. t/pmc/complex.t - Tests 126, 371, 376

126:

.complex_op_is("0+0i", "-inf+0.000000i", 'ln' )

not ok 126 - ln of 0+0i
# Have: -Inf+0.000000i
# Want: -inf+0.000000i

371:

.complex_op_is("0-2i", "-0.000000-0.909297i", 'sinh' )

not ok 371 - sinh of 0-2i
# Have: 0.000000-0.909297i
# Want: -0.000000-0.909297i

376:

.complex_op_is("0+2i", "-0.000000+0.909297i", 'sinh' )

not ok 376 - sinh of 0+2i
# Have: 0.000000+0.909297i
# Want: -0.000000+0.909297i

6. t/pmc/float.t - Test # 23

not ok 23 - neg 0

pasm_output_like( << 'CODE', << 'OUTPUT', "neg 0" );
new P0, 'Float'
set P0, 0.0
neg P0
print P0
end
CODE
/^-0/
OUTPUT

# Failed test 'neg 0'
# at t/pmc/float.t line 509.
# '0'
# doesn't match '/^-0/
# '

Observations:
1. All 6 of these tests are marked to be skipped on Win32. So
perhaps the reason they're failing on OpenBSD is the same as that for
Win32. If so, then we could add 'OpenBSD' to the SKIP messages for
each.

2. 4 of the tests appear to fail depending on how the OS 'spells'
the negation of zero. Could we address this in a hints file?

3. 1 of the tests appears to fail depending on how the OS initial-
cases 'Inf'. Again, could this be addressed in a hints file?

Thank you very much.
kid51

[Andy Dougherty]

On Sun, 2 Nov 2008, James Keenan wrote:

> # New Ticket Created by James Keenan
> # Please include the string: [perl #60312]
> # in the subject line of all future correspondence about this issue.
> # <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=60312 >
>
>
> Thanks to some automated test reporting setups (which I think are
> still coming from magnachef), we are getting a steady stream of test
> reports on the Smolder site from FreeBSD, NetBSD and OpenBSD. Of the
> three, the first two generally pass 100% of tests, but OpenBSD's
> performance has often been much poorer. Of late, OpenBSD has been
> passing 99.95% of its tests, which is a clear improvement over just a
> month ago. So I figured it would probably be good to put all the
> failures we're still getting on OpenBSD in one place to see if
> patterns emerge.
>
> I want to stress that I have neither experience with, nor access to,
> OpenBSD. So I have no particular insight into these failures, nor
> can I directly test patches.
>
> As of today, there are 6 tests failing on OpenBSD, as reported on
> Smolder.

Nice job pulling out the relevant tests and figuring out what's going on!

> Observations:
> 1. All 6 of these tests are marked to be skipped on Win32. So
> perhaps the reason they're failing on OpenBSD is the same as that for
> Win32. If so, then we could add 'OpenBSD' to the SKIP messages for
> each.

I have never liked that plan -- skipping tests simply because they fail
just hides the problem. As a short-term step to keep from being
distracted, sure, it's a good strategy. But such sort term "fixes" tend
to become very long-lived, leading folks to forget there even was a
problem. If the failing tests themselves are the problem, they should
just be deleted.

> 2. 4 of the tests appear to fail depending on how the OS 'spells'
> the negation of zero. Could we address this in a hints file?

This is a long-standing problem: See [perl #28170] and [perl #30737]. The
last time I looked at this, the it seemed we should probably use
signbit(), if available. (If it's not available, a fallback is needed,
but it's likely to usually be available.) However, I don't know why
OpenBSD would differ from NetBSD in this regard. Certainly the math.c
platform files are nearly identical. Are the underlying machines and
perl5 configurations the same for the NetBSD and OpenBSD tests? The
'myconfig' files from each configuration would be helpful in trying to
assess what's the same and what's different.

> 3. 1 of the tests appears to fail depending on how the OS initial-
> cases 'Inf'. Again, could this be addressed in a hints file?

This too is a long-standing problem: See [perl #19183]. It stalled
pending a decision on whether or not parrot should try to enforce a single
spelling of 'Inf' (and 'Nan', etc.) or whether the tests should patch over
the issue.

--
Andy Dougherty doug...@lafayette.edu

[Andy Dougherty]

On Mon, 3 Nov 2008, James Keenan via RT wrote:

> > Are the underlying machines and
> > perl5 configurations the same for the NetBSD and OpenBSD tests?
>

> I believe so. I think magnachef has them set up as virtual machines on
> the same underlying box. He's working on getting me accounts on them
> and, if that comes through, I'll be able to post Configure.pl and make
> output.

Even more immediately useful would be the 'myconfig' info for each
configuration. I designed the file to be useful in precisely this sort of
situation.

--
Andy Dougherty doug...@lafayette.edu

[Chromatic]

On Monday 03 November 2008 09:38:11 Andy Dougherty wrote:

> > 3. 1 of the tests appears to fail depending on how the OS initial-
> > cases 'Inf'. Again, could this be addressed in a hints file?
>
> This too is a long-standing problem: See [perl #19183]. It stalled
> pending a decision on whether or not parrot should try to enforce a single
> spelling of 'Inf' (and 'Nan', etc.) or whether the tests should patch over
> the issue.

What does Perl 5 do? Let's do that. (No one else has made a decision.)

-- c

[James Keenan via RT]

Here's an update based on a recent Smolder report
(http://smolder.plusthree.com/app/public_projects/report_details/21469);

veracity.t: passing

arithmetics.t:
ok 7 - negate -0.0 # TODO -0.0 not implemented, TT #313 : still not passing
(Why is this showing up yellow rather than green on the Smolder report
if it is 'ok'?)

complex.t:
not ok 380 # TODO sinh of -2-3i
not ok 381 # TODO sinh of -2-3i

float.t:
not ok 23 - neg 0 # TODO -0.0 not implemented, TT #313

# Failed (TODO) test 'neg 0'
# at t/pmc/float.t line 512.

# '0'
# doesn't match '/^-0/
# '

So I would say that only one of the six tests I originally flagged has
been clearly resolved.

[Michael Peters]

James Keenan via RT wrote:

> arithmetics.t:
> ok 7 - negate -0.0 # TODO -0.0 not implemented, TT #313 : still not passing
> (Why is this showing up yellow rather than green on the Smolder report
> if it is 'ok'?)

Because it's a TODO test. TODO tests aren't counted in the passing total whether
they pass or not. That's a pretty standard TAP interpretation.

--
Michael Peters
Plus Three, LP

[James Keenan via RT]

On Tue May 12 05:18:47 2009, jk...@verizon.net wrote:
> Here's an update based on a recent Smolder report
>

Thanks to a steady stream of Smolder reports from 'sm...@pc42.my.domain'
-- I don't know who the human there is -- performed on OpenBSD/amd64, we
can see that we are passing all non-SKIPped tests in the 4 files cited.
Example:
http://smolder.plusthree.com/app/public_projects/report_details/25466

So I'm resolving this ticket. Thank you very much.

kid51