[test-more-users] Pending Test::More fixage - DateTime and string overload users take note

Michael G Schwern

unread,

May 19, 2010, 7:55:37 PM5/19/10

to Perl 5 Porters, Perl QA, Module Authors, CPAN Testers Discuss, test-mo...@googlegroups.com

Test::Simple/More/Builder 0.95_02 has just been released. I'm proud to say
that I had very little to do with it. All the commits are the work of Nick
Clayton and him not being afraid to flex his commit bit. :)
http://github.com/schwern/test-more/tree/v0.95_02

0.95_01 introduced fixage [1] (my fault, not Nick's) when testing overloaded
objects. is(), and probably other functions, no longer stringify their
arguments before comparing. It was a long standing bug that they did. This
means string overloaded objects will now use their eq method to compare rather
than "".

For a well written overloaded objects this should have no effect.
Unfortunately, its very easy to forget to implement a comparison function or
turn on fallback. DateTime objects prior to 0.54 did this.

use Test::More;
use DateTime;

my $date1 = DateTime->new( year => 2010 );
my $date2 = DateTime->new( year => 2010 );

is( $date1, $date2 );

done_testing();

The above code will fail with DateTime prior to 0.54 and Test::More after (but
not including) 0.94. Any code which uses DateTime objects should verify
against the Test::More alpha and upgrade DateTime as necessary. Authors of
string overloaded classes should also verify their implementations and unless
you have a good reason not to, turn on fallback.

For compatibility with older versions of DateTime and Test::More, you should
quote DateTime objects before comparing with is().

is( "$date1", "$date2" );

CPAN Testers, please load your smokers with Test::More 0.95_02, compare the
results with Test::More 0.94 and report any differences in test results to
their respective authors. I would like to see a summary of the differences as
well so I know the scope of the fixage.

Thank you and sorry for the inconvenience.

[1] Bug fixes which will reveal failing tests that accidentally pass.

--
ROCKS FALL! EVERYONE DIES!
http://www.somethingpositive.net/sp05032002.shtml

Slaven Rezic

unread,

May 21, 2010, 4:04:24 PM5/21/10

to Michael G Schwern, Perl 5 Porters, Perl QA, Module Authors, CPAN Testers Discuss, test-mo...@googlegroups.com

Michael G Schwern <sch...@pobox.com> writes:

> CPAN Testers, please load your smokers with Test::More 0.95_02, compare the
> results with Test::More 0.94 and report any differences in test results to
> their respective authors. I would like to see a summary of the differences as
> well so I know the scope of the fixage.

Yesterday I had setup a parallel smoker and by now ~2700 distributions
are tested. A preliminary summary of regressions may be seen here:

http://bbbike.dyndns.org/parsmoker?smoke=testsimple0.9502;notes=1

(note, no update guarantees on this machine!)

It will take maybe a week or so to smoke all of CPAN.

Regards,
Slaven

--
Slaven Rezic - slaven <at> rezic <dot> de

tknotes - A knotes clone, written in Perl/Tk.
http://ptktools.sourceforge.net/#tknotes

Michael G Schwern

unread,

May 21, 2010, 7:35:13 PM5/21/10

to sla...@rezic.de, Perl QA, CPAN Testers Discuss, test-mo...@googlegroups.com

On 2010.5.21 1:04 PM, Slaven Rezic wrote:
> Michael G Schwern <sch...@pobox.com> writes:
>
>> CPAN Testers, please load your smokers with Test::More 0.95_02, compare the
>> results with Test::More 0.94 and report any differences in test results to
>> their respective authors. I would like to see a summary of the differences as
>> well so I know the scope of the fixage.
>
> Yesterday I had setup a parallel smoker and by now ~2700 distributions
> are tested. A preliminary summary of regressions may be seen here:
>
> http://bbbike.dyndns.org/parsmoker?smoke=testsimple0.9502;notes=1
>
> (note, no update guarantees on this machine!)

Thank you, Slaven, you are awesome! I'll start sending out bug reports now.

--
You know what the chain of command is? It's the chain I go get and beat you
with 'til you understand who's in ruttin' command here.
-- Jayne Cobb, "Firefly"

demerphq

unread,

May 22, 2010, 6:45:02 AM5/22/10

to Michael G Schwern, sla...@rezic.de, Perl QA, CPAN Testers Discuss, test-mo...@googlegroups.com

On 22 May 2010 01:35, Michael G Schwern <sch...@pobox.com> wrote:
> On 2010.5.21 1:04 PM, Slaven Rezic wrote:
>> Michael G Schwern <sch...@pobox.com> writes:
>>
>>> CPAN Testers, please load your smokers with Test::More 0.95_02, compare the
>>> results with Test::More 0.94 and report any differences in test results to
>>> their respective authors. I would like to see a summary of the differences as
>>> well so I know the scope of the fixage.
>>
>> Yesterday I had setup a parallel smoker and by now ~2700 distributions
>> are tested. A preliminary summary of regressions may be seen here:
>>
>> http://bbbike.dyndns.org/parsmoker?smoke=testsimple0.9502;notes=1
>>
>> (note, no update guarantees on this machine!)
>
> Thank you, Slaven, you are awesome! I'll start sending out bug reports now.

One thought...

There has been turbulence in the Regexp space over the last versions of perl.

Is it possible these changes might intersect with those changes to
make it harder to compare regexes?

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

Michael G Schwern

unread,

May 22, 2010, 2:01:35 PM5/22/10

to demerphq, sla...@rezic.de, Perl QA, CPAN Testers Discuss, test-mo...@googlegroups.com

On 2010.5.22 3:45 AM, demerphq wrote:
> One thought...
>
> There has been turbulence in the Regexp space over the last versions of perl.
>
> Is it possible these changes might intersect with those changes to
> make it harder to compare regexes?

No, not unless this:

"$regex1" eq "$regex2"

and this:

$regex1 eq $regex2

would somehow produce different results. That's the crux of the issue here,
some poorly written overloaded objects do.

--
Reality is that which, when you stop believing in it, doesn't go away.
-- Phillip K. Dick

Slaven Rezic

unread,

May 28, 2010, 2:12:01 AM5/28/10

to Michael G Schwern, Perl 5 Porters, Perl QA, Module Authors, CPAN Testers Discuss, test-mo...@googlegroups.com

Michael G Schwern <sch...@pobox.com> writes:

> Test::Simple/More/Builder 0.95_02 has just been released.

[...]

My smoker has finished after testing nearly 17000 distribution (I tested
only distributions released in this millennium), and here's the list of
the affected distributions. Most of them already have a RT ticket for
this issue:

Regards,
Slaven

--
Slaven Rezic - slaven <at> rezic <dot> de

BBBike - route planner for cyclists in Berlin
WWW version: http://www.bbbike.de
Perl/Tk version for Unix and Windows: http://bbbike.sourceforge.net

Michael G Schwern

unread,

Jun 1, 2010, 1:42:10 PM6/1/10

to test-mo...@googlegroups.com, perl5-...@perl.org, Slaven Rezic

On 2010.5.27 11:12 PM, Slaven Rezic wrote:
> My smoker has finished after testing nearly 17000 distribution (I tested
> only distributions released in this millennium), and here's the list of
> the affected distributions. Most of them already have a RT ticket for
> this issue:

Thanks again for doing this!

By my reading, the important bits seem to be Test::Class and maybe IO::All?
If this was cross referenced with dependency data we could see which of these
distributions are secretly linchpins.

> Acme-Archive-Mbox-0.01
> Acme-Numbers-1.1 https://rt.cpan.org/Ticket/Display.html?id=57869
> Bread-Board-0.13 https://rt.cpan.org/Ticket/Display.html?id=57760
> Business-ISRC-0.01 https://rt.cpan.org/Ticket/Display.html?id=57870
> Class-Builtin-0.04 https://rt.cpan.org/Ticket/Display.html?id=57766
> Class-Constant-0.05 https://rt.cpan.org/Ticket/Display.html?id=57871
> Class-DBI-Lite-1.011

This one is interesting, the problematic line is this:

is( $thing, undef );

$thing is a defined object which stringifies to undef. If $thing were a plain
undefined value, the above would work. is() has some special case logic to
compare undefined values, but it no longer stringifies before doing those
special checks.

It looks like YAML::Object has the same issue. I'm not sure what to do about
that. I like to encourage is( $foo, undef ) rather than ok( !defined $foo )
because it provides diagnostics.

--
185. My name is not a killing word.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/

Reply all

Reply to author

Forward