new Stream Test::Simple question, XS requirement

[bulk88]

Since Test::Stream::Context's core design uses weaken() everywhere, that
means new Test::Simple always requires XS building (for Scalar::Util),
which means it will never run on miniperl, and hence, new Test::Simple
will never be in Perl core, is my understanding correct?

I am asking this since if new Test::Simple is not PP clean and never
will be, there are couple places I would like to throw XS at (the
accessors in Test::Stream::HashBase::Meta in particular).

[Chad Granum]

I have no objections to having OPTIONAL XS available to speed things up for people who want it. However I will not accept making XS a requirement. I thought that since Scalar::Util::weaken was in core that there would be no issues using it in a core module. If using weaken makes Test-Simple non-corable, then I will use a slightly alternate design. But I will wait for rjbs to weigh in on if Scalar::Util::weaken will block Test-Simple from core.

-Chad

[James E Keenan]

The use of functions defined in Scalar-List-Util and implemented via XS
inside the definitions of functions in other libraries which are
distributed with Perl 5 core happens not to be frequent but is not,
AFAIK, forbidden.

From the top of a checkout of the core distro (Linux syntax for xargs):

find cpan/ -type f -name '*.xs' | \
sort | \
cut -d '/' -f1,2 | \
xargs -I % find % -type f -name '*.pm' | \
xargs grep -n -E '(use|require)\s+(Scalar|List)::Util' | \
grep -v Scalar-List-Utils

This shows that Compress-Raw-Bzip2's test suite uses
Scalar::Util::dualvar. Scalar::Uti::dualvar is also used inside Socket
to formulate an error message in one situation.

Scalar-List-Utils's Changes file states that 'weaken()' was added at
some point before version 1.14. Scalar-List-Utils v1.14 was first
distributed with a *production* version of Perl 5 in v5.8.5. Hence,
there might be a limit to the backwards compatibility of a new
Test-Simple distribution.

Thank you very much.
Jim Keenan

[Leon Timmermans]

On Sat, Apr 25, 2015 at 11:57 PM, bulk88 <bul...@hotmail.com> wrote:

Since Test::Stream::Context's core design uses weaken() everywhere, that means new Test::Simple always requires XS building (for Scalar::Util), which means it will never run on miniperl, and hence, new Test::Simple will never be in Perl core, is my understanding correct?

I am asking this since if new Test::Simple is not PP clean and never will be, there are couple places I would like to throw XS at (the accessors in Test::Stream::HashBase::Meta in particular).

That is so mauve…

Leon

[Ricardo Signes]

* bulk88 <bul...@hotmail.com> [2015-04-25T17:57:19]

> Since Test::Stream::Context's core design uses weaken() everywhere, that
> means new Test::Simple always requires XS building (for Scalar::Util), which
> means it will never run on miniperl, and hence, new Test::Simple will never
> be in Perl core, is my understanding correct?

"make minitest" does not load Test::*. I don't see how using weaken blocks
putting Test-Simple v1.3.x into core. Am I missing something?


--
rjbs

[bulk88]

Chad Granum wrote:
> I have no objections to having OPTIONAL XS available to speed things up
> for people who want it. However I will not accept making XS a
> requirement. I thought that since Scalar::Util::weaken was in core that
> there would be no issues using it in a core module.

Core includes XS modules. You relied on an XS module, see below.

> If using weaken
> makes Test-Simple non-corable, then I will use a slightly alternate
> design.

Current new Test::Simple is slightly slower than legacy, will your
non-weaken solution make new Test::Simple twice as slow as legacy?
Because of the rjbs post that supposedly (someone else should confirm
this) minitest does not use Test::*, new Test::Simple can use all the XS
it wants. Maybe you want to change the design rule and say XS is
mandatory to run new Test::More, but also remember, legacy Test::Simple
doesn't require XS. There are 2 levels of "requires XS", 1 is to say
you must have a working C compiler to install new Test::Simple, the
other level is to say, at some point in the past you or your predecessor
must have had access to a C compiler to build the core XS modules. If
your platform's perl doesn't have DynaLoader, or if your OS vendor/IT
dept decided to only distribute XS-free core modules, then sorry, new
Test::Simple is not supported on your system (must have working
Scalar::Util).

To prove new Test::Simple requires XS, I ran.

---------------------------------------------------------
C:\perl521\srcnewb4opt>miniperl -IC:\sources\testmorenew\lib -Ilib
-MTest::More
-e"0"
Can't load module List::Util, dynamic loading not available in this perl.
(You may need to build a new perl executable which either supports
dynamic loading or has the List::Util module statically linked into it.)
at lib/Scalar/Util.pm line 11.
Compilation failed in require at lib/Scalar/Util.pm line 11.
Compilation failed in require at
C:\sources\testmorenew\lib/Test/Stream/Util.pm line 5.
BEGIN failed--compilation aborted at
C:\sources\testmorenew\lib/Test/Stream/Util.pm line 5.
Compilation failed in require at
C:\sources\testmorenew\lib/Test/Stream.pm line 9.
BEGIN failed--compilation aborted at
C:\sources\testmorenew\lib/Test/Stream.pm line 9.
Compilation failed in require at C:\sources\testmorenew\lib/Test/More.pm
line 10.
BEGIN failed--compilation aborted at
C:\sources\testmorenew\lib/Test/More.pm line 10.
Compilation failed in require.
BEGIN failed--compilation aborted.


C:\perl521\srcnewb4opt>
---------------------------------------------------------

Legacy doesn't requre XS

---------------------------------------------------------
C:\perl521\srcnewb4opt>miniperl -Ilib -MTest::More -e"0"

C:\perl521\srcnewb4opt>
---------------------------------------------------------

So what is your (exodist) design rule, atleast for now, on XS and new
Test::Simple?

[Chad Granum]

I am not sure it is entirely my decision. If it is entirely my decision than this is it:
Must have a working scalar::util::weaken to use the new Test::Simple. The Test-Simple dist itself will not require a c compiler to install, as in no xs is included in the dist. A Test-Simple-XS dist may be created if desired that will be used by Test-Simple if it is installed, but it will not be a requirement. Up to rjbs and p5p as to of the xs components will  be part of core.

However I want to know how the rest of perl-qa feels about this first. If people are not bothered by using scalar::util then we have no problem. If they are bothered by scalar::util being used then we need to look at options.

Does anyone have a strong opinion about Test-Simple needing scalar::util?

[Chad Granum]

Just to also be clear. My alternate implementation would be a fallback. If scalar::util works it uses it, if not it falls back to a slightly different system. The switch would happen when Context.pm is loaded. So even if the fallback is slightly slower, few if any systems would ever experience it. The fallback I have in mind also maintains the exact same interface, so it would be seamless to users, and does not deviate from the current context usage in any way.

[James E Keenan]

On 04/26/2015 12:15 AM, Chad Granum wrote:
> I am not sure it is entirely my decision. If it is entirely my decision
> than this is it:
> Must have a working scalar::util::weaken to use the new Test::Simple. The
> Test-Simple dist itself will not require a c compiler to install, as in no
> xs is included in the dist. A Test-Simple-XS dist may be created if desired
> that will be used by Test-Simple if it is installed, but it will not be a
> requirement. Up to rjbs and p5p as to of the xs components will be part of
> core.
>
> However I want to know how the rest of perl-qa feels about this first. If
> people are not bothered by using scalar::util then we have no problem. If
> they are bothered by scalar::util being used then we need to look at
> options.
>
> Does anyone have a strong opinion about Test-Simple needing scalar::util?
> On Apr 25, 2015 8:57 PM, "bulk88" <bul...@hotmail.com> wrote:
>

As I suggested in an earlier post in this thread, I think this is mainly
a backwards-compatibility issue.

The new Test-Simple, like the old one, will be distributed primarily in
two ways:

* As a stand-alone distribution on CPAN, in which case it should work
with older versions of Perl 5 as far back as possible.

* As part of the Perl 5 core distribution, in which case it only has to
work with the version with which it's distributed.

We've got the second case covered, so we only have to worry about the
former. If the new Test-Simple is being installed against a Perl that
already has Scalar::Util::weaken -- which is guaranteed by Perl 5.8.5 or
later -- there is no problem. The version of Test-Simple distributed
with Perl 5.20 was 1.001002 and had a minimum Perl version of 5.006. So
for all practical purposes, the only production versions of Perl we have
to worry about are those from 5.6.0 to 5.8.4.

IIRC, we had to face problems like this several times in the era before
Scalar-List-Utils became part of the core distro and ... we solved them!
I wasn't part of p5p at the time, so I don't recall the details, but
I'll bet it's a SMOP and hence no obstacle to Test-Simple's ongoing
development.

Thank you very much.
Jim Keenan

[Ricardo Signes]

* bulk88 <bul...@hotmail.com> [2015-04-25T23:56:23]

> Because of the rjbs post that supposedly (someone else should confirm this)
> minitest does not use Test::*, new Test::Simple can use all the XS it wants.

Re-confirmation welcome! For the record: My first check was scanning all the
tests run by the minitest make target for 'use Test'. Then I ran minitest with
a dtrace probe for all files opened. Only two files matching /Test/ were
opened:

~$ grep Test dtrace.log | sort | uniq
0 151 open:entry miniperl ../TestInit.pm
1 151 open:entry miniperl ../TestInit.pm
2 151 open:entry miniperl ../TestInit.pm
3 151 open:entry miniperl ../TestInit.pm
3 151 open:entry miniperl ../lib/unicore/TestProp.pl

--
rjbs

[Chad Granum]

For now I am going to add the minimum necessary Scalar::Util version to the requirements list in Makefile.PL. This should meet the toolchain requirement of supporting back to 5.8.1 as Scalar::Util is on cpan for install in perl versions where it is too old. It also sounds like this is not a blocker for putting new Test-Simple into core.

As for people on perls older than 5.8.5 that cannot update Scalar::Util due to XS restrictions, or people who use newer perls but cannot load Scalar::Util, I suspect that in practice it is exceedingly rare that anyone in this position would want to update Test-Simple, let alone need to. For now I am not going to put effort into making Test-Simple work without Scalar::Util. If anybody comes to me and tells me they are in this position, and asks for my help, I would be happy to work on it.

-Chad