Running perl from a test on Windows

[Buddy Burden]

Guys,

Okay, my Google-fu is failing me, so hopefully one of you guys can help me out.

For a test, I need to run a snippet of Perl and collect the output.
However, if it rus in the current interpreter, it will load a module
that I need not to be loaded ('cause I'm also going to test if my code
properly loads it). So I want to run it in a separate instance of
Perl.

First (naive) attempt:

my $output = `$^X -e '$cmd'`;

This works fine on Linux, but fails on Windows. Happily, as soon as I
saw the failures, I recognized I had a quoting problem. No worries, I
said: let's just bypass the shell altogether:

use IPC::System::Simple qw<capturex>;
my $output = capturex($^X, '-e', $cmd);

and then added IPC::System::Simple as a test_requires. That certainly
_changed_ the Windows failures: :-/

C:\strawberry\perl\bin\perl.exe "-MExtUtils::Command::MM" "-e"
"test_harness(0, 'inc', 'blib\lib', 'blib\arch')" t/*.t
t/00.load.t .................. ok
The system cannot find the path specified.
t/data_printer.t ............. skipped: Data::Printer required for
testing pretty dumping
syntax error at -e line 1, at EOF
Execution of -e aborted due to compilation errors.
"C:\strawberry\perl\bin\perl.exe" unexpectedly returned exit value 255
at t/default_funcs.t line 30.
t/default_funcs.t ............
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run

I assume the output is intermingled due to parallelization, but I
wanted to reproduce it faithfully. This seems to be consistent
between ActiveState and Strawberry, but interestingly Cygwin has no
issues. And all other OSes pass, so it's obviously not a syntax error
in my snippet.

So, generally speaking, how _should_ one go about spawning a separate
instance of perl -e inside a test, reliably, on Windows?


-- Buddy

[Gabor Szabo]

On Mon, Jan 14, 2013 at 9:59 PM, Buddy Burden <barefo...@gmail.com> wrote:
> Guys,
>
> Okay, my Google-fu is failing me, so hopefully one of you guys can help me out.
>
> For a test, I need to run a snippet of Perl and collect the output.
> However, if it rus in the current interpreter, it will load a module
> that I need not to be loaded ('cause I'm also going to test if my code
> properly loads it). So I want to run it in a separate instance of
> Perl.
>
> First (naive) attempt:
>
> my $output = `$^X -e '$cmd'`;
>
> This works fine on Linux, but fails on Windows. Happily, as soon as I
> saw the failures, I recognized I had a quoting problem. No worries, I
> said: let's just bypass the shell altogether:

I am not sure if this helps but in Windows you need to put the
double-quotes around $cmd

my $output = qx{$^X -e "$cmd"};

and of course inside $cmd you should use single quotes and not double
quotes if you need
some quotation.

Oh the joy :)

Gabor

[Karen Etheridge]

On Mon, Jan 14, 2013 at 11:59:16AM -0800, Buddy Burden wrote:
> Guys,
>
> Okay, my Google-fu is failing me, so hopefully one of you guys can help me out.
>
> For a test, I need to run a snippet of Perl and collect the output.
> However, if it rus in the current interpreter, it will load a module
> that I need not to be loaded ('cause I'm also going to test if my code
> properly loads it). So I want to run it in a separate instance of
> Perl.

Test::Without::Module should be able to take care of this for you.


--
"Worry: Interest paid on trouble before it comes due." - William Inge
. . . . .
Karen Etheridge, ka...@etheridge.ca GCS C+++$ USL+++$ P+++$ w--- M++

[Eirik Berg Hanssen]

On Mon, Jan 14, 2013 at 9:06 PM, Gabor Szabo <ga...@szabgab.com> wrote:

> On Mon, Jan 14, 2013 at 9:59 PM, Buddy Burden <barefo...@gmail.com>
> wrote:
> > Guys,
> >
> > Okay, my Google-fu is failing me, so hopefully one of you guys can help
> me out.
> >
> > For a test, I need to run a snippet of Perl and collect the output.
> > However, if it rus in the current interpreter, it will load a module
> > that I need not to be loaded ('cause I'm also going to test if my code
> > properly loads it). So I want to run it in a separate instance of
> > Perl.
> >
> > First (naive) attempt:
> >
> > my $output = `$^X -e '$cmd'`;
> >
> > This works fine on Linux, but fails on Windows. Happily, as soon as I
> > saw the failures, I recognized I had a quoting problem. No worries, I
> > said: let's just bypass the shell altogether:
>

On Windows, that still leaves a quoting problem, I believe.
IPC::System::Simple certainly does not seem to handle it: Unless I misread
it entirely, it ends up sending "$^C -e $cmd" as the command line to
Win32::Process::Create.

Let's see ...

C:\Windows\system32>perl -MIPC::System::Simple=capturex -e "print
capturex($^X, '-le', qq{print(qq(--ARG\@{[++\$i]}-->\$_))for\@ARGV;
die(q/Oops - that was odd/)})"
--ARG1-->die(q/Oops
--ARG2-->-
--ARG3-->that
--ARG4-->was
--ARG5-->odd/)

C:\Windows\system32>

Oh yes. It's not doing any quoting.

> I am not sure if this helps but in Windows you need to put the
> double-quotes around $cmd
>
> my $output = qx{$^X -e "$cmd"};
>
> and of course inside $cmd you should use single quotes and not double
> quotes if you need
> some quotation.
>
> Oh the joy :)
>

Or, since $^X presumably is a perl, use C<< qq() >> and/or C<< \x22 >>
instead of C<< "" >>. :)

C:\Windows\system32>perl -e "print qx!$^X -le
\x22print(qq(--ARG\@{[++\$i]}-->\$_))for\@ARGV; die(q/Oops - that was
odd/)\x22!"
Oops - that was odd at -e line 1.

C:\Windows\system32>


Eirik

[Buddy Burden]

Gabor,

> I am not sure if this helps but in Windows you need to put the
> double-quotes around $cmd
>
> my $output = qx{$^X -e "$cmd"};

Yes, that would work if I were running _only_ on Windows. But I need
it work for everything (and the double quotes on Linux will cause any
variables in my perl code to get intepreted by the shell. :-/


-- Buddy

[Gabor Szabo]

Oh sure, I'd have a conditional based on $^O eq 'MSWin32'
and two cases.

Gabor

[Buddy Burden]

Eirik,

> On Windows, that still leaves a quoting problem, I believe.
> IPC::System::Simple certainly does not seem to handle it: Unless I misread
> it entirely, it ends up sending "$^C -e $cmd" as the command line to
> Win32::Process::Create.
>
> Let's see ...
>
> C:\Windows\system32>perl -MIPC::System::Simple=capturex -e "print
> capturex($^X, '-le', qq{print(qq(--ARG\@{[++\$i]}-->\$_))for\@ARGV;
> die(q/Oops - that was odd/)})"
> --ARG1-->die(q/Oops
> --ARG2-->-
> --ARG3-->that
> --ARG4-->was
> --ARG5-->odd/)
>
> C:\Windows\system32>
>
> Oh yes. It's not doing any quoting.

Well, that's ... disappointing. :-/


-- Buddy

[Buddy Burden]

Karen,

> Test::Without::Module should be able to take care of this for you.

Hmmmm ... interesting. That _might_ work ... I'd have to try it out.
I'm not sure just pretending it isn't loaded is sufficient. But I'll
look into it.


-- Buddy

[David Cantrell]

Use the multi-argument form of system() to avoid all shell nastiness,
and use Capture::Tiny to catch its output.

--
David Cantrell | London Perl Mongers Deputy Chief Heretic

All principles of gravity are negated by fear
-- Cartoon Law V

[Eirik Berg Hanssen]

On Tue, Jan 15, 2013 at 3:53 PM, David Cantrell <da...@cantrell.org.uk>wrote:

> On Tue, Jan 15, 2013 at 01:32:09AM -0800, Buddy Burden wrote:
> > Gabor,
> > > I am not sure if this helps but in Windows you need to put the
> > > double-quotes around $cmd
> > >
> > > my $output = qx{$^X -e "$cmd"};
> > Yes, that would work if I were running _only_ on Windows. But I need
> > it work for everything (and the double quotes on Linux will cause any
> > variables in my perl code to get intepreted by the shell. :-/
>
> Use the multi-argument form of system() to avoid all shell nastiness,
> and use Capture::Tiny to catch its output.

That problem is not shell nastiness. It is quoting. Observe:

C:\Windows\system32>perl
system($^X, '-le', 'print for @ARGV; die(qq/Oops/);');
__END__
Oops at -e line 1.

C:\Windows\system32>perl
system($^X, '-le', 'print for @ARGV; die("Oops");');
__END__


C:\Windows\system32>

Where did my exception go?

For that matter, where did my output go? What happened exactly?

Did system() decide not to quote the final argument, since it contained
quotes as well as spaces?

Using C<< "echo" >> instead of C<< $^X >> suggests that's indeed how
system behaves. :-\


Anyway ...

If I recall correctly, the thing is that on Windows, a brand new process
does not get an argument array.

On Windows, it gets a command line.

On Windows, the process gets to parse that command line.

Now, system() does a better job building that command line than
IPC::System::Simple does. But it does not get it "right".

(And how could it, when every executable gets to implement its own
command line parsing?)


No, that's no way to go. On a more constructive note, either
* write a command line as a Windows process expects it, and as suggested
by several already; or
* write a file (tempfile if needs be), and throw $^X at that instead.


Eirik

[Buddy Burden]

First, let me say:

Thanx everyone for all your suggestions!


Eirik,

> That problem is not shell nastiness. It is quoting. Observe:

> :
> :

> If I recall correctly, the thing is that on Windows, a brand new process
> does not get an argument array.
>
> On Windows, it gets a command line.
>
> On Windows, the process gets to parse that command line.
>
> Now, system() does a better job building that command line than
> IPC::System::Simple does. But it does not get it "right".
>
> (And how could it, when every executable gets to implement its own
> command line parsing?)

Well, that's ... even *more* disappointing. <sigh>

> No, that's no way to go. On a more constructive note, either
> * write a command line as a Windows process expects it, and as suggested

> by several already; ...

Yeah, but then I have to do as Gabor suggests and deal with $^O, and
is it different on Win32 vs Cygwin, and are there any other OSes I
need to be worried about, and I'm getting mildly nauseous just
thinking about it. :-/

> ... or

> * write a file (tempfile if needs be), and throw $^X at that instead.

Yep, I think that's the way I need to go. I was hoping to avoid that,
but it is what it is.

Thx again, everyone!

(Wonder if there's a blog in here somewhere ... <ponder ponder>)


-- Buddy

[David Cantrell]

On Tue, Jan 15, 2013 at 11:50:17AM -0800, Buddy Burden wrote:

> Yeah, but then I have to do as Gabor suggests and deal with $^O, and
> is it different on Win32 vs Cygwin, and are there any other OSes I
> need to be worried about, and I'm getting mildly nauseous just
> thinking about it. :-/

Devel::CheckOS *should* correctly detect Windows, no matter how your
perl was built - Win32, Cygwin, Activestate, whatever - if you ask it to
check for MicrosoftWindows. If it doesn't, that's a bug, and I will fix
it quickly if told about it.

Unfortunately I expect that you'll find that Cygwin and Win32 use
different quoting conventions anyway, as Cygwin will use
/some/path/bin/sh vs Win32 using cmd.exe. Which is why you should avoid
the shell entirely.

--
David Cantrell | A machine for turning tea into grumpiness

The word "urgent" is the moral of the story "The boy who cried wolf". As
a general rule I don't believe it until a manager comes to me almost in
tears. I like to catch them in a cup and drink them later.
-- Matt Holiab, in the Monastery

[Buddy Burden]

schwern,

Oh, hey, look: I never responded to this. <smacks self on nose> Bad coder. :-)

>>> One way to deal with this problem is to use the flexibility of TAP and
>>> eshew a testing library.
>>>
>>> print "1..1\n";
>>>
>>> ...run your code normally here...
>>>
>>> print $ok ? "ok 1\n" : "not ok 1\n";

>> Well, yes, but then I have to do all the redirection stuff myself.

> I think you misunderstand. You need a fresh process uncontaminated by
> any other library to run your test in. Each .t file is a fresh process
> over which you have nearly total control (enough for your purposes).
> Load the library and test it directly.
>
> $ cat t/doesnt_load_other_modules.t
> #!/usr/bin/env perl
>
> print "1..1\n";
>
> require My::Library;
> print !$INC{"Some/Other/Library.pm"} ? "ok 1\n" : "not ok 1\n";
>
> Done. Nothing else goes in that .t file. Straight forward Perl. No
> cross platform concerns.

Okay, I think what you're saying would work for _one_ of my problems.
But I actually have two concurrent problems:

* I need to compare the output from another Perl library with the
output from my library. (In this case it happens to be Data::Printer,
but I think that shouldn't matter.
* I need to collect the output from the other Perl library *without
loading it*, because I also want to make sure that my library loads it
for me.

Now, one strategy I could employ here is to put those two tests into
two totally separate .t files. That would work perfectly. But it
just seems messy to me ... like I'm cheating, somehow. I'm testing
two tightly coupled things:
# Does my library load the module?
# Having loaded it, does it use the module properly to produce the
expected output?

Those two things seem like they _ought_ to be two tests in a single
test file, not two separate test files just because I can't figure out
how to make Windows play nice. :-/

And I rather thought this would end up being a solved problem, that
lots of folks would have run into this previously. But I guess either
people haven't, or they haven't often enough to come up with a clever
workaround. So it looks like I have two viable options:

# Give up on the sensibleness of putting both tests into one test file.
# Give up on the convenience of avoiding making a temp file for the
script by using -e.

Neither one is _particularly_ attractive, but I think I'm going with #2 there.

(And thanks for making me spell this out; it's helpful in case I
really do decide to do a blog post.)


-- Buddy

[Daniel Perrett]

"I need to collect the output from the other Perl library *without
loading it*, because I also want to make sure that my library loads it
for me"

Is there a reason the output has to be created during testing rather
than being part of the distribution? What about running it out in a .t
file which precedes the .t file in question?

Daniel

[Buddy Burden]

Daniel,

>> "I need to collect the output from the other Perl library *without
>> loading it*, because I also want to make sure that my library loads it
>> for me"

> Is there a reason the output has to be created during testing rather
> than being part of the distribution?

But that means I'm dependent on a particular version of that library.
More specifically, if the version I have when I _create_ the distro
isn't the same as the version that someone has when they _install_ the
distro, and if the output has changed at all (even a single
character), then my test fails. And, the longer my distro hangs
around, the more likely that is to happen, eventually.

> What about running it out in a .t
> file which precedes the .t file in question?

The problem there is that my tests then become order-dependent, which
breaks parallelization, which many people (including, ANAICT, most
CPAN Testers smokers) turn on. I know *I* do, when I'm testing. ;->


-- Buddy