Test::Buildder v. Test2 subtests

[Ricardo Signes]

I've hit a nasty (to me) difference between Test2::API::run_subtest and
Test::Builder::subtest. Shout out to Matthew Horsfall for helping localize the
problem. Here's my trivial reproducer:

#!perl
use strict;
use warnings;
use Test::More;
use Test2::API;

my $code = sub {
if (fork) {
wait;
cmp_ok($?, '!=', 0, "subprocess died");
} else {
die "123";
};
};

my $call = $ARGV[0] ? \&Test::More::subtest : \&Test2::API::run_subtest;
$call->(test => $code);
done_testing;

When running with Test::More...

~$ perl foo 1
# Subtest: test
123 at foo line 12.
ok 1 - subprocess died
1..1
ok 1 - test
1..1

The die in the subprocess causes the child process to exit nonzero and the
tests run normally. Great! This is how my tests work. When I switch them
to Test2::API::run_subtest, though, I get this:

~$ perl foo 0
# test
not ok 1 - test
# Failed test 'test'
# at foo line 17.
# Caught exception in subtest: 123 at foo line 12.
1..1
not ok 1 - subprocess died
# Failed test 'subprocess died'
# at foo line 10.
# got: 0
# expected: anything else
1..1
not ok 1 - test
# Failed test 'test'
# at foo line 17.
1..1
# Looks like you failed 1 test of 1.

In the forked code, the die causes the subtest to end in failure *in the fork*
meaning we get a repeat result for test 1: one from the child, one from the
parent. Also, the child is now exiting zero instead of nonzero, despite the
fact that it threw an exception.

It gets worse [for me]! Above, I had a cmp_ok on $? that failed, so I had
duplicate *not ok* results for test 1. If I replace that cmp_ok with a pass, I
get duplicate *conflicting* results for test 1... and then the main process
terminates zero. This means that the test seems to pass if you look at its $?,
but the harness sees it as a failure because its total output does include
failures (and non-increasing test numbers).

Adding Test2::IPC does not help. The issue is not synchronising test counters,
or the like. It's that I don't want the subprocess's exit to cause a subtest
exit in the parent process. I think that's right: the only process that should
turn exceptions into subtest fails is the process that created the subtest.

a) am I wrong?
b) advice on how to procede?

--
rjbs