Test::Smoke failing to test most recent commit to Perl 5 blead

[James E Keenan]

The most recent smoke testing of Perl 5 blead I have conducted in my
FreeBSD-10.3 VM is that found at:

http://perl5.test-smoke.org/report/53193

The commit tested there was 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e:

#####
commit 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e
Author: Karl Williamson <k...@cpan.org>
AuthorDate: Wed Jan 11 14:21:16 2017 -0700
Commit: Karl Williamson <k...@cpan.org>
CommitDate: Thu Jan 12 10:14:27 2017 -0700

Rmv unused regex implementation structure element
#####

However, at the point in time when I kicked off the smoke run -- Fri Jan
13 21:37:48 EST 2017 -- HEAD of blead had already received 11 additional
commits:

#####
$ git log | grep '^commit' | head -12
commit cbe2fc5001aa59cdc73e04cc35e097a2ecfbeec0 # <- HEAD, which should
have been tested
commit a4368cc338f9178e7f4861a61ab56710050e50dc
commit b1793c1ad815aa7190b1daead0f0d4fce340aff6
commit 92e8e65053b85a7f695f0a1b5270a3567df87863
commit 75ea5c6c82d3602be563bf9c72e8b8e2c711225b
commit 77c8f26370dcc0e16ca7c5f1b8f3bd1b99a57a28
commit 2ab076704905c338cc874079818784698cd5bc85
commit 563642b4907d9b1b6beaa96b472ae787ae81d56f
commit fc54a9b2090b5f71905241c319706e3cca18acc9
commit af4226f2469589e84d382a08a6a5242303758f0b
commit afa4768ac90fcd6a53a3661885a238d344a02f92
commit 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e # <- commit actually tested
#####

This suggests that the run kicked off, as always, via Test::Smoke's 'sh
./smokecurrent.sh', failed to update blead in the perl-current/ directory:

#####
[perl-current] $ gitcurr
blead
[perl-current] $ git show | head -1
commit 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e
#####

This discrepancy can also be seen by a close look at Tony Cook's log:

http://perl.develop-help.com/?b=blead

On a number of occasions in the past month I've suspected that this was
happening, but this if the first time I assembled all the evidence.

Has anyone else experienced this?

Thank you very much.
Jim Keenan

[James E Keenan]

I am attaching a heavily reduced version of smokecurrent.log for the
smoke-testing run in question. As you can see from approximately lines
54-55 of this log, the program only updated the checkout of blead to
3626fc2; it should, IMO, have updated blead to cbe2fc5.

[James E Keenan]

On 01/14/2017 10:10 AM, James E Keenan wrote:
> On 01/13/2017 11:25 PM, James E Keenan wrote:
>> The most recent smoke testing of Perl 5 blead I have conducted in my
>> FreeBSD-10.3 VM is that found at:

Here is an additional instance of the problem, which I have reported as:
https://rt.cpan.org/Ticket/Display.html?id=120008

#####

This is a bug report which provides an additional instance of a problem
originally reported to the perl.qa mailing list/newsgroup (thread
starting at: http://www.nntp.perl.org/group/perl.qa/2017/01/msg13734.html).

I have a VM of FreeBSD-10.3 in which I customarily test blead and/or
smoke-me branches in the Perl 5 core distribution. When I wish to test
a smoke-me branch, I set the first line of
'p5smoke/install/smokecurrent.gitbranch' to the name of that branch as
it appears on remote 'origin'. I then call 'cd ~/p5smoke/install; sh
./smokecurrent.sh' and let Test-Smoke do its thing.

On Jan 26 at about 2200 localtime (Jan 27 0300 UTC) I conducted a smoke
test run on the 'smoke-me/jkeenan/130635-storable' branch:

#####
$ grep -n smoke_branch jsnd6115793d6cc41755a3ed4baaa38d30653656f41.jsn
85: "smoke_branch" : "smoke-me/jkeenan/130635-storable",
#####

The smoke test run -- whose report you can see at
http://perl5.test-smoke.org/report/53530 -- resulted in FAILs:

#####
Failures: (common-args) -Doptimize="-O2 -pipe -fstack-protector
-fno-strict-aliasing"
[stdio/perlio] -DDEBUGGING -Duseithreads
../t/porting/exec-bit.t.....................................FAILED
97-98
../t/porting/podcheck.t.....................................FAILED
183
#####

These failures had nothing to do with the branch being smoked. That
branch had been created minutes earlier by pushing one commit on top of
Perl 5 blead. Those failures occurred in blead, originating in this commit:

#####
commit 1f74a12bfce8b82144fa7fef29618af1e8023298
Author: Christian Millour <cm....@abtela.com>
Date: Thu Jan 19 11:49:03 2017 +0100

document nature and use of $a and $b in sort()

Signed-off-by: Abigail <abi...@abigail.be>
#####

I switched to a separate git checkout directory that I maintain on the
same VM, updated and tested blead, confirmed and corrected the two test
failures and pushed the corrections to blead.

#####
commit 475b224feea308464e18cc6bef788e7a152afa51
Author: James E Keenan <jke...@cpan.org>
AuthorDate: Thu Jan 26 22:44:03 2017
Commit: James E Keenan <jke...@cpan.org>
CommitDate: Thu Jan 26 22:45:26 2017

Correct file mode and line lengths to keep porting tests happy.
#####

This morning I returned to that VM, moved into p5smoke/install and
updated smokecurrent.gitbranch to have its first line read 'blead':

#####
$ head -5 smokecurrent.gitbranch
blead
smoke-me/jkeenan/130635-storable
smoke-me/jkeenan/revert-cpan-118470
smoke-me/jkeenan/77934-threads
smoke-me/jkeenan/petdance/130519-remove-two-unused-macros
#####

I then called 'sh ./smokecurrent.sh' to kick off another smoke test run,
this time for 'blead'. At that point additional commits had been made
to blead. HEAD was:

#####
commit 9a7b7fba267b8910dbea2920dd675030f796320c
Author: Yves Orton <deme...@gmail.com>
AuthorDate: Fri Jan 27 04:23:05 2017
Commit: Yves Orton <deme...@gmail.com>
CommitDate: Fri Jan 27 04:23:24 2017
#####

When the smoke test run completed, however, I was surprised to see that
the commit being tested -- HEAD in the p5smoke/perl-current and
p5smoke/git-perl directories -- was still commit
1f74a12bfce8b82144fa7fef29618af1e8023298 -- the commit which caused the
porting test errors and for which I had already pushed corrections to blead!

Here is the 'sysinfo' element from
jsn1f74a12bfce8b82144fa7fef29618af1e8023298.jsn:

#####
"sysinfo" : {
"lang" : null,
"smoke_date" : "2017-01-27 08:11:48 -0500",
"git_id" : "1f74a12bfce8b82144fa7fef29618af1e8023298",
"git_describe" : "v5.25.9-49-g1f74a12",
"osname" : "freebsd",
"smoker_version" : "0.046",
"test_jobs" : "8",
"smoke_revision" : "1.70",
"osversion" : "10.3-RELEASE",
"username" : "jkeenan",
"lc_all" : null,
"smoke_branch" : "blead",
"smoke_perl" : "5.20.3",
"reporter" : null,
"cpu_description" : "Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz",
"hostname" : "localhost",
"cpu_count" : "1",
"user_note" : "Occasional failures observed in long-running tests
in:\n t/re/speed.t\n t/re/speed_thr.t\n
cpan/Memoize/t/expmod_t.t\n dist/Time-HiRes/t/alarm.t\n",
"smoke_version" : "1.70",
"reporter_version" : "0.053",
"duration" : 2349,
"perl_id" : "5.25.10",
"config_count" : 1,
"architecture" : "amd64"
},
#####

Not surprisingly, since 'blead' had not been moved forward from the
commit at which the porting test errors were introduced, those errors
were still present:

#####
"failures" : [
{
"extra" : [
"97-98"
],
"test" : "../t/porting/exec-bit.t",
"status" : "FAILED"
},
{
"extra" : [
"183"
],
"status" : "FAILED",
"test" : "../t/porting/podcheck.t"
}
#####

Here is the first part of the smokecurrent.log for this run -- with some
comments:

#####
[2017-01-27 08:11:22-0500] Read configuration from:
/usr/home/jkeenan/p5smoke/install/smokecurrent_config
[2017-01-27 08:11:22-0500] Commitlevel before sync:
d6115793d6cc41755a3ed4baaa38d30653656f41

# d611579 was HEAD in the previous branch being smoked:
smoke-me/jkeenan/130635-storable

[2017-01-27 08:11:22-0500] ==> Starting synctree
[2017-01-27 08:11:22-0500] Reading branch to smoke from:
'/usr/home/jkeenan/p5smoke/install/smokecurrent.gitbranch'
[2017-01-27 08:11:22-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl)
running:
[2017-01-27 08:11:22-0500] qx[/usr/local/bin/git pull --all]
From git://perl5.git.perl.org/perl
1f74a12..9a7b7fb blead -> origin/blead
[2017-01-27 08:11:37-0500] Fetching origin
[2017-01-27 08:11:37-0500] Already up-to-date.
[2017-01-27 08:11:37-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl)
running:
[2017-01-27 08:11:37-0500] qx[/usr/local/bin/git remote prune origin]
[2017-01-27 08:11:38-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl)
running:
[2017-01-27 08:11:38-0500] qx[/usr/local/bin/git checkout blead
[2017-01-27 08:11:38-0500] 2>&1]
Switched to branch 'blead'
[2017-01-27 08:11:41-0500] Your branch is behind 'origin/blead' by 7
commits, and can be fast-forwarded.
[2017-01-27 08:11:41-0500] (use "git pull" to update your local branch)

# Note: No indication that 'git pull' was actually run! Why not?

[2017-01-27 08:11:41-0500] In
pwd(/usr/home/jkeenan/p5smoke/perl-current) running:
[2017-01-27 08:11:41-0500] qx[/usr/local/bin/git reset --hard]

# Note: Although 'man git-reset' is not explicit about this, we can
probably assume
# that 'git reset --hard' with no <commit> resets to HEAD -- i.e., no
update to checkout.

[2017-01-27 08:11:43-0500] HEAD is now at d611579 Fix stack buffer
overflow in deserialization of hooks.
#####

So why does Test-Smoke not update the branch being tested in cases like
this?