Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
BBC testing from upriver downwards
1 view
Skip to first unread message
James E Keenan
Mar 17, 2017, 5:45:02 PM3/17/17
to per...@perl.org
perl-5.25.11 will be released on Monday March 20. Since this will be
the first monthly dev release to reflect the banishment of '.' from the
default @INC, it is the first monthly release in which we can assess the
effect of that change on CPAN.
Up until now we've mainly relied on Andreas and Slaven's CPAN tester
reports in this regard. But we have a problem. We're having problems
with cpantesters.org such that, while we rapidly get PASS/FAIL
indicators for a given distro on a given platform on a given version of
perl at fast-matrix.cpantesters.org, we are having big delays at getting
the full report of a test failure at matrix.cpantesters.org. Currently,
'matrix' is behind 'fast-matrix' by ... more than a few days.
I know that Doug Bell++ and others are examining this problem, but I
would like to try some modest workarounds. What I would like to do is:
1. Compose a list of CPAN distros starting with those farthest up river,
i.e., distros that only depend on the perl 5 core. Within that set of
distros I'd like to order them from most reverse dependencies to fewest.
Then go down river from there.
2. Get an up-to-date minicpan.
3. Use that minicpan as the source of tests of the CPAN distros.
4. Run something like a full CPAN test for perl 5.25.11 -- but be able
to cut that off at either a certain number of distros -- e.g., the 5000
farthest upriver -- or at a certain number of level below the core itself.
(a) I would like the location where the tests are run, the version of
'cpan', 'cpaniminus', etc. to be completely distinct from whatever my
"usual" procedure is on a given platform -- that so I can just blow away
everything I've done once I'm through.
5. Capture reports of test failures so that we can identify their cause
-- e.g., is this failure due to:
(a) no '.' by default in @INC?
(b) some other change during 5.25.x development?
(c) breakage, not yet corrected, from some change in an earlier perl
major release?
(d) bad code in the CPAN distro?
I don't want to re-invent the wheel here. Do we have prior art for this?
Thank you very much.
Jim Keenan
the first monthly dev release to reflect the banishment of '.' from the
default @INC, it is the first monthly release in which we can assess the
effect of that change on CPAN.
Up until now we've mainly relied on Andreas and Slaven's CPAN tester
reports in this regard. But we have a problem. We're having problems
with cpantesters.org such that, while we rapidly get PASS/FAIL
indicators for a given distro on a given platform on a given version of
perl at fast-matrix.cpantesters.org, we are having big delays at getting
the full report of a test failure at matrix.cpantesters.org. Currently,
'matrix' is behind 'fast-matrix' by ... more than a few days.
I know that Doug Bell++ and others are examining this problem, but I
would like to try some modest workarounds. What I would like to do is:
1. Compose a list of CPAN distros starting with those farthest up river,
i.e., distros that only depend on the perl 5 core. Within that set of
distros I'd like to order them from most reverse dependencies to fewest.
Then go down river from there.
2. Get an up-to-date minicpan.
3. Use that minicpan as the source of tests of the CPAN distros.
4. Run something like a full CPAN test for perl 5.25.11 -- but be able
to cut that off at either a certain number of distros -- e.g., the 5000
farthest upriver -- or at a certain number of level below the core itself.
(a) I would like the location where the tests are run, the version of
'cpan', 'cpaniminus', etc. to be completely distinct from whatever my
"usual" procedure is on a given platform -- that so I can just blow away
everything I've done once I'm through.
5. Capture reports of test failures so that we can identify their cause
-- e.g., is this failure due to:
(a) no '.' by default in @INC?
(b) some other change during 5.25.x development?
(c) breakage, not yet corrected, from some change in an earlier perl
major release?
(d) bad code in the CPAN distro?
I don't want to re-invent the wheel here. Do we have prior art for this?
Thank you very much.
Jim Keenan
Todd Rinaldo
Mar 21, 2017, 12:30:01 PM3/21/17
to per...@perl.org
On Mar 17, 2017, at 4:32 PM, James E Keenan <jk...@verizon.net> wrote:perl-5.25.11 will be released on Monday March 20. Since this will be the first monthly dev release to reflect the banishment of '.' from the default @INC, it is the first monthly release in which we can assess the effect of that change on CPAN.
Up until now we've mainly relied on Andreas and Slaven's CPAN tester reports in this regard. But we have a problem. We're having problems with cpantesters.org such that, while we rapidly get PASS/FAIL indicators for a given distro on a given platform on a given version of perl at fast-matrix.cpantesters.org, we are having big delays at getting the full report of a test failure at matrix.cpantesters.org. Currently, 'matrix' is behind 'fast-matrix' by ... more than a few days.
I know that Doug Bell++ and others are examining this problem, but I would like to try some modest workarounds. What I would like to do is:
1. Compose a list of CPAN distros starting with those farthest up river, i.e., distros that only depend on the perl 5 core. Within that set of distros I'd like to order them from most reverse dependencies to fewest. Then go down river from there.
2. Get an up-to-date minicpan.
3. Use that minicpan as the source of tests of the CPAN distros.
4. Run something like a full CPAN test for perl 5.25.11 -- but be able to cut that off at either a certain number of distros -- e.g., the 5000 farthest upriver -- or at a certain number of level below the core itself.
(a) I would like the location where the tests are run, the version of 'cpan', 'cpaniminus', etc. to be completely distinct from whatever my "usual" procedure is on a given platform -- that so I can just blow away everything I've done once I'm through.
5. Capture reports of test failures so that we can identify their cause -- e.g., is this failure due to:
I don't know if this would be helpful, but I've been tracking all of the modules I've been reporting here. Feel free to scribble there if you like so we don't duplicate effort and reports.
Todd
James E Keenan
Apr 4, 2017, 4:45:12 PM4/4/17
to per...@perl.org
On 03/30/2017 04:36 PM, James E Keenan wrote:
>[snip]
>
> 7. Wrote second perl program (attached) called 'order-battle.pl' to
> record the order in which various modules *first* appeared in the
> 'fails' file and the total number of times each module was cited.
> Results are attached as 'order-of-battle-20170330.txt'.
>
> What is the "order of battle"? It's the order in which, as of today, we
> need to get new CPAN releases out so that down-river distros are no
> longer failing due to failures in their upstream dependencies.
>
> For example, today David Golden prepared a new version of Sub::Uplevel
> -- the #1 distro in the order of battle. Once he releases that to CPAN,
> a tremendous number of downstream distros will have their prerequisites
> satisfied and -- assuming they don't have their own configure/build/test
> failures -- will become installable via this perl-5.26.0-friendly cpanm.
>
Over the past week I have conducted two more rounds of testing using the
approach previously discussed in this thread. I did not report on the
second round, but the third round's results subsume the second.
1. I continue to use the file which David Golden prepared last year as
a proxy for the current state of the river. I decompressed it to a
plain-text file called 'river-2016-02-27.txt'.
2. I continue to use get-upriver-distros.pl to parse the plain-text
file. However, I have modified it to try to avoid selecting the
mod_perl or Team-ReadLine-Perl distributions, as their configurations
require manual intervention; I want get-upriver-distros.pl to run by
itself for hours. (I can confirm that Term-ReadLine-Perl is
installable and 5.26.0-ready.) I am fetching the first 5000 distros
found, which were stored in another plain-text file called
third-round-top-5000.txt.
3. Built and installed an updated perl5 blead for testing:
#####
$ ./bin/perl -v | head -2 | tail -1
This is perl 5, version 26, subversion 0 (v5.26.0
(v5.25.11-27-g1b92e69)) built for x86_64-linux
#####
... and installed Miyagawa's up-to-the-minute cpanm against that perl.
#####
$ ./bin/perl -MApp::cpanminus -E 'say $App::cpanminus::VERSION;'
1.7043
#####
I expect this upgrade to have major benefits. Among other things, it
resolved the perl-5.26.0-compatibility of one of Miyagawa's own
distros:
#####
$ ./bin/cpanm Plack::Middleware::Session
--> Working on Plack::Middleware::Session
Fetching
http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Plack-Middleware-Session-0.30.tar.gz
... OK
Configuring Plack-Middleware-Session-0.30 ... OK
Building and testing Plack-Middleware-Session-0.30 ... OK
Successfully installed Plack-Middleware-Session-0.30
1 distribution installed
#####
4. Adapting nudge from KENTNL, I worked from a just-updated local minicpan
repository.
#####
cd ~/testing/blead
date > startcpanm
cat /home/jkeenan/learn/perl/cpan-river/third-round-top-5000.txt | \
xargs bin/cpanm --mirror ~/minicpan --verbose; date > endcpanm
#####
5. Copied and renamed the 'cpanm' build.log to 20170404-5000-build.log.gz.
6. grepped out relevant lines from the build log:
#####
zgrep FAIL 20170404-5000-build.log.gz | grep -v 'Result: FAIL' | \
sed -e 's/^-> //' > 20170404-5000-fails.txt
#####
7. Used a revised version of 'order-battle.pl' to record the order in
which various modules *first* appeared in the 'fails' file and the
total number of times each module was cited. Results are attached as
'order-of-battle-20170404.txt'.
What is the "order of battle"? It's an approximation of the order in
which, as of today, we need to get new CPAN releases out so that
down-river distros are no longer failing due to failures in their
upstream dependencies.
For example, on March 30 David Golden prepared a new version of
Sub::Uplevel -- the #1 distro in the order of battle. He released that
to CPAN on April 01. Once he released that to CPAN, a tremendous number
of downstream distros had their prerequisites satisfied and -- barring
their own configure/build/test failures -- became installable via this
perl-5.26.0-friendly cpanm. Hence, apart from the fact that in the
first round I only looked at the first 1000 entries in the input file
while in the third round I looked at the first 5000 entries, many
entries found in 20170330's file have gone away in 20170404's file and
the tip of the order of battle looks substantially different.
This approach is useful for me because I only have a laptop to work
with. YMMV.
>[snip]
>
> 7. Wrote second perl program (attached) called 'order-battle.pl' to
> record the order in which various modules *first* appeared in the
> 'fails' file and the total number of times each module was cited.
> Results are attached as 'order-of-battle-20170330.txt'.
>
> What is the "order of battle"? It's the order in which, as of today, we
> need to get new CPAN releases out so that down-river distros are no
> longer failing due to failures in their upstream dependencies.
>
> For example, today David Golden prepared a new version of Sub::Uplevel
> -- the #1 distro in the order of battle. Once he releases that to CPAN,
> a tremendous number of downstream distros will have their prerequisites
> satisfied and -- assuming they don't have their own configure/build/test
> failures -- will become installable via this perl-5.26.0-friendly cpanm.
>
Over the past week I have conducted two more rounds of testing using the
approach previously discussed in this thread. I did not report on the
second round, but the third round's results subsume the second.
1. I continue to use the file which David Golden prepared last year as
a proxy for the current state of the river. I decompressed it to a
plain-text file called 'river-2016-02-27.txt'.
2. I continue to use get-upriver-distros.pl to parse the plain-text
file. However, I have modified it to try to avoid selecting the
mod_perl or Team-ReadLine-Perl distributions, as their configurations
require manual intervention; I want get-upriver-distros.pl to run by
itself for hours. (I can confirm that Term-ReadLine-Perl is
installable and 5.26.0-ready.) I am fetching the first 5000 distros
found, which were stored in another plain-text file called
third-round-top-5000.txt.
3. Built and installed an updated perl5 blead for testing:
#####
$ ./bin/perl -v | head -2 | tail -1
This is perl 5, version 26, subversion 0 (v5.26.0
(v5.25.11-27-g1b92e69)) built for x86_64-linux
#####
... and installed Miyagawa's up-to-the-minute cpanm against that perl.
#####
$ ./bin/perl -MApp::cpanminus -E 'say $App::cpanminus::VERSION;'
1.7043
#####
I expect this upgrade to have major benefits. Among other things, it
resolved the perl-5.26.0-compatibility of one of Miyagawa's own
distros:
#####
$ ./bin/cpanm Plack::Middleware::Session
--> Working on Plack::Middleware::Session
Fetching
http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Plack-Middleware-Session-0.30.tar.gz
... OK
Configuring Plack-Middleware-Session-0.30 ... OK
Building and testing Plack-Middleware-Session-0.30 ... OK
Successfully installed Plack-Middleware-Session-0.30
1 distribution installed
#####
4. Adapting nudge from KENTNL, I worked from a just-updated local minicpan
repository.
#####
cd ~/testing/blead
date > startcpanm
cat /home/jkeenan/learn/perl/cpan-river/third-round-top-5000.txt | \
xargs bin/cpanm --mirror ~/minicpan --verbose; date > endcpanm
#####
5. Copied and renamed the 'cpanm' build.log to 20170404-5000-build.log.gz.
6. grepped out relevant lines from the build log:
#####
zgrep FAIL 20170404-5000-build.log.gz | grep -v 'Result: FAIL' | \
sed -e 's/^-> //' > 20170404-5000-fails.txt
#####
7. Used a revised version of 'order-battle.pl' to record the order in
which various modules *first* appeared in the 'fails' file and the
total number of times each module was cited. Results are attached as
'order-of-battle-20170404.txt'.
What is the "order of battle"? It's an approximation of the order in
which, as of today, we need to get new CPAN releases out so that
down-river distros are no longer failing due to failures in their
upstream dependencies.
For example, on March 30 David Golden prepared a new version of
Sub::Uplevel -- the #1 distro in the order of battle. He released that
to CPAN on April 01. Once he released that to CPAN, a tremendous number
of downstream distros had their prerequisites satisfied and -- barring
their own configure/build/test failures -- became installable via this
perl-5.26.0-friendly cpanm. Hence, apart from the fact that in the
first round I only looked at the first 1000 entries in the input file
while in the third round I looked at the first 5000 entries, many
entries found in 20170330's file have gone away in 20170404's file and
the tip of the order of battle looks substantially different.
This approach is useful for me because I only have a laptop to work
with. YMMV.
0 new messages