ext/Module test change ready to pull
Michael G Schwern
ext/Module can be found here:
http://github.com/schwern/perl/tree/ext_tests
All tests pass and its ready to pull.
Once it's integrated, the changes to the dual life modules should be pushed
out to the CPAN authors. I did each module pretty much in its own commit to
make this easier, but I suspect there's already pumpking tools to push blead
changes?
--
Stabbing you in the face so you don't have to.
Michael G Schwern
> A branch containing just the changes to make ext/Module tests run from
> ext/Module can be found here:
>
> http://github.com/schwern/perl/tree/ext_tests
>
> All tests pass and its ready to pull.
>
> Once it's integrated, the changes to the dual life modules should be pushed
> out to the CPAN authors. I did each module pretty much in its own commit to
> make this easier, but I suspect there's already pumpking tools to push blead
> changes?
PS In pulling this branch out of the ExtUtils-Makemaker branch, rather than
making a branch of blead and cherry picking changes into it from the MakeMaker
branch I branched MakeMaker, removed all the MakeMaker related stuff and
rebased it on top of blead. This was easier because most of the changes had
to do with ext/Module. Finally, I rebased ExtUtils-MakeMaker on top of ext_tests.
That I could do all this shuffling around is awesome.
I found these git commands and utilities helpful:
git-filter-branch to remove ext/ExtUtils-MakeMaker from the history entirely.
I wrote a git-kill script to remove individual or ranges of changes.
$ cat ~/bin/git-kill
#!/usr/bin/perl -w
my $begin = shift;
my $end = @ARGV ? shift : $begin;
system git => "rebase", "--onto", "$begin~1", $end;
__END__
I probably could have done all this with git-rebase --interactive but I don't
quite understand how you delete, as opposed to squash, commits with that.
--
You know what the chain of command is? It's the chain I go get and beat you
with 'til you understand who's in ruttin' command here.
-- Jayne Cobb, "Firefly"
Rafael Garcia-Suarez
> A branch containing just the changes to make ext/Module tests run from
> ext/Module can be found here:
>
> http://github.com/schwern/perl/tree/ext_tests
>
> All tests pass and its ready to pull.
>
> Once it's integrated, the changes to the dual life modules should be pushed
> out to the CPAN authors. I did each module pretty much in its own commit to
> make this easier, but I suspect there's already pumpking tools to push blead
> changes?
There would be pumpking tools if there would be places to push to. The
pumpking tool we have (Porting/Maintainers) just gives an email. Bah,
why doesn't everyone just use github :)
Anyway, thanks for this, and it looks good, but before integrating all
of this into the trunk, I'd like to know if it's going to be
integrated in the various maints too. Because the removal of the
PERL_CORE hacks in dual-life modules will necessitate all of this too,
if in maint we want to upgrade a dual-life module. And IIRC Dave was
talking of integrating the whole ext/ work after 5.10.1. (So basically
what I'm saying is that we may not want to push those changes to
dual-life modules right now)
Also, commit 18d035e0237007ca4f1687a6773449f99611b5b2 to Pod::Man
seems unrelated to the branch's whole purpose.
Michael G Schwern
> 2009/3/15 Michael G Schwern <sch...@pobox.com>:
>> A branch containing just the changes to make ext/Module tests run from
>> ext/Module can be found here:
>>
>> http://github.com/schwern/perl/tree/ext_tests
>>
>> All tests pass and its ready to pull.
>>
>> Once it's integrated, the changes to the dual life modules should be pushed
>> out to the CPAN authors. I did each module pretty much in its own commit to
>> make this easier, but I suspect there's already pumpking tools to push blead
>> changes?
>
> There would be pumpking tools if there would be places to push to. The
> pumpking tool we have (Porting/Maintainers) just gives an email. Bah,
> why doesn't everyone just use github :)
It should be sufficient to mail a patch, no?
> Anyway, thanks for this, and it looks good, but before integrating all
> of this into the trunk, I'd like to know if it's going to be
> integrated in the various maints too. Because the removal of the
> PERL_CORE hacks in dual-life modules will necessitate all of this too,
> if in maint we want to upgrade a dual-life module. And IIRC Dave was
> talking of integrating the whole ext/ work after 5.10.1. (So basically
> what I'm saying is that we may not want to push those changes to
> dual-life modules right now)
Yeah, I'd say to put this into the maint branches. Backporting the changes to
the testing system should be trivial. Backporting the ext/ changes will just
take time but is otherwise a rote task. I wouldn't be surprised if most of it
applies clean.
Then I'd say wait to pull it into blead until after 5.10.1 or else blead will
be out of sync with CPAN. That'll delay brining in MakeMaker, but I think we
can wait.
> Also, commit 18d035e0237007ca4f1687a6773449f99611b5b2 to Pod::Man
> seems unrelated to the branch's whole purpose.
Oh, that's a MakeMaker thing. It's already in Pod::Man.
http://rt.cpan.org/Public/Bug/Display.html?id=43588
Normally I'd cherry pick the change from ext_tests into MakeMaker and then
git-kill the change in ext_tests, but MakeMaker has been rebased on top of
ext_tests so won't the cherry pick result in an empty change? How do I move a
change like this?
--
The interface should be as clean as newly fallen snow and its behavior
as explicit as Japanese eel porn.
Aristotle Pagaltzis
> I probably could have done all this with git-rebase
> --interactive but I don't quite understand how you delete,
> as opposed to squash, commits with that.
You delete the line.
Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Dave Mitchell
My current plan is to include the ext/Foo/Bar => ext/Foo-Bar renaming
in 5.10.1; otherwise, merging recent CPAN=>blead pulls into maint is
going to be tough.
Other than that, I'm hoping that around now-ish, blead is going to go
through a phase of being nearly in sync with CPAN, until 5.10.1 is out.
--
A walk of a thousand miles begins with a single step...
then continues for another 1,999,999 or so.
Nicholas Clark
Does it make it easier to validate this if more (if not all) dual-life modules
end up in ext/ with the identical layout to the CPAN tarball?
Nicholas Clark
Dave Mitchell
I assume so!
--
O Unicef Clearasil!
Gibberish and Drivel!
-- "Bored of the Rings"
Nicholas Clark
Me too. So you're happy if this gets merged now?
I think that it's the cleanest way to solve the problem that make distclean
doesn't remove one generated file from ext/Test-Harness
Nicholas Clark
Dave Mitchell
Um, what is "this" exactly?
> I think that it's the cleanest way to solve the problem that make distclean
> doesn't remove one generated file from ext/Test-Harness
I'm happy (I think) for more dual-life modules to end up in ext/.
But wasn't this thread about including Schwern's test changes, which touch
a bunch of of dual-life modules (thus putting them out of sync with CPAN)?
I'd prefer the latter to be deferred until after 5.10.1 unless we have
fairly concrete promises from dual-life authors of plans for imminent
re-sync with CPAN?
(Or am I misunderstanding the whole thing?)
--
Now is the discount of our winter tent
-- sign seen outside camping shop
Nicholas Clark
No, "this" was the "this" that you thought.
Ignoring syncing-ness of CPAN modules, I think that its probably less work
overall to merge Schwern's test changes first, then start moving things from
lib to ext. The more stuff that is in ext/ on the flag day of swapping to
testing inside ext, the more work that change is.
Considering the syncing-ness of CPAN modules, and playing with
git clone git://github.com/schwern/perl.git;
git diff origin/blead origin/ext_tests; and perl Porting/Maintainers, I think
that the authors involved are:
[blead ] Attribute::Handlers rgarcia Rafael Garcia-Suarez <rga...@cpan.org>
[unknown] Compress::Raw::Bzip2 pmqs Paul Marquess <pm...@cpan.org>
[unknown] Compress::Raw::Zlib pmqs Paul Marquess <pm...@cpan.org>
[cpan ] Cwd kwilliams Ken Williams <kwil...@cpan.org>
[unknown] DB::File pmqs Paul Marquess <pm...@cpan.org>
[blead ] Data::Dumper p5p perl5-porters <perl5-...@perl.org>
[cpan ] Devel::PPPort mhx Marcus Holland-Moritz <m...@cpan.org>
[unknown] Digest::MD5 gaas Gisle Aas <ga...@cpan.org>
[unknown] Digest::SHA mshelor Mark Shelor <msh...@cpan.org>
[unknown] Encode dankogai Dan Kogai <dank...@cpan.org>
[unknown] Errno p5p perl5-porters <perl5-...@perl.org>
[unknown] Filter::Util::Call pmqs Paul Marquess <pm...@cpan.org>
[unknown] IO gbarr Graham Barr <gb...@cpan.org>
[cpan ] IPC::SysV mhx Marcus Holland-Moritz <m...@cpan.org>
[unknown] Scalar-List-Utils gbarr Graham Barr <gb...@cpan.org>
[unknown] MIME::Base64 gaas Gisle Aas <ga...@cpan.org>
[unknown] Math::BigInt::FastCalc tels Tels <nospam...@bloodgate.com>
[unknown] Module::Pluggable simonw Simon Wistow <sim...@cpan.org>
[blead ] Safe rgarcia Rafael Garcia-Suarez <rga...@cpan.org>
[unknown] Storable ams Abhijit Menon-Sen <a...@cpan.org>
[unknown] Sys::Syslog saper S�bastien Aperghis-Tramoni <sa...@cpan.org>
[unknown] Test::Harness andya Andy Armstrong <an...@hexten.net>
[unknown] Text::Soundex markm Mark Mielke <ma...@cpan.org>
[unknown] Time::HiRes zefram Andrew Main <zef...@cpan.org>
[unknown] Unicode::Normalize sadahiro SADAHIRO Tomoyuki <SADA...@cpan.org>
[unknown] Win32 jand Jan Dubois <ja...@activestate.com>
[unknown] Win32 jand Jan Dubois <ja...@activestate.com>
[blead ] threads::shared jdhedden Jerry D. Hedden <jdhe...@cpan.org>
I wouldn't be confident that more than 90% of that list would respond in a
timely fashion.
But I'm not sure whether we absolutely need to sync to CPAN, because
Schwern's branch doesn't change any installed, versioned files. Just test
files, a few test modules, and chunk of core code. I think (but might be
wrong) that the PAUSE indexer only starts automatically killing kittens if
two files claim to be the same module, at the same version, but having
different contents.
Nicholas Clark
Andreas J. Koenig
>> But I'm not sure whether we absolutely need to sync to CPAN, because
>> Schwern's branch doesn't change any installed, versioned files. Just test
>> files, a few test modules, and chunk of core code. I think (but might be
>> wrong) that the PAUSE indexer only starts automatically killing kittens if
>> two files claim to be the same module, at the same version, but having
>> different contents.
> I assume the PAUSE indexer has long since worked out how to handle dual lifed
> modules with the same version, but Andreas can answer that.
The PAUSE indexer does not look at the contents of files, it just lets
the higher version number win. And if the same version number of a
module is in perl and has a separate distro, then the separate distro
wins.
--
andreas
Jerry D. Hedden
> A branch containing just the changes to make ext/Module tests run from
> ext/Module can be found here:
>
> http://github.com/schwern/perl/tree/ext_tests
Once this is merged into blead, I will release a new version of
threads and threads::shared to CPAN that incorporate these changes as
well as changes to Makefile.PL to eliminate references to MAN3POD.
Nicholas Clark
But if the separate distro and the core both have files with the same
version number, but the contents differ, this is bad? And kittens suffer?
Specifically, files that are installed, as distinct from test scripts or
Makefile.PL or other "build" stuff.
Nicholas Clark
Michael G Schwern
> Ignoring syncing-ness of CPAN modules, I think that its probably less work
> overall to merge Schwern's test changes first, then start moving things from
> lib to ext. The more stuff that is in ext/ on the flag day of swapping to
> testing inside ext, the more work that change is.
The changes were really very rote. It was just a matter of plowing through
all the ext/ modules and looking for PERL_CORE. Expanding the number of
modules in ext/ before applying the patch isn't a big deal.
Not that I wouldn't mind seeing it applied now.
> Considering the syncing-ness of CPAN modules, and playing with
> git clone git://github.com/schwern/perl.git;
> git diff origin/blead origin/ext_tests; and perl Porting/Maintainers, I think
> that the authors involved are:
>
> [blead ] Attribute::Handlers rgarcia Rafael Garcia-Suarez <rga...@cpan.org>
> [unknown] Compress::Raw::Bzip2 pmqs Paul Marquess <pm...@cpan.org>
> [unknown] Compress::Raw::Zlib pmqs Paul Marquess <pm...@cpan.org>
> [cpan ] Cwd kwilliams Ken Williams <kwil...@cpan.org>
> [unknown] DB::File pmqs Paul Marquess <pm...@cpan.org>
> [blead ] Data::Dumper p5p perl5-porters <perl5-...@perl.org>
> [cpan ] Devel::PPPort mhx Marcus Holland-Moritz <m...@cpan.org>
> [unknown] Digest::MD5 gaas Gisle Aas <ga...@cpan.org>
> [unknown] Digest::SHA mshelor Mark Shelor <msh...@cpan.org>
> [unknown] Encode dankogai Dan Kogai <dank...@cpan.org>
> [unknown] Errno p5p perl5-porters <perl5-...@perl.org>
> [unknown] Filter::Util::Call pmqs Paul Marquess <pm...@cpan.org>
> [unknown] IO gbarr Graham Barr <gb...@cpan.org>
> [cpan ] IPC::SysV mhx Marcus Holland-Moritz <m...@cpan.org>
> [unknown] Scalar-List-Utils gbarr Graham Barr <gb...@cpan.org>
> [unknown] MIME::Base64 gaas Gisle Aas <ga...@cpan.org>
> [unknown] Math::BigInt::FastCalc tels Tels <nospam...@bloodgate.com>
> [unknown] Module::Pluggable simonw Simon Wistow <sim...@cpan.org>
> [blead ] Safe rgarcia Rafael Garcia-Suarez <rga...@cpan.org>
> [unknown] Storable ams Abhijit Menon-Sen <a...@cpan.org>
> [unknown] Sys::Syslog saper Sébastien Aperghis-Tramoni <sa...@cpan.org>
> [unknown] Test::Harness andya Andy Armstrong <an...@hexten.net>
> [unknown] Text::Soundex markm Mark Mielke <ma...@cpan.org>
> [unknown] Time::HiRes zefram Andrew Main <zef...@cpan.org>
> [unknown] Unicode::Normalize sadahiro SADAHIRO Tomoyuki <SADA...@cpan.org>
> [unknown] Win32 jand Jan Dubois <ja...@activestate.com>
> [unknown] Win32 jand Jan Dubois <ja...@activestate.com>
> [blead ] threads::shared jdhedden Jerry D. Hedden <jdhe...@cpan.org>
>
>
> I wouldn't be confident that more than 90% of that list would respond in a
> timely fashion.
They don't need to apply the changes immediately, just before their next
release. No?
> But I'm not sure whether we absolutely need to sync to CPAN, because
> Schwern's branch doesn't change any installed, versioned files. Just test
> files, a few test modules, and chunk of core code. I think (but might be
> wrong) that the PAUSE indexer only starts automatically killing kittens if
> two files claim to be the same module, at the same version, but having
> different contents.
I assume the PAUSE indexer has long since worked out how to handle dual lifed
modules with the same version, but Andreas can answer that.
And yes, I believe the path only touches test code.
My concern with syncing with CPAN would not be about the PAUSE indexer but
that the next CPAN release that we want to push into 5.10.1 is out of sync.
That is, it might still contain PERL_CORE junk in the tests. But it's
relatively easy to fix if it does. And like I said, if the author is making a
release they can incorporate the anti-PERL_CORE patch.
Thus not needing an immediate sync with CPAN, just on their next release.
I think we can do this before 5.10.1 if it would make 5.10.1 release faster.
If not, don't.
--
101. I am not allowed to mount a bayonet on a crew-served weapon.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
Dave Mitchell
Now that I've caught up on most of my old p5p emails and understand the
issues a bit better, my wish is that no further messing with modules is
carried out until after 5.10.1 is released - where messing is defined as
lib/ to ext/ renames, and moving test files into ext/.
If there are pressing arguments for doing otherwise, I'm open to
persuasion!
--
You live and learn (although usually you just live).
Nicholas Clark
Specifically, this manicheck failure:
./ext/Test-Harness/t/ext/Test-Harness/t/source_tests/harness not in MANIFEST
I believe will be properly fixed by having tests for Test::Harness run from
ext/Test-Harness rather than t
However, you could bodge it for now for 5.10.1 by putting
rm -rf ext/Test-Harness/t/ext
in some part of the distclean target.
But apart from that, I can't see any direct short term benefit from making
the change.
Nicholas Clark
Dave Mitchell
> On Sun, Apr 19, 2009 at 09:45:30PM +0100, Dave Mitchell wrote:
> > [ just catching up on an old thread... ]
> >
> > On Wed, Mar 25, 2009 at 10:57:02PM +0000, Nicholas Clark wrote:
>
> > > No, "this" was the "this" that you thought.
> > >
> > > Ignoring syncing-ness of CPAN modules, I think that its probably less work
> > > overall to merge Schwern's test changes first, then start moving things from
> > > lib to ext. The more stuff that is in ext/ on the flag day of swapping to
> > > testing inside ext, the more work that change is.
> >
> > Now that I've caught up on most of my old p5p emails and understand the
> > issues a bit better, my wish is that no further messing with modules is
> > carried out until after 5.10.1 is released - where messing is defined as
> > lib/ to ext/ renames, and moving test files into ext/.
> >
> > If there are pressing arguments for doing otherwise, I'm open to
> > persuasion!
>
> Specifically, this manicheck failure:
>
> ./ext/Test-Harness/t/ext/Test-Harness/t/source_tests/harness not in MANIFEST
Sorry, I'm confused. What's the issue with this file?
> I believe will be properly fixed by having tests for Test::Harness run from
> ext/Test-Harness rather than t
>
> However, you could bodge it for now for 5.10.1 by putting
>
> rm -rf ext/Test-Harness/t/ext
>
> in some part of the distclean target.
>
> But apart from that, I can't see any direct short term benefit from making
> the change.
>
> Nicholas Clark
--
Dave's first rule of Opera:
If something needs saying, say it: don't warble it.
Nicholas Clark
> On Sun, Apr 19, 2009 at 10:11:03PM +0100, Nicholas Clark wrote:
> > On Sun, Apr 19, 2009 at 09:45:30PM +0100, Dave Mitchell wrote:
> > > [ just catching up on an old thread... ]
> > >
> > > On Wed, Mar 25, 2009 at 10:57:02PM +0000, Nicholas Clark wrote:
> >
> > > > No, "this" was the "this" that you thought.
> > > >
> > > > Ignoring syncing-ness of CPAN modules, I think that its probably less work
> > > > overall to merge Schwern's test changes first, then start moving things from
> > > > lib to ext. The more stuff that is in ext/ on the flag day of swapping to
> > > > testing inside ext, the more work that change is.
> > >
> > > Now that I've caught up on most of my old p5p emails and understand the
> > > issues a bit better, my wish is that no further messing with modules is
> > > carried out until after 5.10.1 is released - where messing is defined as
> > > lib/ to ext/ renames, and moving test files into ext/.
> > >
> > > If there are pressing arguments for doing otherwise, I'm open to
> > > persuasion!
> >
> > Specifically, this manicheck failure:
> >
> > ./ext/Test-Harness/t/ext/Test-Harness/t/source_tests/harness not in MANIFEST
>
> Sorry, I'm confused. What's the issue with this file?
It's a generated file, that's not cleaned up by make distclean
(As to why - it's because the relevant Test::Harness regression test sets
things up to archive something, raw TAP files I think, in
ext/Test-Harness/t/something-or-other, and correctly deletes
ext/Test-Harness/t/something-or-other later on.
Only the files generated inside it are of the form $dir/$path_to_test and
$path_to_test is ../ext/Test-Harness/t/source_tests/harness
so the temporary file generated is
ext/Test-Harness/t/something-or-other/../ext/Test-Harness/t/source_tests/harness
ie ext/Test-Harness/t/ext/Test-Harness/t/source_tests/harness
which *isn't* inside the tree ext/Test-Harness/t/something-or-other
so never gets cleaned up.
Hence why it would be fixed naturally by running the tests from
ext/Test-Harness rather than t, as the file would then be found as
t/source_tests/harness, and the output spooled to
ext/Test-Harness/t/something-or-other/t/source_tests/harness
which gets cleaned up when ext/Test-Harness/t/something-or-other is deleted)