RFC: Put s2p on CPAN and remove it from the core perl distribution

[Steve Hay]

The suggestion has come up a couple of times now to dual-life s2p, and
then to remove it from the core perl distribution:

http://www.nntp.perl.org/group/perl.perl5.porters/2013/06/msg203768.html
http://irclog.perlgeek.de/onionsketch/2013-10-17

Does anyone out there object to this proposal, and if so why?

[Johan Vromans]

Steve Hay <steve...@googlemail.com> writes:

> The suggestion has come up a couple of times now to dual-life s2p, and
> then to remove it from the core perl distribution:

> Does anyone out there object to this proposal, and if so why?

No objections. s2p may go.

The same applies to a2p.

Yesterday, Perl5 turned 19. Do we still need pl2pm actively maintained
in the core?

-- Johan

[Brian Fraser]

On Fri, Oct 18, 2013 at 2:23 PM, Steve Hay <steve...@googlemail.com> wrote:

The suggestion has come up a couple of times now to dual-life s2p, and
then to remove it from the core perl distribution:

+1

 

http://www.nntp.perl.org/group/perl.perl5.porters/2013/06/msg203768.html
http://irclog.perlgeek.de/onionsketch/2013-10-17

Does anyone out there object to this proposal, and if so why?

My non-objection is that this falls short. Why not move the entirety of x2p?

[David Golden]

On Fri, Oct 18, 2013 at 1:01 PM, Leon Timmermans <faw...@gmail.com> wrote:

> On Fri, Oct 18, 2013 at 4:56 PM, Brian Fraser <fras...@gmail.com> wrote:
>>
>> My non-objection is that this falls short. Why not move the entirety of
>> x2p?
>
>

> But who would maintain a2p? Who knows enough of C, yacc and awk and is
> willing to do it? I think that's the main problem with that right now.

If no one is maintaining it in core, no one can maintain it on CPAN
just as well. Dual life it, ship it, chown it to ABANDONED and if
someone motivated wants to maintain it, they can step up.

--
David Golden <x...@xdg.me>
Take back your inbox! → http://www.bunchmail.com/
Twitter/IRC: @xdg

[Leon Timmermans]

On Fri, Oct 18, 2013 at 4:56 PM, Brian Fraser <fras...@gmail.com> wrote:

My non-objection is that this falls short. Why not move the entirety of x2p?

But who would maintain a2p? Who knows enough of C, yacc and awk and is willing to do it? I think that's the main problem with that right now.

Leon

[Ricardo Signes]

* David Golden <x...@xdg.me> [2013-10-18T13:07:41]

> On Fri, Oct 18, 2013 at 1:01 PM, Leon Timmermans <faw...@gmail.com> wrote:
> > But who would maintain a2p? Who knows enough of C, yacc and awk and is
> > willing to do it? I think that's the main problem with that right now.
>
> If no one is maintaining it in core, no one can maintain it on CPAN
> just as well. Dual life it, ship it, chown it to ABANDONED and if
> someone motivated wants to maintain it, they can step up.

That's my feeling as well.

--
rjbs

[Nicholas Clark]

I think that it would be better to split apart a2p, find2perl, s2p as
three separate distributions, rather than a single x2p distribution, as it's
more a historical accident that all 3 are together, rather than a design or
implementation reason.

It looks like it would be pretty easy to refactor find2perl and s2p into
regular dual life extensions. Each already has tests.

The fun comes with a2p. It *isn't* a regular extension. It isn't a Perl
program. It's a stand alone C program. It's not something that fits what
ExtUtils::MakeMaker builds. x2p/ isn't a "regular" extension directory, and
doesn't have a "regular" build, so it's not just a case of moving it to dist/
and then uploading it to CPAN. It's going to take quite a bit more unteasing.
(And I'm not even sure how much). It also doesn't have any tests, not even
basic sanity tests. So there's nothing to verify that changing the build
setup continues to produce a viable executable.


So I think deal with find2perl and s2p, and don't let them block on a2p.

Nicholas Clark

[Steffen Mueller]

On 10/22/2013 02:12 PM, Nicholas Clark wrote:
> The fun comes with a2p. It *isn't* a regular extension. It isn't a Perl
> program. It's a stand alone C program. It's not something that fits what
> ExtUtils::MakeMaker builds. x2p/ isn't a "regular" extension directory, and
> doesn't have a "regular" build, so it's not just a case of moving it to dist/
> and then uploading it to CPAN. It's going to take quite a bit more unteasing.
> (And I'm not even sure how much). It also doesn't have any tests, not even
> basic sanity tests. So there's nothing to verify that changing the build
> setup continues to produce a viable executable.

Sounds like one could fairly easily move the C code to an extension
invoked via Perl, then rewrite just the outermost layer of code in Perl
so that a2p itself becomes a perl script that loads the A2P (or
whatever) module, which is an XS module providing the logic generated by
yacc (etc).

No. I'm not volunteering. Not an itch I feel like scratching.

--Steffen

[Leon Timmermans]

On Tue, Oct 22, 2013 at 2:12 PM, Nicholas Clark <ni...@ccl4.org> wrote:

I think that it would be better to split apart a2p, find2perl, s2p as
three separate distributions, rather than a single x2p distribution, as it's
more a historical accident that all 3 are together, rather than a design or
implementation reason.

Agreed, that makes sense.

 

It looks like it would be pretty easy to refactor find2perl and s2p into
regular dual life extensions. Each already has tests.

Yeah, no reason why that should be particularly difficult.

 

The fun comes with a2p. It *isn't* a regular extension. It isn't a Perl
program. It's a stand alone C program. It's not something that fits what
ExtUtils::MakeMaker builds. x2p/ isn't a "regular" extension directory, and
doesn't have a "regular" build, so it's not just a case of moving it to dist/
and then uploading it to CPAN. It's going to take quite a bit more unteasing.
(And I'm not even sure how much).

Actually, it seems it's mostly a matter of copying a few lines from x2p/Makefile.SH to a Makefile.PL. The only complication is supporting MSVC. I'm not sure that's relevant anyway, I can't imagine people on windows using a2p in the first place.
 

It also doesn't have any tests, not even
basic sanity tests. So there's nothing to verify that changing the build
setup continues to produce a viable executable.

That is a bigger issue, and not one I can fix. That said, at this stage I think we should settle for basic sanity tests. OTOH, we only need a motivated awk user to fix this.

Leon

[Aristotle Pagaltzis]

* Leon Timmermans <faw...@gmail.com> [2013-11-04 23:05]:

Why not engage in some pirate testing?

There is a short AWK test suite in the busybosy project:
http://git.busybox.net/busybox/tree/testsuite/awk.tests
http://git.busybox.net/busybox/tree/testsuite/awk_t1.tar.bz2

… and a large one in the GNU AWK project:
http://git.savannah.gnu.org/cgit/gawk.git/tree/test

(Brian Kernighan himself also writes about maintaining a large test
suite for original AWK: http://www.cs.princeton.edu/~bwk/testing.html
However I have been unable to find that anywhere online, as Brian seems
to make available only the sources of the interpreter itself.)

Feed their test cases to a2p, record its outputs, create a test driver
that asserts its status quo, then refactor with impunity. Seems like
a day-at-best, week-at-worst kinda job to me.

(Possibly the test cases themselves can be performed, to check that a2p
code does what the original code would have – but I don’t know how much
conformance a2p actually claims, so that may not be viable. It would be
extra work over just conserving a2p’s current output, too.)

--
Aristotle Pagaltzis // <http://plasmasturm.org/>

[Leon Timmermans]

Since no one appeared to be standing up, I took the liberty to convert them

https://github.com/Leont/app-find2perl
https://github.com/Leont/app-s2p
https://github.com/Leont/app-a2p

As expected, the first two were fairly easy conversions; some minor tweaks in the code and the tests and for s2p a moderately interesting Makefile (it copies s2p to psed during build-time). a2p's build was a bit more interesting^Wcomplicated, but less so than one may expect.

That said, I have little interest in actually maintaining these, unless people on this list want to adopt them I propose a public call for volunteers. The first two don't require any abilities other than ordinary perl and cpan skills.

Leon

[Leon Timmermans]

On Thu, Nov 7, 2013 at 2:33 AM, Leon Timmermans <faw...@gmail.com> wrote:

Since no one appeared to be standing up, I took the liberty to convert them

https://github.com/Leont/app-find2perl
https://github.com/Leont/app-s2p
https://github.com/Leont/app-a2p

As expected, the first two were fairly easy conversions; some minor tweaks in the code and the tests and for s2p a moderately interesting Makefile (it copies s2p to psed during build-time). a2p's build was a bit more interesting^Wcomplicated, but less so than one may expect.

That said, I have little interest in actually maintaining these, unless people on this list want to adopt them I propose a public call for volunteers. The first two don't require any abilities other than ordinary perl and cpan skills.

I'm not sure how to proceed. Is it desirable to convert the first two into dists in the code distribution? Scripts are sufficiently special-cased in perl's build that that I'm not sure it's really worth it (and I'd have to rewrite the tests to not using non-core dependencies but work both inside and outside of core). I could put them on CPAN right away.

Don't think that would work for a2p though without some serious work, given that Windows and VMS (and Netware…) are being unique snowflakes and the lack of precedents of binary utils. I'm not sure either how to proceed if(/when?) we get stuck there.

Leon

[bulk88]

Leon Timmermans wrote:
> Don't think that would work for a2p though without some serious work,
> given that Windows and VMS (and Netware…) are being unique snowflakes
> and the lack of precedents of binary utils. I'm not sure either how to
> proceed if(/when?) we get stuck there.
>
> Leon

---------------------------------------
C:\Documents and Settings\Owner\Desktop\cpan libs\a2p>perl makefile.pl
Warning: Guessing NAME [a2p] from current directory name.
Generating a nmake-style Makefile
Writing Makefile for a2p
Writing MYMETA.yml and MYMETA.json

C:\Documents and Settings\Owner\Desktop\cpan libs\a2p>nmake

Microsoft (R) Program Maintenance Utility Version 7.10.3077
Copyright (C) Microsoft Corporation. All rights reserved.

cp lib/App/a2p.pm blib\lib\App\a2p.pm
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DN
O_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DU
SE_PERLIO -D_USE_32BIT_TIME_T -O1 -MD -Zi -DNDEBUG -GL -DVERSION=\"\"
-DXS_V
ERSION=\"\" "-IC:\perl519\lib\CORE" hash.c
hash.c
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DN
O_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DU
SE_PERLIO -D_USE_32BIT_TIME_T -O1 -MD -Zi -DNDEBUG -GL -DVERSION=\"\"
-DXS_V
ERSION=\"\" "-IC:\perl519\lib\CORE" str.c
str.c
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DN
O_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DU
SE_PERLIO -D_USE_32BIT_TIME_T -O1 -MD -Zi -DNDEBUG -GL -DVERSION=\"\"
-DXS_V
ERSION=\"\" "-IC:\perl519\lib\CORE" util.c
util.c
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DN
O_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DU
SE_PERLIO -D_USE_32BIT_TIME_T -O1 -MD -Zi -DNDEBUG -GL -DVERSION=\"\"
-DXS_V
ERSION=\"\" "-IC:\perl519\lib\CORE" walk.c
walk.c
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DN
O_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DU
SE_PERLIO -D_USE_32BIT_TIME_T -O1 -MD -Zi -DNDEBUG -GL -DVERSION=\"\"
-DXS_V
ERSION=\"\" "-IC:\perl519\lib\CORE" a2p.c
a2p.c
cl -o blib\bin/a2p -nologo -nodefaultlib -debug -opt:ref,icf
-ltcg -lib
path:"c:\perl519\lib\CORE" -machine:x86 hash.obj str.obj util.obj
walk.obj a2p.
obj oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib ad
vapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib
ws2_32.lib
mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib
msvcrt.lib
cl : Command line warning D4002 : ignoring unknown option '-nodefaultlib'
cl : Command line warning D4002 : ignoring unknown option '-debug'
cl : Command line warning D4002 : ignoring unknown option '-ltcg'
cl : Command line warning D4002 : ignoring unknown option
'-libpath:c:\perl519\l
ib\CORE'
cl : Command line warning D4002 : ignoring unknown option '-machine:x86'
hash.obj : warning LNK4218: non-native module found; restarting link
with /LTCG
LINK : fatal error LNK1181: cannot open input file 'oldnames.lib'
NMAKE : fatal error U1077: 'cl' : return code '0x2'
Stop.

C:\Documents and Settings\Owner\Desktop\cpan libs\a2p>nmake -n all

Microsoft (R) Program Maintenance Utility Version 7.10.3077
Copyright (C) Microsoft Corporation. All rights reserved.

rem
rem
rem
rem
rem
rem
cl -o blib\bin/a2p -nologo -nodefaultlib -debug -opt:ref,icf
-ltcg -lib
path:"c:\perl519\lib\CORE" -machine:x86 hash.obj str.obj util.obj
walk.obj a2p.
obj oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib ad
vapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib
ws2_32.lib
mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib
msvcrt.lib
rem

C:\Documents and Settings\Owner\Desktop\cpan libs\a2p>
-------------------------------------------------------

The recipe in the core win32/Makefile is
-------------------------------------------------------
$(X2P) : $(MINIPERL) $(X2P_OBJ) Extensions
$(MINIPERL) -I..\lib ..\x2p\find2perl.PL
$(MINIPERL) -I..\lib ..\x2p\s2p.PL
>>>>> $(LINK32) -subsystem:console -out:$@ @<<
$(LINK_FLAGS) $(LIBFILES) $(X2P_OBJ)
-------------------------------------------------------
But unfortunately I can't think of a EUMM module that makes exes on
Windows. I'll go searching CPAN. Results below

https://metacpan.org/source/CERNEY/Tk-TableMatrix-1.23/pTk/mTk/Tktable/makefile.vc

https://metacpan.org/source/ILYAZ/Math-Pari-2.01080605/libPARI/Makefile.PL
(it has to be easier than this)

win32 perl interp make test compiles an exe at but not EUMM
http://perl5.git.perl.org/perl.git/blob/HEAD:/t/win32/system.t#l107

finally, an EUMM making an exe makefile.pl but I'm not sure that is
Mingw compatible
https://metacpan.org/source/DOUGM/ExtUtils-Embed-1.14/genmake

[Leon Timmermans]

On Mon, Nov 11, 2013 at 1:37 PM, Leon Timmermans <faw...@gmail.com> wrote:

I'm not sure how to proceed. Is it desirable to convert the first two into dists in the code distribution? Scripts are sufficiently special-cased in perl's build that that I'm not sure it's really worth it (and I'd have to rewrite the tests to not using non-core dependencies but work both inside and outside of core). I could put them on CPAN right away.

Don't think that would work for a2p though without some serious work, given that Windows and VMS (and Netware…) are being unique snowflakes and the lack of precedents of binary utils. I'm not sure either how to proceed if(/when?) we get stuck there.

*Bump*

Leon

[Ricardo Signes]

* Leon Timmermans <faw...@gmail.com> [2013-11-11T07:37:11]

> I'm not sure how to proceed. Is it desirable to convert the first two into
> dists in the code distribution? Scripts are sufficiently special-cased in
> perl's build that that I'm not sure it's really worth it (and I'd have to
> rewrite the tests to not using non-core dependencies but work both inside
> and outside of core). I could put them on CPAN right away.

Mostly, I think we've discussed the goal of distification being the ejection of
these programs from core, since they're not being maintained by core. (They're
bug reports waiting to be filed and ignored.)

If that's the goal, and I think it is, then they should be distified. Core
doesn't need to adapt to the dists, as long as the dists, when installed,
shadow or replace the core versions. They can probably languish in core,
getting zero updates, until their ejection in 5.24.

Does that make sense?

--
rjbs

[Leon Timmermans]

Yes that makes sense. I've just release App::find2perl and App::s2p to CPAN. App::a2p is just a «dzil release» away from being on CPAN, but lacking tests I don't really feel confident in doing that.

Leon

[Steffen Mueller]

On 12/03/2013 06:02 PM, Leon Timmermans wrote:
> Yes that makes sense. I've just release App::find2perl and App::s2p to
> CPAN. App::a2p is just a «dzil release» away from being on CPAN, but
> lacking tests I don't really feel confident in doing that.

Thank you very much for doing that.

As far as I am concerned, a2p is untested in core and nobody is rushing
to its aid. So releasing it to CPAN without tests is unlikely to make
matters worse. Let's keep in mind what software we're dealing with here.
I'd be moderately surprised if this thing was very widely used nowadays.

--Steffen

[Ricardo Signes]

* Leon Timmermans <faw...@gmail.com> [2013-12-03T12:02:29]

> App::a2p is just a «dzil release» away from being on CPAN, but
> lacking tests I don't really feel confident in doing that.

* Steffen Mueller <smue...@cpan.org> [2013-12-03T13:10:16]

> As far as I am concerned, a2p is untested in core and nobody is
> rushing to its aid. So releasing it to CPAN without tests is
> unlikely to make matters worse. Let's keep in mind what software
> we're dealing with here. I'd be moderately surprised if this thing
> was very widely used nowadays.

I agree with Steffen.

If you'd rather not have your name plastered on it in case of complaints, let
me know and I'll do the releasing — just don't expect me to add any tests. ;)

--
rjbs

[Leon Timmermans]

On Tue, Dec 3, 2013 at 9:54 PM, Ricardo Signes <perl...@rjbs.manxome.org> wrote:

* Steffen Mueller <smue...@cpan.org> [2013-12-03T13:10:16]

> As far as I am concerned, a2p is untested in core and nobody is
> rushing to its aid. So releasing it to CPAN without tests is
> unlikely to make matters worse. Let's keep in mind what software
> we're dealing with here. I'd be moderately surprised if this thing
> was very widely used nowadays.

I agree with Steffen.

 

In that case…

https://metacpan.org/release/App-find2perl
https://metacpan.org/release/App-s2p
https://metacpan.org/release/App-a2p

If you'd rather not have your name plastered on it in case of complaints, let
me know and I'll do the releasing — just don't expect me to add any tests. ;)

"If you care enough this module to complain, why don't you take it over?…" ;-)

Leon

[Leon Timmermans]

On Wed, Dec 4, 2013 at 1:50 AM, Leon Timmermans <faw...@gmail.com> wrote:

On Tue, Dec 3, 2013 at 9:54 PM, Ricardo Signes <perl...@rjbs.manxome.org> wrote:

* Steffen Mueller <smue...@cpan.org> [2013-12-03T13:10:16]

> As far as I am concerned, a2p is untested in core and nobody is
> rushing to its aid. So releasing it to CPAN without tests is
> unlikely to make matters worse. Let's keep in mind what software
> we're dealing with here. I'd be moderately surprised if this thing
> was very widely used nowadays.

I agree with Steffen.

 

In that case…

https://metacpan.org/release/App-find2perl
https://metacpan.org/release/App-s2p
https://metacpan.org/release/App-a2p

They're all on CPAN; I've just released a new a2p that actually works on msvc. Next step is to announce in 5.20 that it will be removed before 5.22.

Actually, there are a few more of such utilities that we really need to get rid of. Almost all of the code c2ph is older than our youngest committer. h2ph is almost useful,pl but being older than CPAN it's pretty much unusable in distributions, severely limiting its use. pl2pm as Johan suggested is not particularly useful either. Do we want to announce anything about those too?

Leon

[Steffen Mueller]

On 03/13/2014 05:57 PM, Leon Timmermans wrote:
> They're all on CPAN; I've just released a new a2p that actually works on
> msvc. Next step is to announce in 5.20 that it will be removed before 5.22.
>
> Actually, there are a few more of such utilities that we really need to
> get rid of. Almost all of the code c2ph is older than our youngest
> committer. h2ph is almost useful,pl but being older than CPAN it's
> pretty much unusable in distributions, severely limiting its use. pl2pm
> as Johan suggested is not particularly useful either. Do we want to
> announce anything about those too?

Yes, please. Good catch.

--Steffen

[Leon Timmermans]

On Thu, Mar 13, 2014 at 7:01 PM, bulk 88 <bul...@hotmail.com> wrote:

________________________________
> From: faw...@gmail.com
> Date: Thu, 13 Mar 2014 17:57:19 +0100
> Subject: Re: RFC: Put s2p on CPAN and remove it from the core perl
> distribution
> To: perl...@rjbs.manxome.org
> CC: smue...@cpan.org; ni...@ccl4.org; fras...@gmail.com;
> steve...@googlemail.com; perl5-...@perl.org

>
> On Wed, Dec 4, 2013 at 1:50 AM, Leon Timmermans

> <faw...@gmail.com<mailto:faw...@gmail.com>> wrote:
>
> On Tue, Dec 3, 2013 at 9:54 PM, Ricardo Signes

> <perl...@rjbs.manxome.org<mailto:perl...@rjbs.manxome.org>> wrote:
> * Steffen Mueller <smue...@cpan.org<mailto:smue...@cpan.org>>

> [2013-12-03T13:10:16]
> > As far as I am concerned, a2p is untested in core and nobody is
> > rushing to its aid. So releasing it to CPAN without tests is
> > unlikely to make matters worse. Let's keep in mind what software
> > we're dealing with here. I'd be moderately surprised if this thing
> > was very widely used nowadays.
>
> I agree with Steffen.
>
> In that case…
>
> https://metacpan.org/release/App-find2perl
> https://metacpan.org/release/App-s2p
> https://metacpan.org/release/App-a2p
>
> They're all on CPAN; I've just released a new a2p that actually works
> on msvc. Next step is to announce in 5.20 that it will be removed
> before 5.22.

a2p needs a -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE                                         

Patches (either way) welcome ;-)

Leon

[Leon Timmermans]

On Thu, Mar 13, 2014 at 5:57 PM, Leon Timmermans <faw...@gmail.com> wrote:

Next step is to announce in 5.20 that it will be removed before 5.22.

Just added that warning to perldelta for 5.20, and pushed a patch for removing them to branch leont/nox2p

 

Actually, there are a few more of such utilities that we really need to get rid of. Almost all of the code c2ph is older than our youngest committer. h2ph is almost useful,pl but being older than CPAN it's pretty much unusable in distributions, severely limiting its use. pl2pm as Johan suggested is not particularly useful either. Do we want to announce anything about those too?

It seems that discussion is left for 5.21…

Leon