RE: Speak up now about your use of EBCDIC or WE WILL REMOVE IT in a future release of Perl
Terwedow, Egon (E.)
We still customize Perl in our USS environment and we use it in a few of our web interfaces. I would appreciate if the open source community will continue to support the product.
This is not highest priority and we do not need latest releases in line with the rest of the Perl community but it would be important to continue the IBM open source commitment to have Perl on the list of supported products.
Please let me know if you decide to stop the support for Perl so that I can inform the user community in my company that this is no longer a viable solution.
Thanks and kind regards from the global Ford company.
Kind Regards,
Egon Terwedow, Supervisor Global Mainframe Engineering
NE/E-119, (8701.8455) ext. +49.221.901.8455 Mobile +49 173 6467850,VOIP 8703-0374
Ford-Werke GmbH,Henry-Ford-Straße 1, 50735 Köln,Sitz der Gesellschaft: Köln,Registergericht Köln, HRB 54183,Vorsitzender des Aufsichtsrats: Stephen Odell,Geschäftsführung: Bernhard Mattes (Vorsitzender), Wolfgang Booms, Dirk Heller, Caspar Hohage, Dr. Hermann H. Hollmann, Rainer Ludwig, Rüdiger Minrath, Dr. Wolfgang Schneider
-----Original Message-----
From: Craig A. Berry [mailto:craig....@gmail.com]
Sent: Freitag, 23. September 2011 21:02
To: Jesse Vincent
Cc: Perl5 Porters; perl...@perl.org
Subject: Re: Speak up now about your use of EBCDIC or WE WILL REMOVE IT in a future release of Perl
On Fri, Sep 23, 2011 at 11:24 AM, Jesse Vincent <je...@fsck.com> wrote:
> We don't currently have evidence that native support for EBCDIC systems in Perl is in use by anyone or that it currently works at all.
>
> If you use Perl on an EBCDIC system, please either reply to this message or contact me off list. If nobody speaks up for EBCDIC, there's a pretty good chance that it will be on the way to Valhalla in the not-too-distant future.
Just adding perl-mvs to the cc list as interested parties would more
likely be found there than on p5p. The impression I have is that no
one with an interest in open source has had access in some years, and,
by process of deduction, no one with access has an interest in open
source.
Jarkko Hietaniemi
> We'd be absolutely thrilled to engage with them to help ensure that Perl
> works well on z/OS, but without some involvement from IBM, it'd be quite
> difficult for us to do so.
>
> Are IBM's changes to Perl 5.8.7 freely available?
AFAICT all of (or 90+% of it) those changes have been merged into the
mainline. Search the perl5-porters archives for "z/os", maybe 4-6 years
ago. IIRC there has been at least two batches of patches (ha!):
once from some IBM India, and once from some IBM China. Or maybe search
for the approriate hints changes.
> Does IBM have a way to provide free access to z/OS for opensource
> developers?
No. I tried several times, but the people inside IBM (or the IBM people
within z/os) simply don't get this "open source thing". They expect
companies, and companies paying Real Money, and companies
Signing With IBM to do this work. About two years ago I tried to work
with Merijn Broeren's help to make IBM understand, to no avail.
We lucked out about ten years ago in that somebody within IBM
understood the benefit of having such access, and I got access to a
z/os box for a few weeks, and NI-S had access to a TI z/os box, so
we were able to port early 5.8.x releases. But that window is no more
there.
Executive summary: some current users of the Perl on z/OS need to step
up, and do the porting/support, or it will not happen.
Jarkko Hietaniemi
> Executive summary: some current users of the Perl on z/OS need to step
> up, and do the porting/support, or it will not happen.
#ifdef EBCDIC etc. code? Is it such a maintenance burden? I thought it
was reasonably well contained.
Peter Prymmer
What would satisfy this request? A patch to a hints file?
Peter Prymmer
-----Original Message-----
From: j...@iki.fi [mailto:jhiet...@gmail.com] On Behalf Of Jarkko Hietaniemi
Sent: Monday, September 26, 2011 12:11 PM
To: j...@iki.fi
Cc: Jesse Vincent; John Goodyear; Craig A. Berry; perl...@perl.org; Perl5 Porters
Subject: Re: Speak up now about your use of EBCDIC or WE WILL REMOVE IT in a future release of Perl
>
Jarkko Hietaniemi
> Doesn't taking it out represent more (potentially risky) work?
(hopefully eventual) restoring of the platform quite a bit messier
(since the code will live, making patching harder).
Jarkko Hietaniemi
> almost certain to be broken given various changes, and no one knows
> how much work it would be to fix it, nor can anyone test those fixes.
I understand that. But as long as you don't have access, you have no
way of knowing whether or nor how much of it is broken. If the code is
removed, it will be 100% broken.
> Leon
>
Jarkko Hietaniemi
>
> Without reaching something like this, longer term, the steady state of
> EBCDIC is going to be "broken", which benefits no-one.
Fair enough. Not my problem, either way.
> Nicholas Clark
>
Peter Prymmer
EBCDIC support. Wholesale removal of the port(s) leaves a lot
of work for the prospective future patcher to get started with,
whereas a clear #error in running miniperl might be more easily
fixed. I do not think ASCII based perl maintainers have to consider
the EBCDIC ports too much. Writing code (especially tests) that breaks
EBCDIC does happen and can be fixed. The burden of providing the fix
is raised quite a bit higher if the EBCDIC stuff already in is removed.
Peter Prymmer
Dean, Sandy
Sandy Dean
-----Original Message-----
From: Nicholas Clark [mailto:ni...@flirble.org] On Behalf Of Nicholas Clark
Sent: Monday, September 26, 2011 2:47 PM
To: Terwedow, Egon (E.)
Cc: Craig A. Berry; Jesse Vincent; Perl5 Porters; perl...@perl.org
Subject: Re: Speak up now about your use of EBCDIC or WE WILL REMOVE IT in a future release of Perl
On Mon, Sep 26, 2011 at 06:41:03AM +0000, Terwedow, Egon (E.) wrote:
> Hello,
>
> We still customize Perl in our USS environment and we use it in a few of our web interfaces. I would appreciate if the open source community will continue to support the product.
We simply aren't in a position to support the product, unaided.
None of the current core perl developers have access to any EBCDIC platform
since they became core developers. Those core developers that did have
access did so nearly 10 years ago, and have either moved to other projects
or died. IBM does not offer any sort access to EBCDIC based systems for us
to test things.
> This is not highest priority and we do not need latest releases in line with the rest of the Perl community but it would be important to continue the IBM open source commitment to have Perl on the list of supported products.
>
> Please let me know if you decide to stop the support for Perl so that I can inform the user community in my company that this is no longer a viable solution.
De facto there is already no support from the community for Perl on EBCDIC
platforms, because the community has *no* access to them, or proxy access
via the "remote hands" of volunteers.
Note also that removing the EBCDIC-related code from the current version in
no way actually changes your current situation. Nor does leaving it in.
Removing it won't overnight stop the code you have from working.
Leaving it in won't make the current version work on EBCDIC.
Either way, your code is just as supported tomorrow as it was yesterday.
ie it's not. (Unless you have a support contract from IBM that I'm not
aware of.)
We've already dropped support for 5.8.x and 5.10.x, across the board.
(I'm guessing that EBCDIC users are on 5.8.x or earlier)
Really we need help (money alone won't cut it, unless it's enough to *buy*,
host and maintain an EBCDIC system) from either EBCDIC users who value Perl,
or IBM, as the vendor whose commercial platform benefits from having Perl
working on it.
It would be helpful to us, if EBCDIC users, as IBM customers, were to ask
IBM how IBM plan to help support Perl on EBCDIC.
Nicholas Clark
Dean, Sandy
What is IBM's stance on supporting perl from this point
forward? I am a heavy user of perl in the USS environment, and would
need to redesign my applications if perl were to go unsupported.
The open source group currently supporting perl on z/os is unable to
continue because as things break (which they have), they have no
platform on which to test, and no z/os developers who have the time to
step up to the task.
It is looking like IBM will need to officially support this product on
USS or it is going to die a painful death.
Please let me know your thoughts.
Here is their response:
IBM will continue to support the Ported Tools version of Perl for the
forseeable future. It's separate from the open source code base.
.
If and when the decision is made to support a newer version of Perl, IBM
will tackle the EBCDIC aspects.
Jarkko Hietaniemi
There should be no 'forks' of open source projects maintained by
vendors. That just leads into pain and suffering. Or at the very
least divergence and obsolescence.
--
There is this special biologist word we use for 'stable'. It is
'dead'. -- Jack Cohen
Jarkko Hietaniemi
>
>
> On Fri, Sep 30, 2011 at 8:32 AM, Konovalov, Vadim (Vadim)** CTR **
>>
>> There is another approach from their, which is quite unlikely: they could
>> start spending their efforts and resources on improving EBCDIC on perl,
>> but I very much doubt so.
>
> a sensible approach is to define an EBCDIC input-output layer and be done
> with it.
You haven't looked at the EBCDIC code, have you?
Jarkko Hietaniemi
>> You haven't looked at the EBCDIC code, have you?
>
> dd conv=ebcdic | perl ... | dd conv=ascii
> I had an acount on a "zpenguin" for a while and that environment was all
> ascii. Also opposite-endian from x86, which made it a really stupid choice
> for testing bit-twiddling methods for accelerating utf8 character counts via
> machine word operations.
This is not Linux. This is not zpenguin. In the real z/os
environment there's nothing ASCII. perl -le 'print ord("A")' doesn't
do what you expect.
Karl Williamson
> Here is the question I asked IBM:
>
> What is IBM's stance on supporting perl from this point
> forward? I am a heavy user of perl in the USS environment, and would
> need to redesign my applications if perl were to go unsupported.
>
> The open source group currently supporting perl on z/os is unable to
> continue because as things break (which they have), they have no
> platform on which to test, and no z/os developers who have the time to
> step up to the task.
>
> It is looking like IBM will need to officially support this product on
> USS or it is going to die a painful death.
>
> Please let me know your thoughts.
>
> Here is their response:
>
> IBM will continue to support the Ported Tools version of Perl for the
> forseeable future. It's separate from the open source code base.
> .
> If and when the decision is made to support a newer version of Perl, IBM
> will tackle the EBCDIC aspects.
>
the locale is utf8. We also have Encode available to translate EBCDIC
data to Unicode on input, and back on output.
It's quite likely that the current EBCDIC bugs have an easy solution.
I've already figured out a trivial fix for the cases I know about where
there are hard-coded UTF-8 vs UTF-EBCDIC string constants.
(I also have an idea for a trivial change to generate Unicode property
tables translated into EBCDIC, and that might make the rest of the code
cleaner in a number of places.)
But without a test bed, we would never be sure that anything we do
actually works.
It's likely that many CPAN modules are written without consideration of
EBCDIC. Some of these are easy to fix. For example, hard-coded hex
character constants can be made to work simply by writing, e.g. \N{U+DF}
instead of \x{DF}. The distinction between the UTF forms is harder to
deal with. As are assumptions like the ordinal of I is 1 less than the
ordinal of J. I have no idea how common those things are in modules,
but I do know that most of those are ok in the Perl core.
I wasn't around at the time, but I have gotten the sense that there is
bad blood between IBM and the Perl developer community. IBM appears to
be unwilling to do anything until and if there is sufficient demand from
its current or potential customers. If none of those customers can
offer us computer time for testing, that tells me there is insufficient
demand for up-to-date Perls.
Jarkko Hietaniemi
pigheadedness of IBM, why they don't see how the current model doesn't
work. Their current model of maintaining an obsolete fork doesn't work.
Well, to be fair, they do every couple of years throw a bunch of poorly
documented patches over the wall. But they have never responded to any
of our feedback and/or questions about the patches. Some of them have
been integrated, some haven't.
Personally, I'm mightily irritated at my and NI-S' work being wasted
this way. And I think we are fooling ourselves if we claim that it'll
be easy to "merge the changes back in". It was a mess.
The assumptions of ASCII run deep.
Karl Williamson
I would hate to see it wasted as well. And I like the elegance of it.
But, even if IBM doesn't come through, as it seems clear, they won't.
We could still continue to support EBCDIC on Perl if we were able to get
access to an EBCDIC machine for testing and minimal development. That
means that all we would need is for a user with the right environment to
step forward. Failing that, it means that no user (if not personally,
then their company) finds it sufficiently important to have more modern
Perls working in an EBCDIC environment.
Nicholas Clark
>
> > It is my understanding that more modern Perls will compile and run when the
> > locale is utf8. We also have Encode available to translate EBCDIC data to
> > Unicode on input, and back on output.
> >
>
> users are left in the same situation as cp1252 and UTF-8 users (the need to
> encode outputs and decode inputs), except for the inability to read source
> files. That ability can be added by having the parser automatically decode
> source files from EBCDIC (as it does for UTF-8 files under "use utf8;") on
> that platform.
>
> Unless I am missing something, this would make it easy to re-add EBCDIC
> support.
a: I think that you're missing the possibility of EBCDIC users having XS
code that interfaces to local C libraries that are assuming EBCDIC
b: My hunch is that
i) Craig is right - if all the EBCDIC related code were removed in one
(or few) clearly labeled the tools associated with git would make it
fairly easy to track where it should be added back in (even if it
doesn't automatically apply)
ii) that won't make it easy to make things *work*.
but that in terms of total effort, it won't be much different because
*all* the existing EBCDIC code is suspect, and will have to be reviewed,
because in recent years some chunks of it will have been edited "in good
faith" on non-EBCDIC systems, without ever being tested on an EBCDIC
platform.
Hence it has to be reviewed, whether it's being added back, or whether
it stayed in.
and I then feel the urge to restate this:
Without an ongoing facility on which to regularly test the development
version of perl, breakages are going to slowly accumulate. This will happen
even if the people making the changes to the perl core are thinking of
EBCDIC, and trying not to break it.
[Consider what happens at the moment with Win32 and VMS, which are regularly
tested. There are still lots of small test breakages, because most people
are developing on some flavour of Unix, and don't even realise that they
have made a portability error. These get fixed fairly quickly, because "we"
know about them, and some of "us" have access to the relevant platforms,
understand them well enough, and personally have reason to care about them
still working. Which means that future changes don't come to rely on the
wrongness.]
So what's needed are
1) a testing facility just to stand still.
2) the pool of active core developers expands to include people who care
about keeping perl on EBCDIC working.
The existing core developers ("the community") is not able to provide either
of the above. It seems that IBM isn't able to provide either of the above.
HP do not make such a mistake with VMS - they provide access to OpenVMS
systems to facilitate open source porting.
Just for once, HP is doing something right. :-) IBM should learn from HP.
Nicholas Clark
Nicholas Clark
> b: My hunch is that
> i) Craig is right - if all the EBCDIC related code were removed in one
> (or few) clearly labeled the tools associated with git would make it
oops.
I wonder if the rediff.com autoresponder is going to send me a reply for
both e-mails I sent. Hateful clueless thing.
Nicholas Clark