Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

CRLF in Unix being translated on Mainframe to x'25'

3,788 views
Skip to first unread message

Lizette Koehler

unread,
Oct 24, 2011, 4:11:20 PM10/24/11
to
A programmer I am working with is getting a file from Unix which will be sent to the mainframe. This will be using FTP.

The process is we receive the file from the Unix system and it is placed on a Windows 2008 server.

Then from the Windows 2008 it is going to be FTP'd to the mainframe.

Is there a way to keep the CRLF command at the end of each line? And will the mainframe recognize it as a newline?

What we really want is the UNIX file to be individual (FB Lrecl 946) going into the mainframe.

It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'


Thanks for any advice

Lizette


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to list...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Scott Ford

unread,
Oct 24, 2011, 4:29:14 PM10/24/11
to
Lizette:
 
Yep sure is, use binary, it will not translate the characters. I use it all the time for exactly that purpose

Scott J Ford
Software Engineer
http://www.identityforge.com
 


________________________________
From: Lizette Koehler <star...@MINDSPRING.COM>
To: IBM-...@bama.ua.edu
Sent: Monday, October 24, 2011 4:10 PM
Subject: CRLF in Unix being translated on Mainframe to x'25'

Jonathan Goossen

unread,
Oct 24, 2011, 4:35:44 PM10/24/11
to
Lizette,
It has been years since I worked with Unix FTP, but Windows also uses the
CRLF and I routinely FTP between my PC and the Mainframe. All I do is
specify a text transfer and the FTP utility does the translation. I get
one PC text line to one mainframe text line. Mine are usually FB LRECL 80.
So the only difference is the LRECL.

Thank you and have a Terrific day!

Jonathan Goossen, ACG, CL
Tape Specialist
ACT Mainframe Storage Group
Personal: 651-361-4541
Department Support Line: 651-361-5555
For help with communication and leadership skills checkout Woodwinds
Toastmasters

IBM Mainframe Discussion List <IBM-...@bama.ua.edu> wrote on 10/24/2011
03:10:15 PM:

> From: Lizette Koehler <star...@MINDSPRING.COM>
> To: IBM-...@bama.ua.edu
> Date: 10/24/2011 03:11 PM
> Subject: CRLF in Unix being translated on Mainframe to x'25'
This e-mail message and all attachments transmitted with it may
contain legally privileged and/or confidential information intended
solely for the use of the addressee(s). If the reader of this
message is not the intended recipient, you are hereby notified that
any reading, dissemination, distribution, copying, forwarding or
other use of this message or its attachments is strictly
prohibited. If you have received this message in error, please
notify the sender immediately and delete this message and all
copies and backups thereof. Thank you.

McKown, John

unread,
Oct 24, 2011, 4:46:04 PM10/24/11
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@bama.ua.edu] On Behalf Of Lizette Koehler
> Sent: Monday, October 24, 2011 3:10 PM
> To: IBM-...@bama.ua.edu
> Subject: CRLF in Unix being translated on Mainframe to x'25'
>
> A programmer I am working with is getting a file from Unix
> which will be sent to the mainframe. This will be using FTP.
>
> The process is we receive the file from the Unix system and
> it is placed on a Windows 2008 server.
>
> Then from the Windows 2008 it is going to be FTP'd to the mainframe.
>
> Is there a way to keep the CRLF command at the end of each
> line? And will the mainframe recognize it as a newline?
>
> What we really want is the UNIX file to be individual (FB
> Lrecl 946) going into the mainframe.
>
> It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
>
> Thanks for any advice
>
> Lizette

On most ASCII UNIX system, the end-of-line character is a single LF (LineFeed) or 0x0A.
On Windows, the normal end-of-line is a 0x0D0A (CRLF or CarriageReturn-LineFeed).
On z/OS UNIX, in EBCDIC, the end-of-line character is the NL (NewLine) or 0x15. NOT 0x25. 0x25 is a CarriageReturn.

When you create a file on an ASCII UNIX system, the lines end with a 0x0A. If you store this on a "share" where a Windows server can get at it directly and you use a Windows ftp client, the file in the z/OS UNIX system is "hosed up". Just as you have seen, the lines end with a 0x25 whereas they should end with a 0x15. I view this as an error in the Windows ftp client because it demands that text line end with CRLF instead of just LF. What you can do is have your UNIX process add the "extraneous" CR so that the lines end with a CRLF. There is usually a program called "unix2dos" which can do this. If you don't have "unix2dos", it can be done with a "sed" command: "sed -i 's/$/\r/' file.on.windows". At least, this works with GNU's sed. I don't know about others. Some don't expand \r to CR.

You can fix the file on z/OS with tr: tr '\045' '\025' <bad.file.from.windows >good.output.file

\045 is octal == 0x25. \025 is octal and == 0x15. I can't figure out how to get "tr" to accept hex instead of octal. I hate that UNIX was developed on a PDP-8 (octal machine).

All of the above only works if you're ftp'ing into a z/OS UNIX file. Have fun trying to get it to work into a z/OS sequential dataset. I never have. Stupid Windows ... MS-DOS ... CP/M-80!



--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

McKown, John

unread,
Oct 24, 2011, 4:50:23 PM10/24/11
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@bama.ua.edu] On Behalf Of Lizette Koehler
> Sent: Monday, October 24, 2011 3:10 PM
> To: IBM-...@bama.ua.edu
> Subject: CRLF in Unix being translated on Mainframe to x'25'
>
> A programmer I am working with is getting a file from Unix
> which will be sent to the mainframe. This will be using FTP.
>
> The process is we receive the file from the Unix system and
> it is placed on a Windows 2008 server.
>
> Then from the Windows 2008 it is going to be FTP'd to the mainframe.
>
> Is there a way to keep the CRLF command at the end of each
> line? And will the mainframe recognize it as a newline?
>
> What we really want is the UNIX file to be individual (FB
> Lrecl 946) going into the mainframe.
>
> It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
>
> Thanks for any advice
>
> Lizette

Oh, forgot to mention. If you're using FTP to copy from UNIX to Windows, then be sure to use an ASCII transfer and not a BINARY one. If you use ASCII, the ftp from UNIX to Windows will cause the Windows file to have CRLF instead of just LF. My original reply was assuming that the UNIX system directly wrote to the Windows disk or that Windows was directly reading the UNIX disk (like with CIFS or NFS).

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM



Scott Ford

unread,
Oct 24, 2011, 4:57:14 PM10/24/11
to
John:
 
We package and ship our executes in binary format...We build JES2 xmits, download them as binary and the customer uploads from Windoze to z/OS using a binary option, not 'CR LF' , otherwise the data is unusable. I think if memory servers me right its because we ship loadlibs....

Scott J Ford
Software Engineer
http://www.identityforge.com
 


________________________________
From: "McKown, John" <John....@HEALTHMARKETS.COM>
To: IBM-...@bama.ua.edu
Sent: Monday, October 24, 2011 4:43 PM
Subject: Re: CRLF in Unix being translated on Mainframe to x'25'

Shmuel Metz , Seymour J.

unread,
Oct 24, 2011, 5:17:32 PM10/24/11
to
In
<6598855.131948701...@elwamui-darkeyed.atl.sa.earthlink.net>,
on 10/24/2011
at 04:10 PM, Lizette Koehler <star...@MINDSPRING.COM> said:

>A programmer I am working with is getting a file from Unix which will
>be sent to the mainframe. This will be using FTP.

What are the original and final character sets?

>Is there a way to keep the CRLF command at the end of each line?

Do you mean that the original file has CRLF rather than LF? Also, CRLF
isn't a command just a CR character followed by an LF character. The
easiest way is to use binary mode.

>It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'

No; the Unix new line indication is LF (X'0A'), the windoze/DOS new
line indication is CRLF (X'0D0A') and the z/OS new line indication for
EBCDIC is NEL (X'25'). The LF and NEL represent new line, which is
logically distinct from CRLF.

BTW, do local politics allow going directly to z/OS instead of
therough the W2008 box?

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Lizette Koehler

unread,
Oct 24, 2011, 5:43:20 PM10/24/11
to
>
>BTW, do local politics allow going directly to z/OS instead of
>therough the W2008 box?
>



As for politics.
It must be done this way, Unix, to Win2008, to Mainframe.

No options here.

I have tried several tests. Either I only get one record up on the mainframe or the data is definately in bin format when it arrives. I have tried BIN FB, BIN VB, ASCII FB and ASCII VB. So far the 44 records I have to test with are just not behavingin.

The odd thing is that NOTEPAD cannot read this file well. It wraps it (yes tried the WRAP FORMAT in Notepad no luck). So I can see the records circle around in the notepad window.

However, Notepad++ sees it as individual records.

Still plugging along. I am trying the cmd prompt from my desktop to the mainframe. However, it is possible that this file will be brought up via a batch job on the mainframe. I am trying that now.


Lizette

Scott Ford

unread,
Oct 24, 2011, 6:05:58 PM10/24/11
to
Liz:
 
For NOTEPAD++, which I also use, you will need to make sure that the file that arrives into Windows has the x'0D0A' , as Shmuel said its the CRLF characters.
From Windows to the Host(z/OS) you could go binary for sure. I would verify that the data arrives in Windows ok, before doing the next hope to z/OS.
If the politics would allow it ( i dont miss this ) then a Unix -> z/OS would be desirable and a heck of a lot easier.

Scott J Ford
Software Engineer
http://www.identityforge.com
 


________________________________
From: Lizette Koehler <star...@MINDSPRING.COM>
To: IBM-...@bama.ua.edu
Sent: Monday, October 24, 2011 5:41 PM
Subject: Re: CRLF in Unix being translated on Mainframe to x'25'

>

Grinsell, Don

unread,
Oct 24, 2011, 6:14:15 PM10/24/11
to
Lizette,

I think John's suggestion has merit. On the first transfer (Unix to Windows) are you specifying ascii or binary? Specifying ascii should correctly translate the Unix new line to the Windows CRLF. Then your second transfer (Windows to z/OS) should work correctly. This also explains why Notepad doesn't like the data.

Regards.

--

Donald Grinsell
State of Montana
406-444-2983
dgri...@mt.gov

"Voice or no voice, the people can always be brought to the bidding of the leaders. That is easy. All you have to do is tell them they are being attacked, and denounce the pacifists for lack of patriotism and exposing the country to danger. It works the same in every country."
-- Hermann Goering

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-...@bama.ua.edu] On Behalf Of Lizette Koehler
Sent: Monday, 24 October 2011 15:41
To: IBM-...@bama.ua.edu
Subject: Re: CRLF in Unix being translated on Mainframe to x'25'

>

Farley, Peter x23353

unread,
Oct 24, 2011, 6:22:47 PM10/24/11
to
> -----Original Message-----
> From: IBM Mainframe Discussion List [mailto:IBM-...@bama.ua.edu] On
> Behalf Of Lizette Koehler
> Sent: Monday, October 24, 2011 4:10 PM
> To: IBM-...@bama.ua.edu
> Subject: CRLF in Unix being translated on Mainframe to x'25'
>
> A programmer I am working with is getting a file from Unix which will be
> sent to the mainframe. This will be using FTP.
>
> The process is we receive the file from the Unix system and it is placed
> on a Windows 2008 server.
>
> Then from the Windows 2008 it is going to be FTP'd to the mainframe.
>
> Is there a way to keep the CRLF command at the end of each line? And will
> the mainframe recognize it as a newline?
>
> What we really want is the UNIX file to be individual (FB Lrecl 946) going
> into the mainframe.
>
> It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'

Assuming for the moment you will be using the Windows 2008 FTP client to send the file, ISTM that the key is for the Unix FTP transfer to translate the Unix line ending (X'0A') into the Windows line ending (X'0D0A') when transferring the file to the Windows box. Then the normal Windows FTP client will recognize the line endings (and Notepad will edit it properly as well).

Another alternative if that is not feasible from the Unix-Windows path is to use a Windows version of the Unix utility "utod" (unix-to-dos) to translate the line endings. I believe Cygwin has a "utod" somewhere in one of the text-processing packages. The DJGPP Unix ports to command-mode DOS (16-bit only) definitely have "utod" available. That one may not survive to the next version of Windows though. Win7 already stopped supporting 16-bit applications, so I'm sure the Server versions will follow soon (if they haven't already).

If the Unix-to-Windows transfer is being done as a binary transfer, then the Unix guys can do the translation on their end for you ahead of time, using either "utod" if they have it or they could use the "tr" utility to do the translation before they FTP the file to your Windows server.

As long as the Windows copy of the file has line endings X'0D0A' then even a mainframe-initiated transfer should preserve the record lengths.

HTH

Peter
--


This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.

Farley, Peter x23353

unread,
Oct 24, 2011, 6:32:34 PM10/24/11
to
> -----Original Message-----
> From: Farley, Peter x23353
> Sent: Monday, October 24, 2011 6:22 PM
> To: 'IBM Mainframe Discussion List'
> Subject: RE: CRLF in Unix being translated on Mainframe to x'25'
>
> > -----Original Message-----
> > From: IBM Mainframe Discussion List [mailto:IBM-...@bama.ua.edu] On
> > Behalf Of Lizette Koehler
> > Sent: Monday, October 24, 2011 4:10 PM
> > To: IBM-...@bama.ua.edu
> > Subject: CRLF in Unix being translated on Mainframe to x'25'
> >
> > A programmer I am working with is getting a file from Unix which will be
> > sent to the mainframe. This will be using FTP.
> >
> > The process is we receive the file from the Unix system and it is placed
> > on a Windows 2008 server.
> >
> > Then from the Windows 2008 it is going to be FTP'd to the mainframe.
> >
> > Is there a way to keep the CRLF command at the end of each line? And
> will
> > the mainframe recognize it as a newline?
> >
> > What we really want is the UNIX file to be individual (FB Lrecl 946)
> going
> > into the mainframe.
> >
> > It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
> Assuming for the moment you will be using the Windows 2008 FTP client to
> send the file, ISTM that the key is for the Unix FTP transfer to translate
> the Unix line ending (X'0A') into the Windows line ending (X'0D0A') when
> transferring the file to the Windows box. Then the normal Windows FTP
> client will recognize the line endings (and Notepad will edit it properly
> as well).
>
> Another alternative if that is not feasible from the Unix-Windows path is
> to use a Windows version of the Unix utility "utod" (unix-to-dos) to
> translate the line endings. I believe Cygwin has a "utod" somewhere in
> one of the text-processing packages. The DJGPP Unix ports to command-mode
> DOS (16-bit only) definitely have "utod" available. That one may not
> survive to the next version of Windows though. Win7 already stopped
> supporting 16-bit applications, so I'm sure the Server versions will
> follow soon (if they haven't already).
>
> If the Unix-to-Windows transfer is being done as a binary transfer, then
> the Unix guys can do the translation on their end for you ahead of time,
> using either "utod" if they have it or they could use the "tr" utility to
> do the translation before they FTP the file to your Windows server.
>
> As long as the Windows copy of the file has line endings X'0D0A' then even
> a mainframe-initiated transfer should preserve the record lengths.


I also just remembered to check the z/OS "SITE" parameters available when using the Windows FTP client to send to or receive from the z/OS FTP server. This parameter is available and can be set to "LF" if you need to:

SBSENDEOL=value Specifies the line terminator for outbound SBCS
ASCII data. Value is CRLF, CR, LF, or NONE

I am not sure what "outbound" means in this description; it may only refer to receiving files from the mainframe rather than sending them to the mainframe.

HTH

Peter
--


This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.

Paul Gilmartin

unread,
Oct 24, 2011, 7:01:19 PM10/24/11
to
On Mon, 24 Oct 2011 17:20:14 -0500, Shmuel Metz (Seymour J.) wrote:

>
>on 10/24/2011 at 04:10 PM, Lizette Koehler <star...@MINDSPRING.COM> said:
>
>>A programmer I am working with is getting a file from Unix which will
>>be sent to the mainframe. This will be using FTP.
>
>What are the original and final character sets?
>
>>Is there a way to keep the CRLF command at the end of each line?
>
>Do you mean that the original file has CRLF rather than LF? Also, CRLF
>isn't a command just a CR character followed by an LF character. The
>easiest way is to use binary mode.
>
>>It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
>No; the Unix new line indication is LF (X'0A'), the windoze/DOS new
>line indication is CRLF (X'0D0A') and the z/OS new line indication for
>EBCDIC is NEL (X'25'). The LF and NEL represent new line, which is
>logically distinct from CRLF.
>
Ummm. Not quite. z/OS Unix System Services uses EBCDIC NL (x'15')
as the new line indication. Then it violates the specifications of the
code pages by translating LF(x'0A')<->NL(x'15') and NEL(x'85')<->LF(x'25')
converting ASCII(ISO8859-1)<->EBCDIC(IBM-1047). You once said
that CMS, which doesn't perform this twiddle 'DTRT'.

But, IMO, you have supplied less disinformation than any other contributor
to this thread.

>BTW, do local politics allow going directly to z/OS instead of
>through the W2008 box?
>
Of course not. If it allowed the sensible solution, it wouldn't be politics.

More questions for Lizette:

o How is the UNIX file "receive[d] ... placed" on the Windows server:
- Samba (details)?
- other (specify; details)?
My understanding is that while the Samba protocol is capable in some
cases of performing ASCII<->EBCDIC translation, it will never perform
newline conversion. A plausible motivation is either not changing the
file size or making the conversion bijective.

When the file is transferred (by FTP?) from the Windows server to z/OS,
where does the FTP client run? Where does the FTP server run?

And, for other contributors to this thread:
o Why, nowadays, does anyone afflicted with Windows use Notepad rather
than Wordpad?

And why does the silly-assed Solaris/Firefox spellchecker complain about
"bijective"?

-- gil

Tony Harminc

unread,
Oct 24, 2011, 7:15:59 PM10/24/11
to
On 24 October 2011 17:41, Lizette Koehler <star...@mindspring.com> wrote:

> I have tried several tests.  Either I only get one record up on the mainframe or the data is definately in bin format when it arrives.  I have tried BIN FB,  BIN VB,  ASCII FB and ASCII VB.  So far the 44 records I have to test with are just not behavingin.

As long as your original data doesn't contain binary fields, i.e. is
all text in some reasonably normal variant of ASCII, then you should
do fine with character-mode transfer on both legs. Given the platforms
you have, it's unlikely that using BINary on either leg will lead to
anything good.

As for the RECFM, if you can preallocate on z/OS, do so. Then things
should Just Work (tm), and of course you will have allocated enough
space. If you can't preallocate, and you are FTPing *from* Windows
*to* z/OS, you'll need to specify with QUOTE SITE RECFM=FB, and QUOTE
SITE LRECL=946, and if needed, a suitable BLOCKSIZE. You may also need
to allocate a more appropriate amount of space than the default.

> The odd thing is that NOTEPAD cannot read this file well.  It wraps it (yes tried the WRAP FORMAT in Notepad no luck).  So I can see the records circle around in the notepad window.

This is classic Notepad behaviour when it sees a file that's been
transferred from UNIX in BINary mode, and therefore has only LFs as
line delimiters The WRAP option in Notepad doesn't do anything
different about interpreting the ending sequences; it just wraps the
lines so they fit on the screen.

> However, Notepad++ sees it as individual records.

Notepad+ and almost all editors other than Notepad are just that wee
bit smarter.

Tony H.

Tony Harminc

unread,
Oct 24, 2011, 7:41:07 PM10/24/11
to
On 24 October 2011 18:20, Shmuel Metz (Seymour J.)
<shmuel+...@patriot.net> wrote:

> No; the Unix new line indication is LF (X'0A'),

For some values of UNIX.

> the windoze/DOS new line indication is CRLF (X'0D0A')

Yes.

> and the z/OS new line indication for EBCDIC is NEL (X'25').

No. EBCDIC does not contain a control character called NEL, despite
Wikipedia's claims. The new line indication in z/OS UNIX files is NL,
which is at X'15' in all EBCDIC code pages. ASCII does not contain NL
or NEL, however the "extended ASCII ISO 8859-n code pages contain a
control character called NEL at X'85', which has the same semantics as
NL. UNICODE also has NEL at U+0085.

> The LF and NEL represent new line, which is logically distinct from CRLF.

Logically distinct, how?

CRLF and NL/NEL both make sense as a new line value; LF does not.
Logically, LF should move to the next line but keep the current column
position. LFCR also makes sense, but so few platforms ever used it
that it's not worth considering, imho.

Tony H.

Frank Swarbrick

unread,
Oct 24, 2011, 7:55:39 PM10/24/11
to
This does not work going the other direction.  If it were available it would properly be named SBRECVEOL, or something similar; but the fact is it is not an available option.




>________________________________
>From: "Farley, Peter x23353" <Peter....@BROADRIDGE.COM>
>To: IBM-...@bama.ua.edu
>Sent: Monday, October 24, 2011 4:30 PM
>Subject: Re: CRLF in Unix being translated on Mainframe to x'25'

Frank Swarbrick

unread,
Oct 24, 2011, 7:55:59 PM10/24/11
to
What method is being used to transport the file from the UNIX box to the Windows box?
It should not be a "straight binary" transfer because, as you've noticed, this does not convert from the UNIX end-of-line delimiter to the Windows end-of-line delimiter.

It's a fact that some Windows applications (Wordpad and Notepad++) deal with the UNIX style delimiter as if it had been a Windows style delimiter.  And some (Notepad and the default Windows FTP client) do not.

But in the end the problem is not really with UNIX or Windows specifically, and it certainly is not with z/OS, but is the method of transporting the file between UNIX and Windows.

Frank




>________________________________
>From: Lizette Koehler <star...@MINDSPRING.COM>
>To: IBM-...@bama.ua.edu
>Sent: Monday, October 24, 2011 3:41 PM
>Subject: Re: CRLF in Unix being translated on Mainframe to x'25'
>
>>

glen herrmannsfeldt

unread,
Oct 24, 2011, 8:59:49 PM10/24/11
to
Tony Harminc <to...@harminc.net> wrote:

(snip)
> CRLF and NL/NEL both make sense as a new line value; LF does not.
> Logically, LF should move to the next line but keep the current column
> position. LFCR also makes sense, but so few platforms ever used it
> that it's not worth considering, imho.

Well, LF makes sense in one way, that it allows the use of CR for
line overprinting.

CRLF was convenient in the days of paper tapes and offline
punching and printing, but otherwise it seems strange as a
disk storage format. It isn't hard to convert from disk format
to that requried by a terminal (which may or may not be ASCII).

Note, for example, the IBM 2741 does not use EBCDIC, but its
own code, with its own control characters.

I see no reason for storage formats to follow printer control
formats.

-- glen

Anne & Lynn Wheeler

unread,
Oct 24, 2011, 10:03:12 PM10/24/11
to
glen herrmannsfeldt <g...@ugcs.caltech.edu> writes:
> Note, for example, the IBM 2741 does not use EBCDIC, but its
> own code, with its own control characters.

but all the 2741 characters were defined in EBCDIC. when cp67 was
installed at the univ in jan68 ... it had 2741 and 1052 terminal ... but
the univ. had some number of tty/ascii terminals. I had to add tty/ascii
support ... and there were some number of chars in tty that weren't
ebcdic and vis-a-versa ... which resulted in translation issues. cp67
also had automatic termainl identification for 1052/2741 ... so i added
tty support in a way to preserve automatic terminal identification for
all three terminals types. I then wanted to have single dialup number
with common "hung group" (pool) for all three terminal types. problem
was that ibm terminal controller allowed for changing line-scanner for
each port ... but hardwired line-speed.

this somewhat motivated univ to start clone controller effort, reverse
engineer channel interface and build controller interface card for
interdata/3 (mini-computer) ... programmed to simulate mainframe
terminal controller ... supporting both dynamic terminal type and
dynamic line-speed. later four of us got written for (some part of)
clone controller business (interdata picked up support and marketed it,
continued after interdata was purchased by perkin-elmer). misc.
past post mentioning clone controller business
http://www.garlic.com/~lynn/subtopic.html#360pcm

clone controller business was then major motivation for future
system activity ... misc. past posts
http://www.garlic.com/~lynn/submain.html#futuresys

tale about 360 was originally going to be ascii ... but learson made one
of the biggest mistakes of 360:
http://www.bobbemer.com/P-BIT.HTM

i had 2741 at home from mar70 until 1977 ... when it was replaced with
300baud cdi miniterm ... pictures of old 2741 apl typeball
http://www.garlic.com/~lynn/aplball.jpg
http://www.garlic.com/~lynn/aplball2.jpg

2741 wiki
http://en.wikipedia.org/wiki/IBM_2741

from above ... 2741 code controlled tilt/rotate of the typeball
(selecting the characters on surface of the typeball)

so dynamic terminal type for 2741 ... differentiated 2741 from 1052 (and
i added tty/ascii) ... selecting the corresponding controller
line-scanner. then the software used default 2741 translate table on
initial login ... and assuming first letter "l" ... however, if the
first letter was "y" ... then reversed translated and retranslated with
alternate translate table ... and again checked for "l" (actually
checked for both uppercase and lowercase letters).

--
virtualization experience starting Jan1968, online at home since Mar1970

Gibney, Dave

unread,
Oct 24, 2011, 10:49:04 PM10/24/11
to
Listen to Peter. :)

Ok, if you are using a batch job on z/OS, then you are the client for that transfer. If the file on Windows has the CRLF (that is Notepad works), then a ASCII/EBCIDIC transfer to a FB or VB file with a large enough LRECL should work. You can use the LOCSITE to subcommand to control this.

Which end is the client on the Unix to Windows? You need to make sure the NL to CRLF "translation" occurs. We use MoveITFreely for Windows command files. I think it handles this automatically, but it is other people that set that end up around here.

Dave Gibney
Information Technology Services
Washington State University

> -----Original Message-----
> From: IBM Mainframe Discussion List [mailto:IBM-...@bama.ua.edu] On
> Behalf Of Lizette Koehler
> Sent: Monday, October 24, 2011 2:41 PM
> To: IBM-...@bama.ua.edu
> Subject: Re: CRLF in Unix being translated on Mainframe to x'25'
>
> >

glen herrmannsfeldt

unread,
Oct 25, 2011, 12:08:15 AM10/25/11
to
Anne & Lynn Wheeler <ly...@garlic.com> wrote:

(snip, I wrote)
>> Note, for example, the IBM 2741 does not use EBCDIC, but its
>> own code, with its own control characters.

> but all the 2741 characters were defined in EBCDIC. when cp67 was
> installed at the univ in jan68 ... it had 2741 and 1052 terminal ... but
> the univ. had some number of tty/ascii terminals. I had to add tty/ascii
> support ... and there were some number of chars in tty that weren't
> ebcdic and vis-a-versa ... which resulted in translation issues.

and the problem is still there. Not ASCII NOT sign, needed for
PL/I and some other languages. EBCDIC cent sign is rarely used.

So either ASCII tilde or carat translates to EBCDIC not, with
the ~= looking like an "approximately equal to" operator.

-- glen

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 8:56:38 AM10/25/11
to
Correction:

In
<6598855.131948701...@elwamui-darkeyed.atl.sa.earthlink.net>,
on 10/24/2011
at 04:10 PM, Lizette Koehler <star...@MINDSPRING.COM> said:

>A programmer I am working with is getting a file from Unix which will
>be sent to the mainframe. This will be using FTP.

What are the original and final character sets?

>Is there a way to keep the CRLF command at the end of each line?

Do you mean that the original file has CRLF rather than LF? Also, CRLF
isn't a command just a CR character followed by an LF character. The
easiest way is to use binary mode.

>It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'

No; the Unix new line indication is LF (X'0A'), the windoze/DOS new
line indication is CRLF (X'0D0A') and the z/OS new line indication for
EBCDIC is NL (X'15'). The LF and NL represent new line, which is
logically distinct from CRLF.

BTW, do local politics allow going directly to z/OS instead of
therough the W2008 box?

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

McKown, John

unread,
Oct 25, 2011, 9:55:54 AM10/25/11
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@bama.ua.edu] On Behalf Of Shmuel Metz (Seymour J.)
> Sent: Monday, October 24, 2011 5:20 PM
> To: IBM-...@bama.ua.edu
> Subject: Re: CRLF in Unix being translated on Mainframe to x'25'
>
> In
> <6598855.131948701...@elwamui-darkeyed.atl.sa.e
> arthlink.net>,
> on 10/24/2011
> at 04:10 PM, Lizette Koehler <star...@MINDSPRING.COM> said:
>
> >A programmer I am working with is getting a file from Unix which will
> >be sent to the mainframe. This will be using FTP.
>
> What are the original and final character sets?
>
> >Is there a way to keep the CRLF command at the end of each line?
>
> Do you mean that the original file has CRLF rather than LF? Also, CRLF
> isn't a command just a CR character followed by an LF character. The
> easiest way is to use binary mode.
>
> >It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
> No; the Unix new line indication is LF (X'0A'), the windoze/DOS new
> line indication is CRLF (X'0D0A') and the z/OS new line indication for
> EBCDIC is NEL (X'25'). The LF and NEL represent new line, which is
> logically distinct from CRLF.

I agree, with one correction. The hex code for NEL is x'15', not x'25'. x'25' is LF which is __NOT__ what z/OS UNIX files use. use "od -tx1" on a text file to verify this:

echo 'x'|od -tx1
0000000000 A7 15
0000000002

>
> BTW, do local politics allow going directly to z/OS instead of
> therough the W2008 box?
>
> --
> Shmuel (Seymour J.) Metz, SysProg and JOAT

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM



Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:52:31 AM10/25/11
to
In <1887362737215400.WA...@bama.ua.edu>, on
10/24/2011
at 05:56 PM, Paul Gilmartin <PaulGB...@AIM.COM> said:

>Ummm. Not quite.

I know; I realized the error[1] after I had sent it. EBCDIC New Line
is NL; NEL is a different character. The value X'25' is correct for
neither.

>Ummm. Not quite. z/OS Unix System Services uses EBCDIC NL (x'15')
>as the new line indication.

For EBCDIC files, but I believe that it still uses LF for other
character sets.

>Then it violates the specifications of the code pages by
>translating LF(x'0A')<->NL(x'15')

IBM was caught between a rock and a hard place. None of the available
choices was good.

>And why does the silly-assed Solaris/Firefox spellchecker complain
>about "bijective"?

It's part of the culture for spell checkers to flag legitimate word
and to suggest bizarre replacements for them. In particular, they're
aimed at general users and often don't include terms of art from,
e.g., Law, Mathematics.

Why does the m$ grammar checker complain about legitimate
constructions? Worse, why does m$ word complain about the reading
level when it's already condescendingly low?

[1] And sent a corrected version.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:53:34 AM10/25/11
to
In
<CAArMM9SAYjPHt9WA6fs69OUb...@mail.gmail.com>,
on 10/24/2011
at 07:31 PM, Tony Harminc <to...@HARMINC.NET> said:

>On 24 October 2011 18:20, Shmuel Metz (Seymour J.)
><shmuel+...@patriot.net> wrote:

And sent a corrected[1] version :-(

>For some values of UNIX.

Well, z/OS Unix uses NL and *bsd isn't certified as Unix. Is z/OS the
only oddball, or are there others?

>Logically distinct, how?

CRLF is just X'0D0A' (for ASCII-like character sets). It is not
intrinsically a line ending sequence, and can occur in systems where
neither the CR nor the LF has any specific significance.

It's like the difference between "my cat" and "Maine coon". Even if I
own a Maine coon, not all Maine coons are mine, and even if I own a
second cat, it might not be a Maine coon.

[1] I normally work offline. Finagle decreed that I would be
online between the time that I made the error and the
time that I caught it :-(

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:54:14 AM10/25/11
to
In <1319499745.521...@web122104.mail.ne1.yahoo.com>, on
10/24/2011
at 04:42 PM, Frank Swarbrick <frank.s...@YAHOO.COM> said:

>What method is being used to transport the file from the UNIX box to
>the Windows box? It should not be a "straight binary" transfer
>because, as you've noticed, this does not convert from the UNIX
>end-of-line delimiter to the Windows end-of-line delimiter.

Do the data on the original Unix system have Unix line ends, CRLF line
ends or something else?

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:54:26 AM10/25/11
to
In <m3obx56...@garlic.com>, on 10/24/2011
at 10:00 PM, Anne & Lynn Wheeler <ly...@GARLIC.COM> said:

>but all the 2741 characters were defined in EBCDIC.

For the standard golfball. I don't believe that the APL and NLS EBCDIC
code pages existed in 1968.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:55:11 AM10/25/11
to
In
<13480272.131949247...@elwamui-darkeyed.atl.sa.earthlink.net>,
on 10/24/2011
at 05:41 PM, Lizette Koehler <star...@MINDSPRING.COM> said:

>I have tried several tests.

There are hexadecimal editors available that will treat all 256 octets
as just data, displaying CRLF as 0D0A. If you have one available, take
a look at exactly what is in the original file.

>The odd thing is that NOTEPAD cannot read this file well. It wraps
>it (yes tried the WRAP FORMAT in Notepad no luck). So I can see the
>records circle around in the notepad window.

Are there extraneous blocks of white space? If the original file had
Unix (LF) line ends and you did a binary transfer, I'd expect the LF
to be treated as an LF, meaning got to the next line but don't go back
to column 1.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 10:55:11 AM10/25/11
to
In <1319493748.398...@web65509.mail.ac4.yahoo.com>, on
10/24/2011
at 03:02 PM, Scott Ford <scott_...@YAHOO.COM> said:

>For NOTEPAD++, which I also use, you will need to make sure that the
>file that arrives into Windows has the x'0D0A' , as Shmuel said its
>the CRLF characters.

Technically, CR, CRLF, LF and NL are the line ending sequences for
their respective systems; the equivalent of CRLF is CRLF.

>If the politics would allow it ( i dont miss this ) then a Unix ->
>z/OS would be desirable and a heck of a lot easier.

But whether it's one hop or two, she needs to know the characteristics
of the data in order to get it right. My initial take was to do binary
all the way, but what if the data are, e.g., ISO-8859-15?

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Paul Gilmartin

unread,
Oct 25, 2011, 12:52:43 PM10/25/11
to
On Tue, 25 Oct 2011 09:37:27 -0500, Shmuel Metz (Seymour J.) wrote:

> on 10/24/2011 at 05:56 PM, Paul Gilmartin said:
>
>>Ummm. Not quite. z/OS Unix System Services uses EBCDIC NL (x'15')
>>as the new line indication.
>
>For EBCDIC files, but I believe that it still uses LF for other
>character sets.
>
I thought my use of "EBCDIC" would be sufficient qualification. I wish
there were better support for those "other character sets".

>>Then it violates the specifications of the code pages by
>>translating LF(x'0A')<->NL(x'15')
>
>IBM was caught between a rock and a hard place. None of the available
>choices was good.
>
Were it my choice, I'd define a new code page with LF at 0x15 and NEL
at 0x25 to validate the OEMVS311 translation table. I know, there's
some quasi-theological specious argument that 0x00 through 0x3F are
inviolable. In my estimation, this pales before calling OEMVS311 a
mapping between ISO8859-1 and IBM-1047. I wouldn't much care
if my 3215 didn't work any more. If I had a 3215. And if I needed
it to work, I'd simply translate this putative new character set to
IBM-1047 for the 3215, even as EBCDIC is translated to ASCII for
my terminal session.

-- gil

Kirk Wolf

unread,
Oct 25, 2011, 1:02:53 PM10/25/11
to
On Mon, Oct 24, 2011 at 3:43 PM, McKown, John <John....@healthmarkets.com
> wrote:

>
> On most ASCII UNIX system, the end-of-line character is a single LF
> (LineFeed) or 0x0A.
> On Windows, the normal end-of-line is a 0x0D0A (CRLF or
> CarriageReturn-LineFeed).
> On z/OS UNIX, in EBCDIC, the end-of-line character is the NL (NewLine) or
> 0x15. NOT 0x25. 0x25 is a CarriageReturn.
>
> When you create a file on an ASCII UNIX system, the lines end with a 0x0A.
> If you store this on a "share" where a Windows server can get at it directly
> and you use a Windows ftp client, the file in the z/OS UNIX system is "hosed
> up". Just as you have seen, the lines end with a 0x25 whereas they should
> end with a 0x15. I view this as an error in the Windows ftp client because
> it demands that text line end with CRLF instead of just LF. What you can do
> is have your UNIX process add the "extraneous" CR so that the lines end with
> a CRLF. There is usually a program called "unix2dos" which can do this. If
> you don't have "unix2dos", it can be done with a "sed" command: "sed -i
> 's/$/\r/' file.on.windows". At least, this works with GNU's sed. I don't
> know about others. Some don't expand \r to CR.
>
>
A very good explanation of an all-to-common problem. Many non-M$ ftp
clients don't suffer the same stupidity.

<plug>
If you use Co:Z SFTP on z/OS and in text mode the default is to recognize
any combination of CR, LF, NL as a line terminator. On output to Windows,
you can specify whether you want CRLF or LF. With SSH/SFTP, you can use
any Windows client, such as the free popular PuTTY psftp command. Co:Z
SFTP is free to use but enterprise license and support agreements are also
available.
</plug>

Kirk Wolf
Dovetailed Technologies
http://dovetail.com

Hal Merritt

unread,
Oct 25, 2011, 4:24:41 PM10/25/11
to
First, my short list for unexpected/undesired translations is: Windows. IIRC, the only way to prevent this is to use the binary (image) FTP command -and- suffix the file .BIN.

I do seem to recall a FTP option on some flavors of *nix that causes z/os to run the data all together. Some digging in the archives might be useful if you suspect that.




-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-...@bama.ua.edu] On Behalf Of Lizette Koehler
Sent: Monday, October 24, 2011 3:10 PM
To: IBM-...@bama.ua.edu
Subject: CRLF in Unix being translated on Mainframe to x'25'

A programmer I am working with is getting a file from Unix which will be sent to the mainframe. This will be using FTP.

The process is we receive the file from the Unix system and it is placed on a Windows 2008 server.

Then from the Windows 2008 it is going to be FTP'd to the mainframe.

Is there a way to keep the CRLF command at the end of each line? And will the mainframe recognize it as a newline?

What we really want is the UNIX file to be individual (FB Lrecl 946) going into the mainframe.

It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'


Thanks for any advice

Lizette


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to list...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
NOTICE: This electronic mail message and any files transmitted with it are intended
exclusively for the individual or entity to which it is addressed. The message,
together with any attachment, may contain confidential and/or privileged information.
Any unauthorized review, use, printing, saving, copying, disclosure or distribution
is strictly prohibited. If you have received this message in error, please
immediately advise the sender by reply email and delete all copies.

Steve Comstock

unread,
Oct 25, 2011, 6:28:34 PM10/25/11
to
On 10/24/2011 2:10 PM, Lizette Koehler wrote:
> A programmer I am working with is getting a file from Unix
> which will be sent to the mainframe. This will be using FTP.
>
> The process is we receive the file from the Unix system and
> it is placed on a Windows 2008 server.
>
> Then from the Windows 2008 it is going to be FTP'd to the mainframe.
>
> Is there a way to keep the CRLF command at the end of each line?
> And will the mainframe recognize it as a newline?
>
> What we really want is the UNIX file to be individual (FB Lrecl 946)
> going into the mainframe.
>
> It seems the Unix CRLF is x'0A' whereas the mainframe is x'25'
>
>
> Thanks for any advice
>
> Lizette
>
Our File RePackager product can handle this. Once you
have a file on your mainframe, you can use the utility
to convert the file to an HFS or zFS file with a
different delimiter, code page, even a different
record type.

Regarding your post, the "Unix CRLF" is X'0D0A';
x'0A' is just LF.

It seems ambiguous from your post if the final file is
to be an MVS dataset with lrecl=946 or a z/OS UNIX file
with lrecl=946. Does the 946 include the record delimiter?
If you are going to have each file be a single record,
why even bother with a record delimiter?


So, first, you need to determine the code page of your file,
and what delimiters are being used on input. If it really
is x'0A' and not x'0D0A', then you could use something
like this for utility input commands:

ifile path='in_path_name',filedata=T,cp=923,recdelim=lf
ofile path='out_path_name',filedata=T,cp=37,recdelim=nl

this would convert the file from codepage 923 (ISO-8859-15)
to codepage 37 (USA EBCDIC), including converting the
record delimiter from ASCII LF to EBCDIC NL.

Just a thought.

--

Kind regards,

-Steve Comstock
The Trainer's Friend, Inc.

303-393-8716
http://www.trainersfriend.com

* Special promotion: 15% off on all DB2 training classes
scheduled by September 1, taught by year end 2011

* Check out our entire DB2 curriculum at:
http://www.trainersfriend.com/DB2_and_VSAM_courses/DB2curric.htm

Shmuel Metz , Seymour J.

unread,
Oct 25, 2011, 8:56:03 PM10/25/11
to
In <4433525376678963.WA...@bama.ua.edu>, on
10/25/2011
at 11:37 AM, Paul Gilmartin <PaulGB...@AIM.COM> said:

>Were it my choice, I'd define a new code page with LF at 0x15 and NEL
>at 0x25 to validate the OEMVS311 translation table.

That would cause problems for data where LF really means LF and CR
really means CR. Whether any such are still in use I don't know.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Steve

unread,
Oct 26, 2011, 10:24:21 AM10/26/11
to
it's easy via a unix script - something like this.......(there are other options besides "crnl" which might be useful depending on your particular situation. this worked for us....


cp -F crnl unixfile.xml "//'dataset.xml'"


See the USS Ported Tool Manual......


you can also use IEBGENER if you can deal with a VB file.

//INPUT EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD PATH='/xxx/xxx/xxx.txt',
// PATHOPTS=(ORDONLY),
// RECFM=VB,LRECL=255,BLKSIZE=27998,
// FILEDATA=TEXT
//SYSUT2 DD DSN=DATASET.FILE,
etc....

Steve Lescure

unread,
Oct 26, 2011, 12:35:20 PM10/26/11
to
and i shoud have mentioned...once the file is in a dataset using "cp" you can use a SORT to get it into the FB format you want.

Paul Gilmartin

unread,
Oct 29, 2011, 4:56:28 PM10/29/11
to
On Tue, 25 Oct 2011 20:42:25 -0500, Shmuel Metz (Seymour J.) wrote:
>
>>Were it my choice, I'd define a new code page with LF at 0x15 and NEL
>>at 0x25 to validate the OEMVS311 translation table.
>
>That would cause problems for data where LF really means LF and CR
>really means CR. Whether any such are still in use I don't know.
>
What's "really" LF? 0x0A? 0x25? 0x15? And in what data do each of
these cause problems or not cause problems? Device drivers should
be aware of the code page of the data, and take suitable action on
embedded control codes.

I did some experimenting. On Ubuntu Linux:

501 $ uname -a
Linux Linux-Mac 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:50:42 UTC 2011 i686 i686 i386 GNU/Linux
502 $
502 $ printf '\x0A' | iconv -f iso8859-1 -t ibm-1047 | od -x
0000000 0025

Which is obviously correct according to the definitions of the
respective code pages.


On z/OS:

user@MVS.:128$ uname -a
OS/390 MVS3 22.00 03 2066
user@3MVS.:129$
user@3MVS.:129$ printf '\x0A' | iconv -f iso8859-1 -t ibm-104>
0000000000 1500

This is equally obviously incorrect. This causes problems, reported
in these fora, when programmers transfer text files in binary and
rely on iconv(1) to perform the conversion. They get different
results according to whether the conversion is performed on the
Linux platform or on z/OS.

I grant that the existing behavior on z/OS must be preserved for
compatibility with existing code. But there's still a way out:

Define a new code page (perhaps IBM-1047-X) with NEL at 0x25 and
LF at 0x15; otherwise identical to IBM-1047. Replace mention of
IBM-1047 with IBM-1047-X in documentation where appropriate (i.e.
nearly everywhere). Register IBM-1047-X with ICANN (or suitable
standards organization). Deprecate use of "IBM-1047" for reasons
of ambiguity.

Honesty is the best policy,
gil

Shmuel Metz , Seymour J.

unread,
Oct 30, 2011, 3:57:47 PM10/30/11
to
In <2944060858263639.WA...@bama.ua.edu>, on
10/29/2011
at 03:51 PM, Paul Gilmartin <PaulGB...@AIM.COM> said:

>What's "really" LF?

Move to the same column of the next line, of course. What else could
it be?

>0x0A?

That makes no sense in the context of "data where LF really means LF
and CR really means CR."

>But there's still a way out:

No; you're replacing one set of problems with another. At best it's
the least of the evils.

>Define a new code page (perhaps IBM-1047-X) with NEL at 0x25 and LF
>at 0x15; otherwise identical to IBM-1047.

With what translation from ISO-8859-1?

>Honesty is the best policy,

Define "honesty" in the context of current and proposed code-page
translations.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
ISO position; see <http://patriot.net/~shmuel/resume/brief.html>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

John McKown

unread,
Oct 30, 2011, 7:52:28 PM10/30/11
to
On Sat, 2011-10-29 at 19:28 -0500, Shmuel Metz (Seymour J.) wrote:
> In <2944060858263639.WA...@bama.ua.edu>, on
> 10/29/2011
> at 03:51 PM, Paul Gilmartin <PaulGB...@AIM.COM> said:
>
> >What's "really" LF?
>
> Move to the same column of the next line, of course. What else could
> it be?

Depends on the printer. 0x0A on many DecWriters did both a CR and an LF
function. That's why UNIX defaulted that way, from what I was told. No
need to do any character translation or additions if you just did a "cp"
to the device. Of course, Windows via MS-DOS via CP/M-80 used CRLF for
the same reason. The PC printers of the day required a separate LF and
CR to go to the beginning of the next line. And the CR was done first so
that the mechanical time to return the head was taken up by rolling the
platten to the next line due to the fact that the CR functino took a
"significant" amount of time compared to the LF or printing a simple
character. Again, as I was told.

>

--
John McKown
Maranatha! <><

Anne & Lynn Wheeler

unread,
Oct 30, 2011, 8:22:33 PM10/30/11
to
joa...@SWBELL.NET (John McKown) writes:
> Depends on the printer. 0x0A on many DecWriters did both a CR and an LF
> function. That's why UNIX defaulted that way, from what I was told. No
> need to do any character translation or additions if you just did a "cp"
> to the device. Of course, Windows via MS-DOS via CP/M-80 used CRLF for
> the same reason. The PC printers of the day required a separate LF and
> CR to go to the beginning of the next line. And the CR was done first so
> that the mechanical time to return the head was taken up by rolling the
> platten to the next line due to the fact that the CR functino took a
> "significant" amount of time compared to the LF or printing a simple
> character. Again, as I was told.

re:
http://www.garlic.com/~lynn/2011n.html#45 CRLF in Unix being translated on Mainframe to x'25'

another recent post about adding tty/ascii terminal support to cp67
(already had 2741 & 1052 support)
http://www.garlic.com/~lynn/2011n.html#70 1979 SHARE LSRAD Report

one of the things done in terminal support was line was "padded" with
idle characters after a "CR" ... formula that calculated how many
characters had been printed in the line, how fast the carriage/typehead
returned and how fast characters were transmitted ... in order to allow
carriage/typehead to have returned before start printing the next line.

for other trivia ... this is old item about the name cp/m being derived
from (ibm mainframe virtual machine) cp/67 ... kildall (author of cp/m)
having used cp/67 at navy post graduate school in 1972 ... gone 404,
but lives on at wayback machine
http://web.archive.org/web/20071011100440/http://www.khet.net/gmc/docs/museum/en_cpmName.html

--
virtualization experience starting Jan1968, online at home since Mar1970

glen herrmannsfeldt

unread,
Oct 30, 2011, 11:54:42 PM10/30/11
to
John McKown <joa...@swbell.net> wrote:
> On Sat, 2011-10-29 at 19:28 -0500, Shmuel Metz (Seymour J.) wrote:
>> In <2944060858263639.WA...@bama.ua.edu>, on
>> 10/29/2011
>> at 03:51 PM, Paul Gilmartin <PaulGB...@AIM.COM> said:

>> >What's "really" LF?

>> Move to the same column of the next line, of course.
>> What else could it be?

> Depends on the printer. 0x0A on many DecWriters did both a
> CR and an LF function.

I believe the 2741 also combines the functions. It might be
that CR always causes LF, but not the other way.

> That's why UNIX defaulted that way, from what I was told. No
> need to do any character translation or additions if you just
> did a "cp" to the device. Of course, Windows via MS-DOS via
> CP/M-80 used CRLF for the same reason. The PC printers of the
> day required a separate LF and CR to go to the beginning of
> the next line.

I always thought it started with DEC. Also, it is needed if
you print from punched paper tape directly, without a computer.

> And the CR was done first so that the mechanical time to return
> the head was taken up by rolling the platten to the next line
> due to the fact that the CR functino took a "significant" amount
> of time compared to the LF or printing a simple character.

If it still isn't enough, you have to add NULs. For the ASR33,
with no buffering, you can print while the head is returning,
and get flying characters. The DECwriters that I remember,
I believe LA36, buffer enough for the return and then print
faster until they catch up.

DECwriters designed as line printers (not terminals) buffer
the whole line before printing.

> Again, as I was told.

-- glen

0 new messages