new line in EBCDIC

[Ze'ev Atlas]

Hi Phil

I began to look at the concept of newline in EBCDIC.  We do x'15' as the NL = NEL as codepage 1047 states:

PCRE2 version 10.42 2022-12-11                         

Compiled with                                          

  EBCDIC code support: LF is 0x15                      

  EBCDIC code page IBM1047 or similar                  

  8-bit support                                        

  No Unicode support                                   

  No just-in-time compiler support                     

  Default newline sequence is LF                       

  \R matches all Unicode newlines                      

  \C is supported                                      

  Internal link size = 2                               

  Parentheses nest limit = 250                         

  Default heap limit = 20000000 kibibytes              

  Default match limit = 10000000                       

  Default depth limit = 10000000                       

  pcre2test has neither libreadline nor libedit support

the test does recognize x'15' as $ bit not as \n... any idea?  the text contains x'15' immediately after the string 'newlines'

/(.*)$/            

    test newlines  

 0: test newlines  

 1: test newlines  

inside             

 0: inside         

 1: inside         

                   

/(.*)\n/           

    test newlines  

No match           

inside             

No match           

Thank you

Ze'ev Atlas

[Ze'ev Atlas]

attaching my input and output files.

Notice that when I replaced x'15' with \n, the results where somewhat better.  Also note that the files are converted to ASCII, so you have to trust me about the X'15'

Ze'ev Atlas

--
You received this message because you are subscribed to the Google Groups "PCRE2 discussion list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pcre2-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pcre2-dev/13990021.2239362.1672271907003%40mail.yahoo.com.

[Philip Hazel]

It should work, of course. Looking at the code, I can't immediately see anything wrong, but of course I cannot test in an EBCDIC environment. One thing you could do is to run the test with the -d option. On Linux, I get this:

/(.*)\n/
------------------------------------------------------------------
  0  15 Bra
  3   7 CBra 1
  8     Any*
 10   7 Ket
 13     \x0a
 15  15 Ket
 18     End
------------------------------------------------------------------

Note the explicit \x0a (ASCII newline) that is shown. In an EBCDIC world that should show as \x15. If it does not, we will have to find out why not. 

Regards,

Philip

[Ze'ev Atlas]

I tried the -d, and I get the \x15, yet it does not recognize it correctly, but does when I have 'n instead, see attached.

So far I could not find why

Thank you

Ze'ev Atlas

--
You received this message because you are subscribed to the Google Groups "PCRE2 discussion list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pcre2-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pcre2-dev/CADz7ckZFpksgVBnryAxO%3D8S88EZ5u%2BbX5OuPwHepPPKwum9Ptg%40mail.gmail.com.

[Ze'ev Atlas]

I have to take my complaint back!

I ran the same thing on Windows (using pcre2test version PCRE 10.39) and I got the exact same results, except where the EBCDIC vs. ASCII came to show slightly different results.  My main issue is that the z/OS environment does NOT have newline character on regular basis, but when I force it, I get the same results.  To those who are interested, z/OS native files are separated by length.  Normally, text files consists of fixed length rows (usually 80 bytes long,) or variable length rows with header that define the record length (and thus limit it to maximum of 32K.)

I will explain it in my documentation and perhaps create some functionality to add newline character for those who must have it :)

Ze'ev Atlas

To view this discussion on the web visit https://groups.google.com/d/msgid/pcre2-dev/648736784.3024453.1672341456015%40mail.yahoo.com.

[Philip Hazel]

I'm not sure I entirely understand what you are saying about the Windows version. On Linux it behaves exactly as I would expect:

PCRE2 version 10.42 2022-12-11
/(.*)\n/
abcd
No match
abcd\n
 0: abcd\x0a
 1: abcd

Regards,

Philip

[Ze'ev Atlas]

The Windows version behaves exactly as the Linux version.   I get exactly the same results in Windows and z/OS except of the EBCDIC  vs ASCII.  I WILL TRY TO EXPLAIN THIS PARTICULAR DIFFERENCE LATER.

Ze'ev

Sent from Yahoo Mail on Android