EBCDIC special characters not matching

[Prabhanshu Choubey]

Hi All,

The earlier logic which Mark and Chh proposed finally worked. Thanks a lot!!

Just one issue, there are some fields with Data as "Bank of America's" in EBCDIC for which the special character(') is not getting resolved and prints some junk character.

Even for testing when I am printing this statement:

record=new String(byte,Charset.forName("Cp1047"));

It is not resolving the special characters and printing some junk character.

How can we handle this, if you can propose a good solution/alternative.

Thanks,

Victor

[Barzilai Spinak]

Is the problem during the parsing or the generation of  the EBCDIC data.
Could you share a hex dump of the message, and the acompanying packager config so we can debug it?

On 21/3/22 10:33, Prabhanshu Choubey wrote:
> Hi All,
>
> The earlier logic which Mark and Chh proposed finally worked. Thanks a lot!!
>

> Just one issue, there are some fields with Data as "*Bank of America's*" in EBCDIC for which the special character(') is not getting resolved and

> prints some junk character.
>
> Even for testing when I am printing this statement:
>
> record=new String(byte,Charset.forName("Cp1047"));
>
> It is not resolving the special characters and printing some junk character.
> How can we handle this, if you can propose a good solution/alternative.
>
> Thanks,
> Victor
>
>

> --
> --
> jPOS is licensed under AGPL - free for community usage for your open-source project. Licenses are also available for commercial usage. Please
> support jPOS, contact: sa...@jpos.org
> ---
> You received this message because you are subscribed to the Google Groups "jPOS Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jpos-users+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jpos-users/CAFpRj01outkarSmBDh_CWOdsbiqW9GXi%3DK3_AWYA%2BFe5gc%3DKPA%40mail.gmail.com
> <https://groups.google.com/d/msgid/jpos-users/CAFpRj01outkarSmBDh_CWOdsbiqW9GXi%3DK3_AWYA%2BFe5gc%3DKPA%40mail.gmail.com?utm_medium=email&utm_source=footer>.


--
Barzilai Spinak - jPOS Evangelist | *Transactility, Inc.* | Montevideo, Uruguay | *e*: bar...@transactility.com

[Mark Salter]

And also, please do share if this file has undergone any transmission or conversion before you are processing it too please.

--
Mark


Sent from ProtonMail mobile



-------- Original Message --------

To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/01d16e0f-3053-8df1-ea48-bd67ae8b99ea%40transactility.com.

[murtuza chhil]

I have used and mentioned  IBM1047 (and so does JPOS  in its ISOUtil) and you are using Cp1047. Maybe there are differences in the encoding/decoding.

I have a vague memory of comparing the previous way jpos ISOUtil used to do it with an array of mapped bytes and using IBM1047, all the byte translations did not match up maybe thats the problem.
The commit I refer to is...

https://github.com/jpos/jPOS/commit/6e2d3d38394d3dfe0f1c4496dea30b3cdb9936d4

Having said this, I have used IBM1047 and have not run into problems.

-chhil

[murtuza chhil]

Did a small test to see if my vague recollection was valid, no it wasn’t, the bytes match.
Used the old way byte array used for mapping and the new way to encode decode and did not find any differences.

The apostrophe that the OP talks about being a problem does not seem to be a problem based on my ebcdic to ascii and ascii to ebcdic test for ascii 0x27 (apostrophe)

public static void main(String[] args) {
        findDiffOldAsciiToEbcdicAndNewAsciiToEbcdic();
        findDiffOldEbcdicToAsciiAndNewEbcdicToAscii();

        byte[] boaEbcdicOldWay = ISOUtil.asciiToEbcdic("Bank of America's".getBytes(Charset.forName("ISO8859_1")));// old
                                                                                                                   // way
        byte[] boaAscii = ISOUtil.ebcdicToAsciiBytes(boaEbcdicOldWay);// old way
        System.out.println(ISOUtil.hexdump(boaAscii));
        System.out.println(ISOUtil.hexdump(boaEbcdicOldWay));
        Charset EBCDIC = Charset.forName("CP1047");
        ByteBuffer ebcNewWay = EBCDIC.encode("Bank of America's");
        System.out.println(ISOUtil.hexdump(ebcNewWay.array()));
        if (Arrays.equals(ebcNewWay.array(), boaEbcdicOldWay)) {
            System.out.println("Old = new");
        }

    }

    private static void findDiffOldEbcdicToAsciiAndNewEbcdicToAscii() {
        Charset EBCDIC = Charset.forName("CP1047");
        byte[] bArr = new byte[1];
        for (int i = 0; i < 256; i++) {
            bArr[0] = (byte) i;

            String ascOldWay = ISOUtil.ebcdicToAscii(bArr);// uses the old array mapping
            String decoded = EBCDIC.decode(ByteBuffer.wrap(bArr))
                                   .toString();// uses the IBM1047

            byte[] ascNewWay = decoded.getBytes();
            if (ascNewWay[0] != ascOldWay.getBytes()[0]) {

                System.out.println(ISOUtil.hexdump(ascOldWay.getBytes()));
                System.out.println(ISOUtil.hexdump(ascNewWay));

            }
        }

    }

    private static void findDiffOldAsciiToEbcdicAndNewAsciiToEbcdic() {
        Charset EBCDIC = Charset.forName("CP1047");
        byte[] bArr = new byte[1];
        for (int i = 0; i < 256; i++) {
            bArr[0] = (byte) i;
            byte[] ebcOldWay = ISOUtil.asciiToEbcdic(bArr);
            byte[] ebcNewWay = EBCDIC.encode(new String(bArr, Charset.forName("ISO8859_1")))
                                     .array();
            if (ebcOldWay[0] != ebcNewWay[0]) {
                System.out.println(i);
                System.out.println(ISOUtil.hexdump(ebcOldWay));
                System.out.println(ISOUtil.hexdump(ebcNewWay));

            }

        }
    }

-chhil

​