IPM clearing file : DE48 PDS issue

127 views
Skip to first unread message

Abhinav Kumar

unread,
May 18, 2023, 4:19:16 PM5/18/23
to jPOS Users

I have been working on developing a package creator for generating an IPM clearing file that includes individual messages/transactions. One of the fields in this file is DE48, with a maximum length of 999 characters. DE48 contains various PDS (including cavV and transaction ID) fields. I have successfully created a packaged message with the PDS inside DE48, and everything functions properly as long as the length of DE48 does not exceed 170 positions (approximately 85 bytes). Strangely, no errors are thrown during the packaging process. However, when I attempt to parse it using the IPM tool, an error occurs stating "DE 2: Invalid number." If I reduce the length by removing some fields, the parsing works correctly.

I am unsure where the issue lies, but I have included a snapshot of my packager configuration and the TLV code. I would greatly appreciate any assistance you can provide.


PACKAGER XML :


<isofieldpackager
            id="48"
            name="Additional Data"
            length="999"
            class="org.jpos.iso.IFE_LLLCHAR"
            token="0000"
            emitBitmap="false"
            packager="org.jpos.tlv.packager.TaggedSequencePackager">
        <isofield
                id="1"
                length="99"
                token="####"
                name="Default Tag Packager"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="2"
                length="3"
                token="0023"
                name="Currecy code and exponents"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="3"
                length="7"
                token="0026"
                name="File reversal indicator"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="4"
                length="25"
                token="0105"
                name="File ID"
                class="org.jpos.tlv.packager.IFTA_LLLNUM"/>
        <isofield
                id="5"
                length="25"
                token="0110"
                name="Transmission ID"
                class="org.jpos.tlv.packager.IFTA_LLLNUM"/>
        <isofield
                id="6"
                length="1"
                token="0122"
                name="Processing Mode"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="7"
                length="60"
                token="0148"
                name="Currecy code and exponents"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="8"
                length="30"
                token="0165"
                name="Currecy code and exponents"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="9"
                token="0184"
                name="Trn Ind"
                length="199"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="10"
                token="0185"
                name="Recurring Trn Ind"
                length="199"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="11"
                token="0186"
                name="Recurring Trn Ind"
                length="1"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="12"
                length="1"
                token="0191"
                name="Orignating message format"
                class="org.jpos.tlv.packager.IFTA_LLLCHAR"/>
        <isofield
                id="13"
                length="16"
                token="0301"
                name="File Amount, Checksum"
                class="org.jpos.tlv.packager.IFTA_LLLNUM"/>
        <isofield
                id="14"
                length="8"
                token="0306"
                name="File Message Counts"
                class="org.jpos.tlv.packager.IFTA_LLLNUM"/>
    </isofieldpackager>


SAMPLE CODE :

ISOMsg field48 = new ISOMsg(48);
tagValueSequence.add(new LiteralTagValue("0023", "CT6"));
tagValueSequence.add(new LiteralTagValue("0148", "7040"));
tagValueSequence.add(new LiteralTagValue("0165", "M"));
tagValueSequence.add(new LiteralTagValue("0158", ISOUtil.padleft(this.transactionIrd,12,' ')));
// tagValueSequence.add(new LiteralTagValue("0184", cbsTxnId));
 tagValueSequence.add(new LiteralTagValue("0185", "kL"));
tagValueSequence.writeTo(field48);
isoMsg.set(field48);

Mark Salter

unread,
May 18, 2023, 4:34:49 PM5/18/23
to jpos-...@googlegroups.com
What does :-

tagValueSequence.writeTo(field48);

place in the variable field48?

Seems to me you are not using the package, but handling field 48 yourself and have bit shared that code - which j guess is not forming the result correctly to match the ipm parse.

Does IPM indicate what it sees in field 1 (perhaps longer or shorter than you expect, so that it looks for field 2 in the wrong place?

-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
--
--
jPOS is licensed under AGPL - free for community usage for your open-source project. Licenses are also available for commercial usage. Please support jPOS, contact: sa...@jpos.org
---
You received this message because you are subscribed to the Google Groups "jPOS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jpos-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/d982f8bf-c14c-4347-bd17-70a6b1210734n%40googlegroups.com.
signature.asc

Mark Salter

unread,
May 18, 2023, 4:37:47 PM5/18/23
to jpos-...@googlegroups.com
Sorry for my terrible typing on my mobile. I think I will stop replying on it!



-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/y4qdsuDQQGYXcxwoFl51kXAt_mvAEMvF9gYRpQ2IUb_l_nrqpV0gKYegvESgBITAYHvJJOrAPCDKoSkNUxZp5abdahyIZJCug3SVTS3Hlpo%3D%40pm.me.
signature.asc

murtuza chhil

unread,
May 18, 2023, 11:28:35 PM5/18/23
to jPOS Users
Code seems to all right when compared to the test case https://github.com/jpos/jPOS/blob/master/jpos/src/test/java/org/jpos/tlv/packager/ISOTaggedSequenceTest.java

When we ingest IPM files we need to worry about the @@ delimiters at every 1012 bytes. in the file.
I don't know the file format you are working with and perhaps you need to put those delimiters in for the receiver to handle it correctly.
You may want to check if you have set the length of the message before the message is correct. [ length][Message]

I just did a sanity test to see the data in ascii instead of ebcdic and it looked all right

public static void testPacking() throws ISOException, FileNotFoundException { ISOMsg msg = new ISOMsg("0100"); GenericTagSequence tagValueSequence = new GenericTagSequence(); tagValueSequence.add(new LiteralTagValue("0023", "CT6")); tagValueSequence.add(new LiteralTagValue("0148", "7040")); tagValueSequence.add(new LiteralTagValue("0165", "M")); tagValueSequence.add(new LiteralTagValue("0158", ISOUtil.padleft("123456",12,' '))); tagValueSequence.add(new LiteralTagValue("0184", ISOUtil.padleft("", 199, '1'))); tagValueSequence.add(new LiteralTagValue("0185", "kL")); ISOMsg field48 = new ISOMsg(48); tagValueSequence.writeTo(field48); msg.set(field48); msg.recalcBitMap(); GenericPackager packager = new GenericPackager(new FileInputStream("location to your packager.xml")); // I just added ISPPackager element with isofields 0 and 1 to your packager msg.setPackager(packager); byte[] packed = packager.pack(msg); System.out.println(ISOUtil.hexdump(packed)); msg.dump(System.out, ""); // from the hexdump excluded the MTI and bitmap and just used data for DE48 String s = "F2F6F3F0" + "F0F2F3F0F0F3C3E3F6F0F1F4F8F0F0F4" + "F7F0F4F0F0F1F6F5F0F0F1D4F0F1F5F8" + "F0F1F2404040404040F1F2F3F4F5F6F0" + "F1F8F4F1F9F9F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1F1" + "F1F1F1F1F1F1F1F1F1F1F1F1F1F0F1F8" + "F5F0F0F292D3"; System.out.println(ISOUtil.ebcdicToAscii(ISOUtil.hex2byte(s))); // LLL is 263 and the TLVs looked correct. }

Output

0000 30 31 30 30 00 00 00 00 00 01 00 00 F2 F6 F3 F0 0100............ 0010 F0 F2 F3 F0 F0 F3 C3 E3 F6 F0 F1 F4 F8 F0 F0 F4 ................ 0020 F7 F0 F4 F0 F0 F1 F6 F5 F0 F0 F1 D4 F0 F1 F5 F8 ................ 0030 F0 F1 F2 40 40 40 40 40 40 F1 F2 F3 F4 F5 F6 F0 ...@@@@@@....... 0040 F1 F8 F4 F1 F9 F9 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0050 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0060 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0070 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0080 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0090 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00a0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00b0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00c0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00d0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00e0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 00f0 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 ................ 0100 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F1 F0 F1 F8 ................ 0110 F5 F0 F0 F2 92 D3 ...... <isomsg> <!-- org.jpos.iso.packager.GenericPackager --> <field id="0" value="0100"/> <isomsg id="48"> <0023><field id="1" value="CT6"/></0023> <0148><field id="2" value="7040"/></0148> <0165><field id="3" value="M"/></0165> <0158><field id="4" value=" 123456"/></0158> <0184><field id="5" value="1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111"/></0184> <0185><field id="6" value="kL"/></0185> </isomsg> </isomsg> 2630023003CT6014800470400165001M0158012 123456018419911111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111110185002kL
​

263 length and then tag (4 wide ) length (3 wide) values

-chhil


Message has been deleted

Mark Salter

unread,
May 19, 2023, 2:48:42 AM5/19/23
to jpos-...@googlegroups.com
I wanted to see what your code is producing for field 48 - i suspect you have an error is the code I cannot see.

It needs to not produce any length for field 48, since if it does the packager will add another length and this is likely the issue.

-- 
Mark
signature.asc

Abhinav Kumar

unread,
May 19, 2023, 2:49:16 AM5/19/23
to jPOS Users
tagValueSequence.writeTo(field48);
temporary ISOfield contains the all the de48 value(literal TLV) and then it writes to actual ISO message using 
isoMsg.set(field48); 

place in the variable field48?
yes
 
Seems to me you are not using the package, but handling field 48 yourself and have bit shared that code - which j guess is not forming the result correctly to match the ipm parse.

Does IPM indicate what it sees in field 1 (perhaps longer or shorter than you expect, so that it looks for field 2 in the wrong place?
Unfortunately, I don't have access to MC proprietary  tool. But some there is a link that can help to parse the file (https://paymentcardtools.com/cli-utils/ipm2json). This is also giving the same error. 

Additionally, specs states that de48 will have data representation as an...999; LLLVAR and would contain PDS (formatted using a tag-length-data encoding) but the moment data goes above 87 characters IPM parse breaks. It is possible that the open source tool might be having a different length validation(87 looks a bit odd to me) but so far it has worked for all the other fields. I have submitted the file again to be validated from the MC tool, will wait for the results (fingers crossed) 


Thanks chill for validation but I am on the side where in I generate the file but it fails when I use the tools mentioned above, is it possible that @@ might be causing issue in DE48 ? 

Mark Salter

unread,
May 19, 2023, 2:52:50 AM5/19/23
to jpos-...@googlegroups.com
To that last point about the blocking, you do need to relocate and add the x0000 every block if you are sending to Mastercard.

Are you receiving an IPM file at all too? The blocking will need handling there too.

--
Mark
signature.asc

chhil

unread,
May 19, 2023, 2:57:04 AM5/19/23
to jpos-...@googlegroups.com
The file I process has @@ every 1021 bytes in the file. I strip out the @@ every 1021 and then process the file. So if you are generating the file do you need to place the @@ ?

There are different file formats and it may not be related at all. 

-chhil


You received this message because you are subscribed to a topic in the Google Groups "jPOS Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jpos-users/3oVNn4XAACw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jpos-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/35cb59b5-536c-4560-8635-dd5c3aa1ef7dn%40googlegroups.com.

Abhinav Kumar

unread,
May 19, 2023, 3:02:03 AM5/19/23
to jPOS Users
@Mark,

Below is the code snapshot with relevant O/P, I removed all the other fields for simplicity 

CODE:
InputStream is = new FileInputStream(fileFolder);
GenericPackager packager = new GenericPackager(is);
isoMsg.setPackager(packager);
isoMsg.setMTI(mti);

GenericTagSequence tagValueSequence = new GenericTagSequence();
ISOMsg field48 = new ISOMsg(48);
tagValueSequence.add(new LiteralTagValue("0023", pds0023));
tagValueSequence.add(new LiteralTagValue("0052", pds0052));
tagValueSequence.add(new LiteralTagValue("0148", pds0148));
tagValueSequence.add(new LiteralTagValue("0165", pds0165));
tagValueSequence.add(new LiteralTagValue("0158", ISOUtil.padleft(pds0158,12,' ')));
tagValueSequence.add(new LiteralTagValue("0184", "111211"));

tagValueSequence.add(new LiteralTagValue("0185", "kL"));
tagValueSequence.add(new LiteralTagValue("0186", pds00186));
tagValueSequence.writeTo(field48);
isoMsg.set(field48);
isoMsg.recalcBitMap();

ISOUtil.hexdump(isoMsg.pack())
0000  F1 F2 F4 F0 00 00 00 00  00 01 00 00 F0 F8 F8 F0  ................
0010  F0 F2 F3 F0 F0 F3 C3 E3  F6 F0 F0 F5 F2 F0 F0 F3  ................
0020  F2 F1 F2 F0 F1 F4 F8 F0  F0 F4 F7 F0 F4 F0 F0 F1  ................
0030  F6 F5 F0 F0 F1 D4 F0 F1  F5 F8 F0 F1 F2 40 40 40  .............@@@
0040  40 40 40 40 40 40 40 F5  F7 F0 F1 F8 F4 F0 F0 F6  @@@@@@@.........
0050  F1 F1 F1 F2 F1 F1 F0 F1  F8 F5 F0 F0 F2 92 D3 F0  ................
0060  F1 F8 F6 F0 F0 F1 F2                              .......


ISOUtil.ebcdicToAscii((isoMsg.pack()))
1240 0880023003CT60052003212014800470400165001M0158012          5701840061112110185002kL01860012


"To that last point about the blocking, you do need to relocate and add the x0000 every block if you are sending to Mastercard.

Are you receiving an IPM file at all too? The blocking will need handling there too."

Yes, x0000  is added every block and the same works fine if I keep the length to 87 or less for DE48 (above example fails). Maybe I am missing out on something which I am not able to see. I will cross check the piece adding the x0000 block.

Mark Salter

unread,
May 19, 2023, 3:21:22 AM5/19/23
to jpos-...@googlegroups.com
My recollection is 1012 byte blocking, just checking in cas you have a typo Chhil.



-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/CAPazefAVoMS%3DPkQqbVS3%3Dn7TVGu6EcGmFDJ3qY5pz%2BT%2BAeOZFQ%40mail.gmail.com.
signature.asc

Mark Salter

unread,
May 19, 2023, 3:25:27 AM5/19/23
to jpos-...@googlegroups.com
Can you share just your field48 content please - as it goes into the iso.sg.set(48) ?

>> ISOUtil.ebcdicToAscii((isoMsg.pack()))
After this point all binary data like lengths could be broken. You should not do this, but should be building ascii directly if ascii is needed (mastercard can handle both).



-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
--
--
jPOS is licensed under AGPL - free for community usage for your open-source project. Licenses are also available for commercial usage. Please support jPOS, contact: sa...@jpos.org
---
You received this message because you are subscribed to the Google Groups "jPOS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jpos-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/50d98b77-3c53-41b1-a9b0-1eec43777567n%40googlegroups.com.
signature.asc

Abhinav Kumar

unread,
May 19, 2023, 3:38:27 AM5/19/23
to jPOS Users
I have appended the values as the end of each line for each fields as it is going into the TLV 

tagValueSequence.add(new LiteralTagValue("0023", pds0023)); --CT6
tagValueSequence.add(new LiteralTagValue("0052", pds0052)); --212
tagValueSequence.add(new LiteralTagValue("0148", pds0148)); --7040
tagValueSequence.add(new LiteralTagValue("0165", pds0165)); --M
tagValueSequence.add(new LiteralTagValue("0158", ISOUtil.padleft(pds0158,12,' '))); --57
tagValueSequence.add(new LiteralTagValue("0184", "111211")); --static value
tagValueSequence.add(new LiteralTagValue("0185", "kL")); --static value
tagValueSequence.add(new LiteralTagValue("0186", pds00186)); --2

>>ISOUtil.ebcdicToAscii((isoMsg.pack())) 
After this point all binary data like lengths could be broken. You should not do this, but should be building ascii directly if ascii is needed (mastercard can handle both).

Agreed, this was done just to print it in human readable form, it is not being used for IPM generation. 

chhil

unread,
May 19, 2023, 3:46:31 AM5/19/23
to jpos-...@googlegroups.com
It was a typo, 1012 bytes is correct.

Abhinav Kumar

unread,
May 19, 2023, 5:41:09 AM5/19/23
to jPOS Users
Thanks @Mark, @Chhil, 

I was able to figure out the issue and fix it. 

Each IPM message is prefixed with a 4 byte message length (RDW) and somehow the length was leaking into  5 bytes which was resulting in an error. Used ByteBuffer wrapper and it works like a charm now.

Thanks again! 

chhil

unread,
May 19, 2023, 6:46:33 AM5/19/23
to jpos-...@googlegroups.com
Good to know it works. 

One of the things I mentioned in my first response. 

"You may want to check if you have set the length of the message before the message is correct. [ length][Message]"

-chhil

To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/bcca3bce-1816-41aa-b9e8-e53348cab5d0n%40googlegroups.com.

Mark Salter

unread,
May 19, 2023, 11:10:08 AM5/19/23
to jpos-...@googlegroups.com
I can't ask again, you are obviously unable to share the details requested.

Good luck :-)



-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/06aeb747-35cd-491e-b697-7aba64d9a66fn%40googlegroups.com.
signature.asc

Mark Salter

unread,
May 19, 2023, 11:11:15 AM5/19/23
to jpos-...@googlegroups.com



-- 
Mark


Sent from Proton Mail mobile



-------- Original Message --------
On 19 May 2023, 10:41, Abhinav Kumar < crusky...@gmail.com> wrote:
signature.asc

Abhinav Kumar

unread,
May 19, 2023, 2:00:05 PM5/19/23
to jPOS Users
Hi @mark, 

Already shared the values in the thread but the issue was with RDW (4 byte length) 

@Chhil, 

Somehow, I missed  the part where in you had asked to check the length. 

Anyways, better late than never ! :) 


Thanks, cheers! 

Collins Adom Baffour

unread,
Feb 19, 2026, 9:49:14 PM (4 days ago) Feb 19
to jPOS Users
You can also check cardutil.java, which can be used for working with Mastercard IPM files. It is the java implementation of  Cardutil by Anthony
Reply all
Reply to author
Forward
0 new messages