Re: [jpos-commits] SF.net SVN: jpos:[2828] trunk/jpos6/modules/jpos

[David Bergert]

Is there any difference between using the EOF vs EOM ? We are using EOF to do this today.

David Bergert, CISSP, CISA, CPISM/A
www.paymentsystemsblog.com



On Dec 1, 2009, at 8:46 AM, ni...@users.sourceforge.net wrote:

> Revision: 2828
> http://jpos.svn.sourceforge.net/jpos/?rev=2828&view=rev
> Author: ninki
> Date: 2009-12-01 14:46:49 +0000 (Tue, 01 Dec 2009)
>
> Log Message:
> -----------
> Added an EOM separator to FSDMsg fields. When unpacking, this takes the rest of the message. Useful for message formats that are too complicated to be handled by FSDMsg.
>
> Modified Paths:
> --------------
> trunk/jpos6/modules/jpos/src/org/jpos/util/FSDMsg.java
>
> Added Paths:
> -----------
> trunk/jpos6/modules/jpos/test/org/jpos/util/FSDMsgEndOfMessageTestCase.java
> trunk/jpos6/modules/jpos/test/org/jpos/util/eom-base.xml
>
> Modified: trunk/jpos6/modules/jpos/src/org/jpos/util/FSDMsg.java
> ===================================================================
> --- trunk/jpos6/modules/jpos/src/org/jpos/util/FSDMsg.java 2009-11-28 00:41:29 UTC (rev 2827)
> +++ trunk/jpos6/modules/jpos/src/org/jpos/util/FSDMsg.java 2009-12-01 14:46:49 UTC (rev 2828)
> @@ -27,11 +27,14 @@
> import java.io.UnsupportedEncodingException;
> import java.net.MalformedURLException;
> import java.net.URL;
> +import java.util.Arrays;
> import java.util.Collections;
> import java.util.HashMap;
> +import java.util.HashSet;
> import java.util.Iterator;
> import java.util.LinkedHashMap;
> import java.util.Map;
> +import java.util.Set;
>
> import org.jdom.Element;
> import org.jdom.JDOMException;
> @@ -79,11 +82,12 @@
> * <dt>FS</dt><dd>Field separator using '034' as the separator.</dd>
> * <dt>US</dt><dd>Field separator using '037' as the separator.</dd>
> * <dt>GS</dt><dd>Group separator using '035' as the separator.</dd>
> - * <dt>RS</dt><dd>Field separator using '036' as the separator.</dd>
> + * <dt>RS</dt><dd>Row separator using '036' as the separator.</dd>
> * <dt>PIPE</dt><dd>Field separator using '|' as the separator.</dd>
> * <dt>EOF</dt><dd>End of File - no separator character is emitted, but also no padding is done. Also if the end of file is reached
> * parsing a message, then no exception is thrown.</dd>
> * <dt>DS</dt><dd>A dummy separator. This is similar to EOF, but the message stream must not end before it is allowed.</dd>
> + * <dt>EOM</dt><dd>End of message separator. This reads all bytes available in the stream.
> * </dl>
> * </p>
> * <p>
> @@ -103,8 +107,10 @@
> public static char RS = '\036';
> public static char EOF = '\000';
> public static char PIPE = '\u007C';
> + public static char EOM = '\000';
>
> - private static final String DUMMY_SEPARATOR = "DS";
> + private static final Set<String> DUMMY_SEPARATORS = new HashSet<String>(Arrays.asList("DS", "EOM"));
> + private static final String EOM_SEPARATOR = "EOM";
>
> Map fields;
> Map separators;
> @@ -297,7 +303,7 @@
> }
>
> private boolean isDummySeparator(String separator) {
> - return DUMMY_SEPARATOR.equals(separator);
> + return DUMMY_SEPARATORS.contains(separator);
> }
>
> private boolean isBinary(String type) {
> @@ -436,7 +442,14 @@
> boolean expectSeparator = isSeparated(separator);
> boolean separated = expectSeparator;
>
> - if (isDummySeparator(separator)) {
> + if (EOM_SEPARATOR.equals(separator)) {
> + // Grab what's left. is.available() should work because it is normally a ByteArrayInputStream
> + byte[] rest = new byte[is.available()];
> + is.read(rest, 0, rest.length);
> + for (int i = 0; i < rest.length; i++) {
> + sb.append((char) (rest[i] & 0xff));
> + }
> + } else if (isDummySeparator(separator)) {
> /*
> * No need to look for a seperator, that is not there! Try and take
> * len bytes from the is.
>
> Added: trunk/jpos6/modules/jpos/test/org/jpos/util/FSDMsgEndOfMessageTestCase.java
> ===================================================================
> --- trunk/jpos6/modules/jpos/test/org/jpos/util/FSDMsgEndOfMessageTestCase.java (rev 0)
> +++ trunk/jpos6/modules/jpos/test/org/jpos/util/FSDMsgEndOfMessageTestCase.java 2009-12-01 14:46:49 UTC (rev 2828)
> @@ -0,0 +1,52 @@
> +/*
> + * jPOS Project [http://jpos.org]
> + * Copyright (C) 2000-2009 Alejandro P. Revilla
> + *
> + * This program is free software: you can redistribute it and/or modify
> + * it under the terms of the GNU Affero General Public License as
> + * published by the Free Software Foundation, either version 3 of the
> + * License, or (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU Affero General Public License for more details.
> + *
> + * You should have received a copy of the GNU Affero General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +package org.jpos.util;
> +
> +import java.io.ByteArrayInputStream;
> +
> +import junit.framework.TestCase;
> +
> +public class FSDMsgEndOfMessageTestCase extends TestCase {
> + // Eclipse wants :-
> + // private static final String SCHEMA_DIR_URL = "file:modules/jpos/test/org/jpos/util/";
> + // Original
> + private static final String SCHEMA_DIR_URL = "file:../modules/jpos/test/org/jpos/util/";
> + FSDMsg imsg;
> +
> + FSDMsg omsg;
> +
> + public void setUp() throws Exception {
> + imsg = new FSDMsg(SCHEMA_DIR_URL + "eom-");
> + omsg = new FSDMsg(SCHEMA_DIR_URL + "eom-");
> + }
> +
> + public void testPack() throws Exception {
> + imsg.set("length", "11");
> + imsg.set("rest", "ABCDEFGHIJKLMNOPQRST");
> + assertEquals("11ABCDEFGHIJKLMNOPQRST", imsg.pack());
> + }
> +
> + public void testUnpack() throws Exception {
> + ByteArrayInputStream is = new ByteArrayInputStream("11ABCDEFGHIJKLMNOPQRST".getBytes());
> + omsg.unpack(is);
> +
> + assertEquals("11", omsg.get("length"));
> + assertEquals("ABCDEFGHIJKLMNOPQRST", omsg.get("rest"));
> + }
> +}
>
>
> Property changes on: trunk/jpos6/modules/jpos/test/org/jpos/util/FSDMsgEndOfMessageTestCase.java
> ___________________________________________________________________
> Added: svn:mime-type
> + text/plain
>
> Added: trunk/jpos6/modules/jpos/test/org/jpos/util/eom-base.xml
> ===================================================================
> --- trunk/jpos6/modules/jpos/test/org/jpos/util/eom-base.xml (rev 0)
> +++ trunk/jpos6/modules/jpos/test/org/jpos/util/eom-base.xml 2009-12-01 14:46:49 UTC (rev 2828)
> @@ -0,0 +1,5 @@
> +<schema id='fsdmsg-test'>
> + <field id="length" type="S" length="2" />
> + <field id="rest" type="AEOM" length="32" />
> +</schema>
> +
>
>
> Property changes on: trunk/jpos6/modules/jpos/test/org/jpos/util/eom-base.xml
> ___________________________________________________________________
> Added: svn:mime-type
> + text/plain
>
>
> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>
> --
> You received this message because you are subscribed to the Google
> Groups "jpos-commits" group.
>
> To unsubscribe from this group, send email to
> jpos-commits...@googlegroups.com
>
> For more options, visit this group at
> http://groups.google.com/group/jpos-commits?hl=en

[Jonathan]

David,
EOF is used to create an EOF field in FSDMsg when it runs out of data
during parsing. Instead, I want to grab everything that is left, when
I get a field marked EOM. In fact, the EOM field should always be the
last one in the schema. If you need a mnemonic:
EOM = End Of too MUCH
EOF = End Of too FEW

What I want it for is to handle the transaction request message in NDC
+. Andy already warned me not to enhance FSDMsg to handle that very
complicated format. So, now I am going to put most of the message into
an "And The Rest" field, and parse it with Java code.

Ciao,
Jonathan

[David Bergert]

Got it - that makes sense:

We have a fixed length message that we do this for in a similar manner:

<field id="message-tail" type="AEOF" length="3932" /> <!-- rest of the message -->

I suspect that with your changes one could do, instead or if the length is variable the following:

<field id="message-tail" type="AEOM"> <!-- rest of the message -->

Thanks and Ciao ;)

David Bergert, CISSP, CISA, CPISM/A
www.paymentsystemsblog.com

[Jonathan]

David,
I'm an idiot. I forgot about setting the length to some crazy amount.

My implementation also relies on the InputStream.available() returning
the number of bytes left to read. This is correct if you use
BaseChannel.receive(), but may not be correct in other scenarios. Your
way of doing this will also work with BaseChannel.receive(), but it
might break if you are calling getMessageLength() and getMessage()
driectly. There are cases where you only partially read the message
from the stream, then InputStream.available() will return less than
what really should be read.

Naturally, you won't see an EOF field in the FSDMsg after unpacking
with my version.

Ciao,
Jonathan