Nested Segments and multiple lists

[Ivan Antunovic]

Hello people,

I would kindly ask if someone could give me an assistance.  I googled for the whole day, but I still canßt figure it out. Here is my code:

<segment name="gprsRecord" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.GPRSRecord">

<segment name="pgwRecord" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.PGWRecord">

<field name="sgsnAddressLength" default="0" type="java.lang.Integer"  />

<!-- sgsnAddresses -->

<segment name="servingNodeAddress" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ServingNodeAddress">

<segment name="seqOf" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.GSNAddress"

collection="list" minOccurs="1" occursRef="sgsnAddressLength"  >

</segment>

</segment>

<segment name="listOfServiceData" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ListOfServiceData">

<segment name="seqOf" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ChangeOfServiceCondition" 

collection="list" minOccurs="0" maxOccurs="10">

.

.

.

</segment>   

</segment>

<!-- I have the same segment again here, since I need to save some fields twice into CSV file -->

<segment name="listOfServiceData" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ListOfServiceData">

<segment name="seqOf" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ChangeOfServiceCondition" 

collection="list" minOccurs="1" maxOccurs="10">

<field name="dataVolumeGPRSUplink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>

<field name="dataVolumeGPRSDownlink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>

<field name="changeCondition" type="java.lang.String" typeHandler="changeConditionHandler"/> 

</segment>   

</segment>

</segment>

</segment>

For the first segment I am receiving: "Invalid segment 'seqOf', in segment 'servingNodeAddress' Referenced field 'sgsnAddressLength' must precede this field". 

Well sgsnAddressLength field already precedes servingNodeAddress.

For the last segment (2nd occurance of listOfServiceData segment) I am receiving: "Invalid segment 'seqOf', in segment 'listOfServiceData',  A segment of indeterminate size may not follow another component of indeterminate size".

I have written maxOccurs=10 for the last segment, so I am not quite sure how is that indeterminate size?

[Nico Schlebusch]

Hi Ivan

First of all, why are you only using segments to map your file structure? Please tell us more of the file you are trying to map. File type? delimited, fixed length, xml etc.
Also, try to make the names you assign to a recordGroup, record, segment unique, so that the error messages can direct you to the correct problem, instead of you having to guess which one of the 3 seqOf segments have the problem.

For the first segment I am receiving: "Invalid segment 'seqOf', in segment 'servingNodeAddress' Referenced field 'sgsnAddressLength' must precede this field". 

Well sgsnAddressLength field already precedes servingNodeAddress.

This is my understanding of segments and records, which might not be 100% accurate. You can use a record and a segment interchangeable in most places. Segments are really only needed when you have a "nested bean object or collection of bean objects" or when reading/writing to an XML file. From the documentation - http://beanio.org/2.1/docs/reference/index.html#Segments

A segment is a group of fields within a record. Segments are most often used to bind a group of fields to a nested bean object or collection of bean objects, and are configured in a mapping file using a segment element.

So from my understanding of segments and records and the quoted part of the documentation above, you have a record inside another record.

Repeating segments - http://beanio.org/2.1/docs/reference/index.html#RepeatingSegments

Just like repeating fields, if the number of occurrences of a segment is dependent on a preceding field in the same record, the occursRef attribute can be set to the name of the field that controls the number of occurrences.

[emphasis mine]. I also think that a nested record can not read a value from it parent via occursRef. From this, if you want to use occursRef the field declaring the value (seqOf) and the segment that then uses this value must be in the same record (or segment).

I really think you are overusing segments, a record group and records should be able to map what you are trying to do. Try to keep the mapping as simple as possible (by using recordGroups and records) in the beginning before going down the road of the more advanced concepts where segments come into play.

For the last segment (2nd occurance of listOfServiceData segment) I am receiving: "Invalid segment 'seqOf', in segment 'listOfServiceData',  A segment of indeterminate size may not follow another component of indeterminate size".

I have written maxOccurs=10 for the last segment, so I am not quite sure how is that indeterminate size?

I don't have a definitive answer for you here, but I think if you solve the first problem, it will be easier to solve the second one. Again, I think the overuse of segments is causing this problem.

If this doesn't help, please post more example data that would help us understand the problem better.

Kind regards,
Nico Schlebusch

[Ivan Antunovic]

Dear Nico, 

thank you for answering me. I am trying to map ASN1 (cr24 - https://www.cisco.com/c/en/us/td/docs/wireless/asr_5000/21-3_N5-5/GTPP/21-3-GTPP-Reference/21-1-GTPP-Reference_chapter_011.html?dtid=osscdc000283) file and parse it back into CSV file. 

Classes are generated by ASN1 compiler, and so are the names of member fields. I have changed the names, as you suggested, to make it more understandable. So this is the class structure I am having ( setters and getters omitted):

Here are also text-uploader links, in case forum messes with the text.

Java Classes: http://textuploader.com/d67eb

XML File: http://textuploader.com/d67e8

public class GprsEvent implements Event
{
 private GPRSRecord gprsRecord ;
}


public class GPRSRecord implements Serializable
{
 private PGWRecord pgwRecord = null;
}


public class PGWRecord implements Serializable
{
 // non-complex/non-nested data types
 .
 .
 .
 // nested data types - segments in XML file
 // First List in the XML example -> name="gsnAddressList"
 private ServingNodeAddress servingNodeAddress;
 // Second List in the XML example -> name="changeOfServiceConditionList"
 private ListOfServiceData listOfServiceData ;
}


// First List in the XML example -> name="gsnAddressList"
public class ServingNodeAddress implements Serializable
{
 private List<GSNAddress> gsnAddressList = null;
}


// nested data type class related to Second List in the XML example -> name="changeOfServiceConditionList"
public class ListOfServiceData implements Serializable
{
 private List<ChangeOfServiceCondition> changeOfServiceConditionList = null;
}


// nested data type class related to Second List in the XML example -> name="changeOfServiceConditionList"
public class ChangeOfServiceCondition implements Serializable
{
 // non-complex/non-nested data types
 .
 .
 .
 // nested data type
 private EPCQoSInformation qoSInformationNeg = null;


}


// nested data type class related to Second List in the XML example -> name="changeOfServiceConditionList"
public class EPCQoSInformation implements Serializable
{
 // non-complex/non-nested data types
 private BerInteger qCI = null;
 private BerInteger maxRequestedBandwithUL = null;
 private BerInteger maxRequestedBandwithDL = null;
 private BerInteger guaranteedBitrateUL = null;
 private BerInteger guaranteedBitrateDL = null;
 private BerInteger aRP = null;
}

So here is the XML:

<record class="net.atos.tools.radiation.convert.event.gprs.GprsEvent"
 name="event" maxOccurs="unbounded" minOccurs="0" order="1">

    <segment name="gprsRecord" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.GPRSRecord">
 <segment name="pgwRecord" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.PGWRecord">

 <segment name="servingNodeAddress" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ServingNodeAddress">

 <segment name="gsnAddressList" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.GSNAddress"
 collection="list" minOccurs="1" maxOccurs="1" >

 </segment>
 </segment>
 
 <segment name="listOfServiceData" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ListOfServiceData">

 <segment name="changeOfServiceConditionList"  class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ChangeOfServiceCondition"
 collection="list" minOccurs="0" maxOccurs="10">
 .
 . <!-- fields -->
 .
 <segment name="qoSInformationNeg" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.EPCQoSInformation">
 <field name="qCI" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 <field name="maxRequestedBandwithUL" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 <field name="maxRequestedBandwithDL" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 <field name="guaranteedBitrateUL" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 <field name="guaranteedBitrateDL" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 <field name="aRP" type="org.openmuc.jasn1.ber.types.BerInteger" typeHandler="berIntegerHandler"/>
 </segment>
 <field name="datavolumeFBCUplink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>
 <field name="datavolumeFBCDownlink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>
 <field name="serviceConditionChange" type="java.lang.String" typeHandler="changeConditionHandler"/>
 .
 . <!-- fields -->
 .
 </segment>  
 </segment>

 <segment name="listOfServiceData" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ListOfServiceData">

 <segment name="changeOfServiceConditionList" class="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.ChangeOfServiceCondition"
 collection="list" minOccurs="1" maxOccurs="1">
 <!-- again the same fields as above, since I have to store them twice -->
 <field name="datavolumeFBCUplink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>
 <field name="datavolumeFBCDownlink" type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.DataVolumeGPRS" typeHandler="dataVolumeGPRSLinkHandler"/>
 <field name="serviceConditionChange" type="java.lang.String" typeHandler="changeConditionHandler"/>
 </segment>  
 </segment>
 </segment>  
 </segment>
</record>

I hope this made it atleast more clear to understand the problem. 

[Ivan Antunovic]

Okay I simply set minOccurs=maxOccurs and that did the trick.

However I have another question, I saw that in BeanIO reference guide (http://beanio.org/2.1/docs/reference/index.html), BeanIO supports date forma.t I would like to know if I can configure time format for non-standard Java class. I wouldn't like to hardcode it in my handler something like:

                Date date;

java.text.SimpleDateFormat outputSimpleDateFormatter = new SimpleDateFormat(OUTPUT_TIME_STAMP_PATTERN);

try

{

SimpleDateFormat simpleDateFormat = GRAMMAR_EVENT_VALUE_TIMESTAMP_DISPLAY_FORMAT.get();

if ( utcTimeStamp.contains(PLUS_SIGN_ASCII_HEX_VALUE) )

{

date = simpleDateFormat.parse(utcTimeStamp.replace(PLUS_SIGN_ASCII_HEX_VALUE, UTC_PLUS));

}

else

{

date = simpleDateFormat.parse(utcTimeStamp.replace(MINUS_SIGN_ASCII_HEX_VALUE, UTC_MINUS));

}

return outputSimpleDateFormatter.format(date);

} 

catch (java.text.ParseException e)

{

throw new ConvertException("Error while converting UTC Time Stamp.");

}

I have my compiler-generated class called TimeStamp, with a handler:

<typeHandler name="timeStampHandler"

class="net.atos.tools.radiation.convert.type.TimeStampHandler">

<property name="pattern" value="yyyy-MM-dd HH:mm:ss" />

<property name="timeZoneId" value="Europe/Zagreb" />

</typeHandler>

Here is an example of the field:

<field name="recordOpeningTime"

          type="net.atos.tools.radiation.convert.event.ggsn.egcdr.ggsn.charging.datatypes.TimeStamp"

  ypeHandler="timeStampHandler" />

Best regards!

[Nico Schlebusch]

Hi Ivan,

Glad to see you have made progress. Which method are you implementing/overriding in the code snippet?

Did you look at what the following classes provide for you:

• org.beanio.types.DateTypeHandler
• You can most likely override some of these methods to achieve what you want
  
• or the superclass - org.beanio.types.DateTypeHandlerSupport
• Also look at the interfaces these 2 classes implement
  

Another example of a custom TypeHandler:

• https://stackoverflow.com/a/9542872/2020886 - Look at the answer from 'Kevin' he is the author of BeanIO.

Hope this helps.

Kind regards,
Nico Schlebusch