Multiple header and footer

[spring-user]

Hiya,
I have a fixed length file with multiple headers and footers, the details are always the same format.
Does beanio support this? Any sample code I can look at?

My file format is something like this

HEADER|DATETIME1|FILENAME
A RECORDDESC1
B RECORDDESC2
C RECORDDESC3
FOOTER|3
HEADER|DATETIME2|FILENAME
D RECORDDESC4
E RECORDDESC5
F RECORDDESC6
FOOTER|3

thanks in advance!

[Kevin]

Hello,

Yes, BeanIO can support that.  You'll just need to define a group for each header-detail-footer combination.

Thanks,

Kevin

[RJ Mugas]

Hi,

I have an issue with custom my Header and Trailer for BeanIO, consider this beanio xml below

  

<stream name="stream" format="fixedlength" >

<record name="header" minOccurs="1" maxOccurs="1" class="myHeaderBean">

  <field name="recordType" length="1" position="0" rid="true" literal="H" />

<field name="generatedDate" length="8" position="1" />

  </record>

<record name="details" minOccurs="1" maxOccurs="unbounded" class="myBean"> 

<field name="recordType" length="1" position="0" rid="true" literal="D" />

<!-- other fields follows -->

</record>

<record name="trailer" minOccurs="1" maxOccurs="1" class="myTrailerBean">

  <field name="recordType" length="1" position="0" rid="true" literal="T" />

<field name="recordTotal" length="12" position="1" />

  </record>

</stream>

and this is my spring-batch job xml config

<batch:job id="myJob" incrementer="incrementer">

<batch:step id="stepProcess">

<batch:tasklet transaction-manager="transactionManager">

<batch:chunk reader="myReader" processor="myProcessor"

writer="myWriter" commit-interval="100" skip-limit="1000"

retry-limit="10">

<batch:skippable-exception-classes>

<batch:include class="org.beanio.InvalidRecordException" />

<batch:exclude class="java.io.FileNotFoundException" />

</batch:skippable-exception-classes>

<batch:retryable-exception-classes>

<batch:include

class="org.springframework.dao.DeadlockLoserDataAccessException" />

</batch:retryable-exception-classes>

</batch:chunk>

</batch:tasklet>

</batch:step>

</batch:job>

<bean id="myReader" class="org.beanio.spring.BeanIOFlatFileItemReader">

<property name="streamMapping" value="${mapping.file.location}" />

<property name="streamName" value="${stream.name}" />

<property name="resource" value="${input.file.location}" />

</bean>

<bean id="incrementer"

class="org.springframework.batch.core.launch.support.RunIdIncrementer" />

<bean id="myProcessor"

class="myProcessor" />

<bean id="myWriter"

class="myWriter" />

Java codes goes like this

@Component

public class myProcessor 

implements ItemProcessor<myBean, myEntity> {

@Override

public myEntity process(myBean myBean) 

throws Exception {

// code to build myEntity here..

return myEntity;

}

}

@Component

public class myWriter 

implements ItemWriter<myEntity> {

@Override

public void write(List<? extends myEntity> myEntities) throws Exception {

// code to save myEntities here

}

}

Issue -> ClassCastException on process because myHeaderBean can't be translated to myBean -- which is the only bean passed during processs

Question -> How to avoid passing myHeaderBean and myTrailerBean during process? Sorry can't find a good article on internet for this issue.

Thanks in advance

[Nico Schlebusch]

Hi,

I think you have a few options here.

1. If you use a JEE 7 application server, use the standard JEE 7 Batch jobs instead of Spring Batch. I don't know the differences between them, but then you get an Object as parameter in your ItemProcessor's processItem method. Using this approach I would just test the instance of the object received to see if I want to process the object or not.
2. You can use a record group to read the header, the detail records and trailer record in one go. Then your ItemProcessor will receive an object (for example: MyGroup that encapsulates the header, detail records and the trailer. Then it is easy to just process the detail records.
   
   <group name="myGroup" class="com.example.MyGroup" minOccurs="1" maxOccurs="unbounded">

1. 
     <record name="header" minOccurs="1" maxOccurs="1" class="myHeaderBean">
         <field name="recordType" length="1" position="0" rid="true" literal="H" />
       <field name="generatedDate" length="8" position="1" />
       </record>
     <record name="details" minOccurs="1" maxOccurs="unbounded" class="myBean">
       <field name="recordType" length="1" position="0" rid="true" literal="D" />
   
       <!-- other fields follows -->
     </record>
     <record name="trailer" minOccurs="1" maxOccurs="1" class="myTrailerBean">
         <field name="recordType" length="1" position="0" rid="true" literal="T" />
       <field name="recordTotal" length="12" position="1" />
     </record>
   

1. </group>
   
   
2. Don't map the Header and Trailer records in your BeanIO mapping file, only the Detail record if you really don't care about the header and trailer records. Al you then need to do is to add the ignoreUnidentifiedRecords="true" attribute to your <stream .. /> definition.
   
   <stream name="stream" format="fixedlength" ignoreUnidentifiedRecords="true">
   

Kind regards,
Nico Schlebusch

[RJ Mugas]

Hi Nico,

Thanks for your response.  We have implemented the 2nd Option and the app went on PRD.

Appreciate your help, and sorry for replying this long :)

Regards,

[Nico Schlebusch]

Hi,

Great stuff! Glad you got it working.

Kind regards,
Nico Schlebusch