UN-EDIFACT Mapping

[Jean-Francois Clavier]

Hi All,

I'm trying smooks as it seems to perfectly fullfill my needs to translate EDI message from/to flatfile.
I first tried to create a mapping from unedifact DESADV D96A to a flatfile using smooks 1.7.1 as the documentation for smooks 2 is not available yet.
If a solution is available for vesion 2, I will be pleased to test it.
I tried to test the edifact-to-xml file with smooks 2 (.m2\repository\org\smooks\cartridges\edi\edifact-schemas\2.0.0-M2-SNAPSHOT) but got an error at test time.
org.smooks.cdr.SmooksConfigurationException: org.smooks.cdr.SmooksConfigurationException: java.lang.RuntimeException: Schema Definition Error: Error loading schema due to org.xml.sax.SAXParseException; systemId: file:/C:/Users/JEAN-F~1/AppData/Local/Temp/EDIFACT-Interchange2354620367605440522.dfdl.xsd
when loading the file in notepad++ I can see that the copyright characters © are transformed in xA9 and xerces tells that encoding is incorrect. Files in .m2 are corrects. I'm on Windows 10.

Looking to the edi to java example I see that we can read an edi file to catch data in beans.
Looking to file routing/splitter router example I see that we can use a freemarker template to create an output stream using beans data.

My first question is, would it be possible to read EDI file using unedifact:reader, bind selected data to beans and use template to create flatfile.

I tried with the following code but got an error "freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> infolog  [in template "free-marker-template" at line 1, column 10]"

  <unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.7.0" ignoreNewLines="true" />
  <import file="/org/milyn/edi/unedifact/d96a/interchange-bindingconfig.xml" />
  <import file="/org/milyn/edi/unedifact/d96a/message-bindingconfig.xml" />
  <import file="/org/milyn/edi/unedifact/d96a/DESADV/bindingconfig.xml" />

    <jb:bean beanId="infolog" class="infolog.model.SolinestFile" createOnElement="BGM">
        <jb:wiring property="header" beanIdRef="header" />
        <jb:wiring property="Items" beanIdRef="ItemList" />
        <jb:wiring property="trailer" beanIdRef="trailer" />
    </jb:bean>

    <jb:bean beanId="header" class="infolog.model.Header" createOnElement="${infolog}/header">
        <jb:wiring property="bgm" beanIdRef="bgm" />
        <jb:wiring property="rffon" beanIdRef="rffon" />
    </jb:bean>
    <jb:bean beanId="bgm" class="infolog.model.Bgm" createOnElement="BGM">
        <jb:value property="documenttype" data="Document_message_name__coded"/>
        <jb:value property="documentnumber" data="Document_message_number"/>
    </jb:bean>

...
    <ftl:freemarker applyOnElement="UNT">
        <ftl:template><!--00.00FGE50MLOML    FGE50MLOML           0                      FGE50MLOML                                                                                                                                                                                
-->
        </ftl:template>
        <ftl:use>
            <!-- Output the templating result to the "desadvSplitStream" file output stream... -->
            <ftl:outputTo outputStreamResource="desadvSplitStream"/>
        </ftl:use>
    </ftl:freemarker>

    <!-- Create/open a file output stream.  This is writen to by the freemarker template (above).. -->
    <file:outputStream resourceName="desadvSplitStream" openOnElement="UNT">
        <file:fileNamePattern>desadv-${infolog.header.bgm.documentnumber}.DAT</file:fileNamePattern>
        <file:destinationDirectoryPattern>infolog</file:destinationDirectoryPattern>
        <file:listFileNamePattern>desadv-${infolog.header.bgm.documentnumber}.lst</file:listFileNamePattern>

        <file:highWaterMark mark="3"/>
    </file:outputStream>
</smooks-resource-list>

I then tried to use ${unEdifactMessage.message.bGMBeginningOfMessage.e1004DocumentMessageNumber} without success.
I can access interchange header i.e. ${unEdifactMessage.interchangeHeader.syntaxIdentifier.id} but not message level.
Is there a way to bind the message level ?

My second question is for the flatfile to unedifact mapping.

Is it possible to use ejc on unedifact model ? If so, how (I got an error building EJC example Could not resolve dependencies for project org.milyn:milyn-smooks-example-ejc-model-use:bundle:1.7.1: Failed to collect dependencies at org.milyn:milyn-smooks-example-ejc-model-gen:jar:1.7.1) ?
so far, I understand that we can only populate output file using java. Is it right ?

Thank you

[Claude Mamo]

Welcome to the forum Jean-Francois! We're still preparing the docs for Smooks 2 but the unpublished docs for 2.0.0-M1 can be found at https://github.com/smooks/smooks.github.io/blob/v2.0.0-M1/docs.markdown.

I tried to test the edifact-to-xml file with smooks 2 (.m2\repository\org\smooks\cartridges\edi\edifact-schemas\2.0.0-M2-SNAPSHOT)

In general, snapshots are unstable so you should test with 2.0.0-M1: https://github.com/smooks/smooks-examples/tree/v1.0.0/edifact-to-xml.

My first question is, would it be possible to read EDI file using unedifact:reader, bind selected data to beans and use template to create flatfile.

I can't see why not. You can skip the Java binding and leverage the DomModelCreator to transparently bind fragments to DOM nodes and then go on to read the nodes directly from the FreeMarker visitor. An example is shown in the docs. Give this a try and let me know if the same error pops out.

Is it possible to use ejc on unedifact model ?

It's possible but you shouldn't need to for Smooks 1.7. The Java bindings were generated beforehand and published to Maven Central. Check out the docs.

so far, I understand that we can only populate output file using java. Is it right ?

Smooks is a Java framework so it's a given that Java must be used to populate to a file from within Smooks. However, as mentioned above, visitors like FreeMarker can read data from Java objects other than Java beans. Another example is the DFDL visitor introduced in Smooks 2 which can be used to write flat files directly from XML.

Claude

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/5973965d-23e8-4f6e-b159-f3e46df07fafo%40googlegroups.com.

[Jean-Francois Clavier]

Thanks for your quick answer Claude

I tried to follow your recommandations but still need your help.

Le vendredi 30 octobre 2020 18:22:37 UTC+1, Claude a écrit :

Welcome to the forum Jean-Francois! We're still preparing the docs for Smooks 2 but the unpublished docs for 2.0.0-M1 can be found at https://github.com/smooks/smooks.github.io/blob/v2.0.0-M1/docs.markdown.

I tried to test the edifact-to-xml file with smooks 2 (.m2\repository\org\smooks\cartridges\edi\edifact-schemas\2.0.0-M2-SNAPSHOT)

In general, snapshots are unstable so you should test with 2.0.0-M1: https://github.com/smooks/smooks-examples/tree/v1.0.0/edifact-to-xml.

I just tried with 2.0.0-M1 but got the same error.I guess the reason is I'm on Windows and there is an invalid transcoding of the file EDIFACT-Interchange.dfdl.xsd.mustache when resolving version in temp.

 

My first question is, would it be possible to read EDI file using unedifact:reader, bind selected data to beans and use template to create flatfile.

I can't see why not. You can skip the Java binding and leverage the DomModelCreator to transparently bind fragments to DOM nodes and then go on to read the nodes directly from the FreeMarker visitor. An example is shown in the docs. Give this a try and let me know if the same error pops out.

I saw the documentation but purhaps I miss somethings. In the example there is no mappingModel as the source file is xml.

Which variables can I use in my template to populate the destination file with data from the edifact source message? Which selector can be used to access message and message items ?

As mentioned in my original post, I then tried to use ${unEdifactMessage.message.bGMBeginningOfMessage.e1004DocumentMessageNumber} without success.

I can access interchange header i.e. ${unEdifactMessage.interchangeHeader.syntaxIdentifier.id} but not message level.

 

Is it possible to use ejc on unedifact model ?

It's possible but you shouldn't need to for Smooks 1.7. The Java bindings were generated beforehand and published to Maven Central. Check out the docs.

so far, I understand that we can only populate output file using java. Is it right ?

Smooks is a Java framework so it's a given that Java must be used to populate to a file from within Smooks. However, as mentioned above, visitors like FreeMarker can read data from Java objects other than Java beans. Another example is the DFDL visitor introduced in Smooks 2 which can be used to write flat files directly from XML.

 

Claude

To unsubscribe from this group and stop receiving emails from it, send an email to smook...@googlegroups.com.

[Claude Mamo]

I just tried with 2.0.0-M1 but got the same error.I guess the reason is I'm on Windows and there is an invalid transcoding of the file EDIFACT-Interchange.dfdl.xsd.mustache when resolving version in temp.

Can you open a ticket? We'll investigate it before releasing M2.

I saw the documentation but purhaps I miss somethings. In the example there is no mappingModel as the source file is xml.

The example is a general one: you need to add the reader.

Which variables can I use in my template to populate the destination file with data from the edifact source message?

With DomModelCreator, the variables are the nodes captured from the XML stream.

Which selector can be used to access message and message items ?

I'm not too familiar with the EDIFACT mapping but you can easily find out which selector/s to use by looking at the generated XML stream via HtmlReportGenerator.

Claude

On Mon, Nov 2, 2020 at 9:55 AM Jean-Francois Clavier <jeanfranco...@gmail.com> wrote:

Thanks for your quick answer Claude

I tried to follow your recommandations but still need your help.

Le vendredi 30 octobre 2020 18:22:37 UTC+1, Claude a écrit :

Welcome to the forum Jean-Francois! We're still preparing the docs for Smooks 2 but the unpublished docs for 2.0.0-M1 can be found at https://github.com/smooks/smooks.github.io/blob/v2.0.0-M1/docs.markdown.

I tried to test the edifact-to-xml file with smooks 2 (.m2\repository\org\smooks\cartridges\edi\edifact-schemas\2.0.0-M2-SNAPSHOT)

In general, snapshots are unstable so you should test with 2.0.0-M1: https://github.com/smooks/smooks-examples/tree/v1.0.0/edifact-to-xml.

I just tried with 2.0.0-M1 but got the same error.I guess the reason is I'm on Windows and there is an invalid transcoding of the file EDIFACT-Interchange.dfdl.xsd.mustache when resolving version in temp.

 

My first question is, would it be possible to read EDI file using unedifact:reader, bind selected data to beans and use template to create flatfile.

I can't see why not. You can skip the Java binding and leverage the DomModelCreator to transparently bind fragments to DOM nodes and then go on to read the nodes directly from the FreeMarker visitor. An example is shown in the doI saw the documentation but purhaps I miss somethings. In the example there is no mappingModel as the source file is xml.cs. Give this a try and let me know if the same error pops out.

To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/aa9d331b-a5e9-4ed1-a036-ba8ab6706649o%40googlegroups.com.

[Jean-Francois Clavier]

Hi Claude,

I opened a ticket.

When I look to the htmlreport I can see the interchange tree.

I can use variables such as ${unEdifactInterchange.interchangeHeader.syntaxIdentifier.id} but when I want to access th the message element I don't find the right path.

unEdifactInterchange:

<org.milyn.smooks.edi.unedifact.model.r41.UNEdifactInterchange41>

<interchangeHeader>

<syntaxIdentifier>

<id>UNOC</id>

<versionNum>3</versionNum>

</syntaxIdentifier>

<sender>

<id>3011743300100</id>

<codeQualifier>14</codeQualifier>

</sender>

<recipient>

<id>3016611910017</id>

<codeQualifier>14</codeQualifier>

</recipient>

<date>

<date>200918</date>

<time>1422</time>

</date>

<controlRef>2654</controlRef>

</interchangeHeader>

<interchangeTrailer>

<controlCount>1</controlCount>

<controlRef>2654</controlRef>

</interchangeTrailer>

<messages>

<org.milyn.smooks.edi.unedifact.model.r41.UNEdifactMessage41>

<interchangeHeader reference="../../../interchangeHeader"/>

<messageHeader>

<messageRefNum>1</messageRefNum>

<messageIdentifier>

<id>DESADV</id>

<versionNum>D</versionNum>

<releaseNum>96A</releaseNum>

<controllingAgencyCode>UN</controllingAgencyCode>

<associationAssignedCode>EAN005</associationAssignedCode>

</messageIdentifier>

</messageHeader>

<messageTrailer>

<segmentCount>179</segmentCount>

<messageRefNum>1</messageRefNum>

</messageTrailer>

<message class="org.milyn.edi.unedifact.d96a.DESADV.Desadv">

<bGMBeginningOfMessage>

<c002DocumentMessageName>

<e1001DocumentMessageNameCoded>351</e1001DocumentMessageNameCoded>

</c002DocumentMessageName>

<e1004DocumentMessageNumber>80975091</e1004DocumentMessageNumber>

<e1225MessageFunctionCoded>9</e1225MessageFunctionCoded>

</bGMBeginningOfMessage>

        <dTMDateTimePeriod>

The following example give the expected result

syntaxIdentifier : UNOC
date : 20201104113651

<?xml version="1.0"?>
<smooks-resource-list
  xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
  xmlns:edi="http://www.milyn.org/xsd/smooks/edi-1.2.xsd"
  xmlns:unedifact="http://www.milyn.org/xsd/smooks/unedifact-1.4.xsd"
  xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
  xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.2.xsd"
  xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd"
  xmlns:file="http://www.milyn.org/xsd/smooks/file-routing-1.1.xsd"
  xmlns:frag="http://www.milyn.org/xsd/smooks/fragment-routing-1.4.xsd">

  <!--
  Configure the EDI Reader to parse the message stream into a stream of SAX events.
  -->

  <unedifact:reader mappingModel="urn:org.milyn.edi.unedifact:d96a-mapping:1.7.0" ignoreNewLines="true" />
  <import file="/org/milyn/edi/unedifact/d96a/interchange-bindingconfig.xml" />
  <import file="/org/milyn/edi/unedifact/d96a/message-bindingconfig.xml" />
  <import file="/org/milyn/edi/unedifact/d96a/DESADV/bindingconfig.xml" />

    <ftl:freemarker applyOnElement="UNT">
        <ftl:template><!--

syntaxIdentifier : ${unEdifactInterchange.interchangeHeader.syntaxIdentifier.id}
date : ${.now?string("yyyyMMddHHmmss")}

-->
        </ftl:template>
        <ftl:use>
            <!-- Output the templating result to the "desadvSplitStream" file output stream... -->
            <ftl:outputTo outputStreamResource="desadvSplitStream"/>
        </ftl:use>
    </ftl:freemarker>

    <!-- Create/open a file output stream.  This is writen to by the freemarker template (above).. -->
    <file:outputStream resourceName="desadvSplitStream" openOnElement="UNT">

        <file:fileNamePattern>desadv-${unEdifactInterchange.interchangeHeader.sender.id}.DAT</file:fileNamePattern>
        <file:destinationDirectoryPattern>target/result</file:destinationDirectoryPattern>
        <file:listFileNamePattern>desadv-infolog.lst</file:listFileNamePattern>

        <file:highWaterMark mark="1"/>
    </file:outputStream>
</smooks-resource-list>

Another point is that it only works if I use UNH or UNT selector, not with i.e. BGM

Le mercredi 4 novembre 2020 09:18:41 UTC+1, Claude a écrit :

I just tried with 2.0.0-M1 but got the same error.I guess the reason is I'm on Windows and there is an invalid transcoding of the file EDIFACT-Interchange.dfdl.xsd.mustache when resolving version in temp.

Can you open a ticket? We'll investigate it before releasing M2.

I saw the documentation but purhaps I miss somethings. In the example there is no mappingModel as the source file is xml.

The example is a general one: you need to add the reader.

Which variables can I use in my template to populate the destination file with data from the edifact source message?

With DomModelCreator, the variables are the nodes captured from the XML stream.

Which selector can be used to access message and message items ?

I'm not too familiar with the EDIFACT mapping but you can easily find out which selector/s to use by looking at the generated XML stream via HtmlReportGenerator.

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/aa9d331b-a5e9-4ed1-a036-ba8ab6706649o%40googlegroups.com.

[Claude Mamo]

I opened a ticket.

Thanks for creating an issue. The fix will be rolled out in 2.0.0-M2 but, as a workaround in 2.0.0-M1, set the JVM "file.encoding" property to UTF-8.

Another point is that it only works if I use UNH or UNT selector, not with i.e. BGM

I'd have to see the whole mapped XML doc but keep in mind that the FreeMarker visitor won't have access to all the targeted element's descendants unless you use DomModelCreator.

Claude

To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/2fd0e022-ba53-435c-bd66-0de1f83928f0o%40googlegroups.com.

[Jean-Francois Clavier]

I understand that we need to use DomModelCreator, but I would need an example of how to use it with unedifact:reader.

I have the same problem with smooks 2 DFDL edifact model. I don't see how to bind data.

Currrently, I'm using UNEdifactInterchange41, UNEdifactMessage41 to solve this, but will it be available with DFDL schemas ?

Jean-François

Le mardi 10 novembre 2020 08:30:26 UTC+1, Claude a écrit :

I opened a ticket.

Thanks for creating an issue. The fix will be rolled out in 2.0.0-M2 but, as a workaround in 2.0.0-M1, set the JVM "file.encoding" property to UTF-8.

Another point is that it only works if I use UNH or UNT selector, not with i.e. BGM

I'd have to see the whole mapped XML doc but keep in mind that the FreeMarker visitor won't have access to all the targeted element's descendants unless you use DomModelCreator.

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/2fd0e022-ba53-435c-bd66-0de1f83928f0o%40googlegroups.com.

[Claude Mamo]

I understand that we need to use DomModelCreator, but I would need an example of how to use it with unedifact:reader.

Found this example: https://github.com/lunatech-labs/smooks-examples/blob/df19c723ef1888cbe21256b110a27f70883264cb/parse-custom-edi/src/main/resources/smooks-config.xml.

 

Currrently, I'm using UNEdifactInterchange41, UNEdifactMessage41 to solve this, but will it be available with DFDL schemas ?

Java bindings for the DFDL schemas will be made available in 2.0.0-M2. Having said this, you can generate the bindings on your own as shown in this example: https://github.com/claudemamo/smooks-java-to-edifact.

Claude

To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/abe723df-185a-40f7-becb-c5372f7c702bo%40googlegroups.com.