Edifact Orders & Invoice messages D.96A edifact2java

[giova....@gmail.com]

Hi Dears,
this is my first experince relative to EDIFACT and Smooks framework.
I need to make a simple gateway to import/export EDIFACT messages (ORDERS & INVOICE release D.96A) to our ERP software.
Reading docs and example seems that is possible to import and EDI file and mapping segment and fields (through available cartridge ) to JAVA object and Bean allowing to perform task to this objects.

edifact-to-* examples are very poor because limited to obtaing only a Interchange object and that does not allow to really undestand how mapping to POJO works.

I'm seeking for a sample that import EDIFACT file for example ORDERS and get OrderList segment mapping them to java object to allow to easily get order item and relative field like item code, quantity price and so on.

Viceversa, starting from a java object (for example orderList) to populate edifact segments.

Some examples like edi-to-* seems to implement something similar and are less "low-level" rather other examples.

I'm very confused and I need some less low-level examples that can help me understand the functioning and potential of the framework.

Thanks for you help.

[giova....@gmail.com]

Hi Dear,

Anyone who could point me in the right direction?

Thanks.

[Claude Mamo]

Welcome to the Smooks community. 

edifact-to-* examples are very poor because limited to obtaing only a Interchange object and that does not allow to really undestand how mapping to POJO works.

The edifact-to-java example illustrates Java binding without any mapping. As you hinted, the edi-to-java example illustrates mapping and binding with the JavaBean cartridge but, in this example, you can definitely substitute EDI with EDIFACT, or any other data format for that matter. I can understand the confusion: we should document these two different approaches for binding EDIFACT documents to Java POJOs. Your use case suggests that the edi-to-java example is a good starting point for your project.

You are more than welcome to edit the agenda and join the community call this Thursday. We can discuss and offer guidance. Like most flexible frameworks, there's more than one way to skin a cat in Smooks.

Claude

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/smooks-user/196970cc-48ef-4121-9945-33f6d60608dfn%40googlegroups.com.

[Giovanni Licata]

Welcome to the Smooks community. 

Thanks 

edifact-to-* examples are very poor because limited to obtaing only a Interchange object and that does not allow to really undestand how mapping to POJO works.

The edifact-to-java example illustrates Java binding without any mapping. As you hinted, the edi-to-java example illustrates mapping and binding with the JavaBean cartridge but, in this example, you can definitely substitute EDI with EDIFACT, or any other data format for that matter. I can understand the confusion: we should document these two different approaches for binding EDIFACT documents to Java POJOs. Your use case suggests that the edi-to-java example is a good starting point for your project.

Let me better understand ... from my perspective I don't see two different approaches ... but in edi-to-java example I see that it allows to maps POJO (Sax Event stream) to java class  (Header, Order, Customer ...) while edifact-to-java example seems to does not support this functionality.

So following the example edi-to-java I should create from scratch the following;
1) edifact-to-java-order-mapping.dfdl.xsd (particularly EANCOM D.96A ORDERS amd INVOICES);
2) populate smooks-config.xml with <jb:bean> entries to maps SAX element/events to java classes.

If  I understand correctly... please could you point me to the correct documentation to implement the two points seen above?

Thanks.

 

You are more than welcome to edit the agenda and join the community call this Thursday. We can discuss and offer guidance. Like most flexible frameworks, there's more than one way to skin a cat in Smooks.

Claude

On Mon, Dec 16, 2024 at 3:23 PM giova....@gmail.com <giova....@gmail.com> wrote:

Hi Dear,

Anyone who could point me in the right direction?

Thanks.

Il giorno giovedì 12 dicembre 2024 alle 15:14:36 UTC+1 giova....@gmail.com ha scritto:

Hi Dears,
this is my first experince relative to EDIFACT and Smooks framework.
I need to make a simple gateway to import/export EDIFACT messages (ORDERS & INVOICE release D.96A) to our ERP software.
Reading docs and example seems that is possible to import and EDI file and mapping segment and fields (through available cartridge ) to JAVA object and Bean allowing to perform task to this objects.

edifact-to-* examples are very poor because limited to obtaing only a Interchange object and that does not allow to really undestand how mapping to POJO works.

I'm seeking for a sample that import EDIFACT file for example ORDERS and get OrderList segment mapping them to java object to allow to easily get order item and relative field like item code, quantity price and so on.

Viceversa, starting from a java object (for example orderList) to populate edifact segments.

Some examples like edi-to-* seems to implement something similar and are less "low-level" rather other examples.

I'm very confused and I need some less low-level examples that can help me understand the functioning and potential of the framework.

Thanks for you help.

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/smooks-user/196970cc-48ef-4121-9945-33f6d60608dfn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CADvAAugSdN7ZvA3EZuu3aPLmAnq4cSqRGPR2ReNVxTmoBqi8bw%40mail.gmail.com.

[Giovanni Licata]

In other words... is It possible mapping and binding with the Java Bean Cartridge starting from the default packaged sax mapping

<edifact:parser schemaUri="/d96a/EDIFACT-Messages.dfdl.xsd">

Thanks

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CADvAAugSdN7ZvA3EZuu3aPLmAnq4cSqRGPR2ReNVxTmoBqi8bw%40mail.gmail.com.

[Claude Mamo]

That's right, readers like edi:parser are orthogonal to visitors such as jb:bean. Meaning that you can swap out edi:parser with edifact:parser and you will obtain the same behavior from Smooks.

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CAF1itZ9kMO3%3DUPO2zVDVC3ACOtv9i2HbL8StYb6DWYW-rSnD6A%40mail.gmail.com.

[Giovanni Licata]

Hi Dear,

thanks for your clarification.

In your opinion, to create a simple transformation gateway from CSV->EDIFACT and from EDICAT->CSV (that does not require any data enrichment) is it better to create an intermediate JavaBean or is the optimal way to use the PIPELINE and REWRITE constructs?

I have already tried some examples and I noted that DFDL compilation time, especially in EDIFACT, is very high.

How does the cache engine work? The "Cache on disk" attribute is sufficient or have I to check schema hit/miss on cache via code?

Thanks

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CADvAAugwHEkaP%3DoCY%3DZxfQJsG-Qgs6TKKPXU4U420xDouRz19Q%40mail.gmail.com.

[Claude Mamo]

In your opinion, to create a simple transformation gateway from CSV->EDIFACT and from EDICAT->CSV (that does not require any data enrichment) is it better to create an intermediate JavaBean or is the optimal way to use the PIPELINE and REWRITE constructs?

 

It's hard to say, the devil is in the details ;-). However, I reckon that the simplest approach is reading the input using csv:reader and then rewriting the events so that you feed them to edifact:unparser. The config could be something along these lines:

<?xml version="1.0"?>
<smooks-resource-list
    xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"
    xmlns:csv="https://www.smooks.org/xsd/smooks/csv-1.7.xsd"
    xmlns:edifact="https://www.smooks.org/xsd/smooks/edifact-2.0.xsd"
    xmlns:core="https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd">

    <csv:reader fields="..." rootElementName="MyRoot" recordElementName="..."/>

    <core:smooks filterSourceOn="/MyRoot">
       <core:action>
          <core:inline>
             <core:replace/>
          </core:inline>
       </core:action>
       <core:config>
          <smooks-resource-list>
             <core:rewrite>
                <core:rewrite>
                   <ftl:freemarker applyOnElement="#document">
                      <ftl:template>my-edifact-message.xml.ftl</ftl:template>
                   </ftl:freemarker>
                </core:rewrite>
             </core:rewrite>
             <edifact:unparser schemaUri="/d96a/EDIFACT-Messages.dfdl.xsd" unparseOnNode="*">
                <edifact:messageTypes>
                   <edifact:messageType>...</edifact:messageType>
                </edifact:messageTypes>
             </edifact:unparser>
          </smooks-resource-list>
       </core:config>
    </core:smooks>
</smooks-resource-list>

I have already tried some examples and I noted that DFDL compilation time, especially in EDIFACT, is very high.

This should only happen once, on the first execution. Subsequent executions should be hitting the cached schema. Can you confirm? Keep in mind that compilation time is proportional to the number of messageTypes you have configured in the edifact:(un)parser resource.

How does the cache engine work? The "Cache on disk" attribute is sufficient or have I to check schema hit/miss on cache via code?

Should be documented in https://www.smooks.org/documentation/#edifact . You don't need to worry about caching unless you want to cache on disk as well. Caching on disk is useful when you want to re-use the cache after the process has exited. You just need to enable it, the rest is transparent.

Claude

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CAF1itZ9rnyD___fz_o93RDaL_NaHi76OBzyfTGuzg7kxDUXT%2BQ%40mail.gmail.com.

[Giovanni Licata]

Hi, 

perfect. 

Thanks. 

You received this message because you are subscribed to a topic in the Google Groups "Smooks Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/smooks-user/EW4EkYhUzpY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to smooks-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CADvAAuiYfuWBnBMNGOr65gDjy%3DYWPuSRyM2p8WMpPPeLd0xR%2BA%40mail.gmail.com.

[Giovanni Licata]

Hi Dear, 

regarding the transformation from edifact to csv I would like to know if there is a csv unparser/writer or the file must be written using freemarker and if it is necessary pass through JavaBean. 

Thanks

[Claude Mamo]

Yeah, currently the guidance is to write CSV using {https://www.smooks.org/xsd/smooks/freemarker-2.1.xsd}:freemarker or {https://www.smooks.org/xsd/smooks/dfdl-1.0.xsd}:unparser. Far from ideal since CSV is a common format but hopefully someone from the community can contribute a CSV writer. We should propose it in the dev mailing list. Another option is to implement you own visitor that writes out the CSV.

the file must be written using freemarker and if it is necessary pass through JavaBean.

In your case, it might be easier to write the CSV using dfdl:unparser. You can just feed the events directly to dfdl:unparser from edifact:parser without requiring any intermediate rewrites. So something like this:

<smooks-resource-list xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"
                      xmlns:core="https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd"
                      xmlns:edifact="https://www.smooks.org/xsd/smooks/edifact-2.0.xsd"
                      xmlns:dfdl="https://www.smooks.org/xsd/smooks/dfdl-1.0.xsd">

    <edifact:parser schemaUri="/d03b/EDIFACT-Messages.dfdl.xsd">
        <edifact:messageTypes>
            <edifact:messageType>PAXLST</edifact:messageType>
        </edifact:messageTypes>
    </edifact:parser>

    <core:smooks filterSourceOn="#document">

        <core:action>
            <core:inline>
                <core:replace/>
            </core:inline>
        </core:action>
        <core:config>
            <smooks-resource-list>

                <dfdl:unparser schemaUri="/csv.dfdl.xsd" unparseOnNode="*"/>

            </smooks-resource-list>
        </core:config>
    </core:smooks>

</smooks-resource-list>

It really depends on how difficult the mapping between the target (i.e., CSV) and the source (i.e., EDIFACT) is. Here is an example of a CSV DFDL schema: https://github.com/smooks/smooks-examples/blob/v5/pipelines/src/main/resources/csv.dfdl.xsd . Keep in mind that DFDL is not a mapping language by design. If the mapping gets hairy (e.g., some segments need to be ignored), I'd suggest you resort to an intermediate mapping step such as core:rewrite (you could use ftl:freemarker to implement the rewrite) before feeding the events to dfdl:unparser.

Claude

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CAF1itZ_NY0zLRStz9_BacSUdEwbtJZt6SgA%3DLGzMn42n5gPkSg%40mail.gmail.com.

[Giovanni Licata]

Hi Dear, 

Il lun 23 dic 2024, 19:23 Claude Mamo <claud...@gmail.com> ha scritto:

Yeah, currently the guidance is to write CSV using {https://www.smooks.org/xsd/smooks/freemarker-2.1.xsd}:freemarker or {https://www.smooks.org/xsd/smooks/dfdl-1.0.xsd}:unparser. Far from ideal since CSV is a common format but hopefully someone from the community can contribute a CSV writer. We should propose it in the dev mailing list. Another option is to implement you own visitor that writes out the CSV.

Understand

I don't know if you know edifact d96a ORDERS message but it Is more complex than edi paxlist and generally not all segments must be used. 

I'm trying to get bean to Java step by step to take confidence with message segment in order to understand the convenient way. I think that the easy way is to fill a CSV using freemarker template from Java bean object but I'm not really sure. 

Now I have a problem because I cant get a value ... maybe something wrong in the cartdrige. I should debug... 

Thanks. 

Merry Christmas. 

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/CADvAAuiHsqhfxDbEBTFZqqXgm8KS_tpG7Z9dkQW4BCQpc1sPiw%40mail.gmail.com.

[Claude]

Could you mask the sensitive data and give samples of the EDIFACT input as well as the desired CSV output? Perhaps I can flesh out a quick example and add it to the catalogue.

Claude

[Giovanni Licata]

Hi Dear,
thanks for your support.
Let me take another moment to understand better the input format... and I think mapping it to Java Bean is the best lab (I'll send you smooks-config with mapping and jb:bean models at the end)
After getting familiar with the file format... I'll check if it makes sense to think of the csv as a transformation because the ORDERS message is too heterogeneous to think of it as a list of records (the order in fact includes not only the list of items but also delivery dates, delivery addresses, payment terms etc. etc. etc.).
For this reason, maybe the right idea would be to think of a pipeline that translates the SAX D96A stream (very complex to import) into a more readable XML and/or JSON that can be easily imported by third-party software (I wanted to do something as generic as possible). For example, the edifact-to-xml example results in an XML file that is very difficult to import because the third-party software needs to know the meaning of the edifact segments.

While the edi-to-xml example produces a user-friendly XML that allows any third-party software to import orders from this file (without need to know segment name HDR ORD ...).

Maybe I should use a modified version of EDIFACT-Messages.dfdl.xsd for edifact:parser or should I abandon the idea of using edifact:parser and use edi:parser and try to modify the mapping edi-to-xml-order-mapping.dfdl.xsd? And above all, how should I handle the reverse situation, i.e. switching from XML to edifact? Following the xml-to-edi example? By using edi:parser would I lose the whole issue of field validation that perhaps is handled in edifact:parser?

Also, in the middle of reading/generating this XML file (let's call it import file) the edifact files should be enriched with segments and data specific to the transmission that would not make sense to have them managed directly by third-party software.

Another idea could be to populate the edifact file via ftl:freemarker starting from the SAX XML (as is done in the pipeline example but there you start from a csv whose stream is very simple to manage in the subsequent steps).

As you can see, I'm still very confused, so I hope to take stock of the situation as soon as possible and answer you with clearer ideas. However, from my point of view, in my opinion the most correct idea is to map edifact to xml as edi-to-xml does without losing the functionality of the edifact parser.

This is an example file https://developer.kramp.com/edi-edifact-d96a-purchase-order and from the editor https://www.mescedia.org/tools/editor you can see the structure and meaning of the valued fields.

Thank you so much.