Edi writer configuration programaticaly by Java code

[Ivan Metla]

Hi guys, I'm trying to configure edi unparser by code, but can't find 'unparseOnElement' on the EdiReaderConfigurator class. Can configure edi unparser programmatically? 

SMOOKS version 2.0.0-M2

My xml configuration:

<?xml version="1.0"?>
<smooks-resource-list
xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"
xmlns:edi="https://www.smooks.org/xsd/smooks/edi-2.0.xsd">

<!--
Configure the EDI Writer
-->
<edi:unparser unparseOnElement="/Order" schemaURI="/edi/850/mapping.dfdl.xsd"
segmentTerminator="~%NL;" dataElementSeparator="*" validationMode="Full"/>
</smooks-resource-list>

My java code instead of that XML configuration:

final var ediReaderConfigurator = new EdiReaderConfigurator("/edi/850/mapping.dfdl.xsd");
ediReaderConfigurator.setDataElementSeparator("*");
ediReaderConfigurator.setSegmentTerminator("~%NL;");
ediReaderConfigurator.setValidationMode(ValidationMode.Full);
smooks.setReaderConfig(ediReaderConfigurator);

[Claude Mamo]

EdiReaderConfigurator is a convenience class for configuring edi:parser which is a reader. edi:unparser is a visitor so you would need to instantiate and configure DfdlUnparser. In 2.0.0-M3, the code for programmatically configuring an edi:unparser would look something like this:

Smooks smooks = new Smooks();

DfdlSchema dfdlSchema = new DfdlSchema(new URI("/edi/850/mapping.dfdl.xsd"), new HashMap<String, String>() {{
    this.put("segmentTerminator", "~%NL;");
    this.put("dataElementSeparator", "*");
    this.put("compositeDataElementSeparator", ":");
    this.put("escapeCharacter", "?");
    this.put("repetitionSeparator", "*");
    this.put("decimalSign", ".");
    this.put("triadSeparator", ",");
}}, ValidationMode.Full, false, false, null);
DfdlUnparser dfdlUnparser = new DfdlUnparser(dfdlSchema.compile());

// this is required because of a bug in DfdlUnparser
ResourceConfig resourceConfig = new DefaultResourceConfig();
resourceConfig.setParameter("schemaURI", "");
smooks.getApplicationContext().getRegistry().lookup(new LifecycleManagerLookup()).applyPhase(dfdlUnparser, new PostConstructLifecyclePhase(new Scope(smooks.getApplicationContext().getRegistry(), resourceConfig, dfdlUnparser)));
///////////////////////////////////////////////////

smooks.addVisitor(dfdlUnparser, "/Order");

Claude

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/f84eeb00-db1e-4360-9001-977c2951608cn%40googlegroups.com.

[Ivan Metla]

Thank you a lot for quick answere. 

I used your example provided before https://github.com/claudemamo/smooks-xml-to-edi

I updated smooks-edi-cartridge to 2.0.0-M3 and change Main#runSmooksTransform.java file to:

    protected static String runSmooksTransform() throws Throwable {
        Locale defaultLocale = Locale.getDefault();
        Locale.setDefault(new Locale("en", "IE"));

        Smooks smooks = new Smooks();

        DfdlSchema dfdlSchema = new DfdlSchema(new URI("/edi-to-xml-mapping.dfdl.xsd"),

            new HashMap<String, String>() {{

                this.put("segmentTerminator", "%NL;");
                this.put("dataElementSeparator", "+");
                this.put("compositeDataElementSeparator", "^");

                this.put("escapeCharacter", "?");
                this.put("repetitionSeparator", "*");
                this.put("decimalSign", ".");
                this.put("triadSeparator", ",");
            }}, ValidationMode.Full, false, false, null);
        DfdlUnparser dfdlUnparser = new DfdlUnparser(dfdlSchema.compile());

// this is required because of a bug in DfdlUnparser
        ResourceConfig resourceConfig = new DefaultResourceConfig();
        resourceConfig.setParameter("schemaURI", "");
        smooks.getApplicationContext().getRegistry().lookup(new LifecycleManagerLookup())
            .applyPhase(dfdlUnparser, new PostConstructLifecyclePhase(
                new Scope(smooks.getApplicationContext().getRegistry(), resourceConfig,
                    dfdlUnparser)));
///////////////////////////////////////////////////

        smooks.addVisitor(dfdlUnparser, "/Order");

        try {
            StringResult result = new StringResult();
            // Filter the input message to the outputWriter, using the execution context...
            smooks.filterSource(new StreamSource(new ByteArrayInputStream(messageIn)), result);

            Locale.setDefault(defaultLocale);
            return result.getResult();
        } finally {
            smooks.close();
        }
    }

But received an error:

Exception in thread "main" Schema Definition Error: unknown variable decimalSign
Schema context: element reference {}Order Location line 2 in file:/home/ivan/projects/smooks-xml-to-edi/target/classes/edi-to-xml-mapping.dfdl.xsd
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE(SDE.scala:179)
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$(SDE.scala:178)
at org.apache.daffodil.processors.TermRuntimeData.SDE(RuntimeData.scala:101)
at org.apache.daffodil.processors.SchemaSetRuntimeData.SDE(SchemaSetRuntimeData.scala:45)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError(ThrowsSDE.scala:58)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$(ThrowsSDE.scala:58)
at org.apache.daffodil.processors.SchemaSetRuntimeData.schemaDefinitionError(SchemaSetRuntimeData.scala:26)
at org.apache.daffodil.processors.VariableMap.setExtVariable(VariableMap1.scala:334)
at org.apache.daffodil.processors.VariableUtils$.$anonfun$setExternalVariables$1(VariableMap1.scala:106)
at org.apache.daffodil.processors.VariableUtils$.$anonfun$setExternalVariables$1$adapted(VariableMap1.scala:106)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at org.apache.daffodil.processors.VariableUtils$.setExternalVariables(VariableMap1.scala:106)
at org.apache.daffodil.externalvars.ExternalVariablesLoader$.loadVariables(ExternalVariablesLoader.scala:46)
at org.apache.daffodil.processors.DataProcessor.loadExternalVariables(DataProcessor.scala:277)
at org.apache.daffodil.processors.DataProcessor.withExternalVariables(DataProcessor.scala:301)
at org.apache.daffodil.japi.DataProcessor.withExternalVariables(Daffodil.scala:665)
at org.smooks.cartridges.dfdl.DfdlSchema.compile(DfdlSchema.java:124)
at org.smooks.examples.xml2edi.Main.runSmooksTransform(Main.java:98)
at org.smooks.examples.xml2edi.Main.main(Main.java:128)

Process finished with exit code 1

--

Regards,

Ivan Metla

AgileVision sp. z o.o.

Software Development Services

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAuiZv3mpLtDMKcyb18xbqBM5qum3CnOU0EGzcq0tC6KKSg%40mail.gmail.com.

[Claude Mamo]

Oh yeah, the variable names needs to be prefixed with their namespaces. Also, I got the variable names wrong. You can find the correct names here: https://github.com/smooks/smooks-edi-cartridge/blob/v2.0.0-M3/edi-cartridge/src/main/java/org/smooks/cartridges/edi/EdiDataProcessorFactory.java#L69-L75

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CABuL3Dh5nDAkoKEUQEHwzcnLAi4F_FfntX_G6ZN%2BH4SpY0Cqdg%40mail.gmail.com.

[Ivan Metla]

Thanks a lot, it is working on the version 2.0.0-M2.

But on the version 2.0.0-M3 i received an exception:

Caused by: org.apache.daffodil.japi.DaffodilUnparseErrorSAXException: Unparse Error: Expected element start event for {}header, but received element end event for {}Order.
Data location was preceding byte 0
at org.apache.daffodil.japi.DaffodilUnparseContentHandler.endDocument(Daffodil.scala:1044)
at org.smooks.cartridges.dfdl.unparser.DfdlUnparser.visitAfter(DfdlUnparser.java:99)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:165)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:162)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.EventInterceptor.visitAfter(EventInterceptor.java:187)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:165)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:162)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.TextConsumerInterceptor.visitAfter(TextConsumerInterceptor.java:91)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:165)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:162)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.StreamResultWriterInterceptor.lambda$intercept$0(StreamResultWriterInterceptor.java:118)
at org.smooks.engine.memento.DefaultMementoCaretaker.stash(DefaultMementoCaretaker.java:107)
at org.smooks.engine.delivery.interceptor.StreamResultWriterInterceptor.intercept(StreamResultWriterInterceptor.java:111)
at org.smooks.engine.delivery.interceptor.StreamResultWriterInterceptor.visitAfter(StreamResultWriterInterceptor.java:76)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:165)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitAfterInvocation.invoke(AbstractInterceptorVisitor.java:162)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.ExceptionInterceptor.intercept(ExceptionInterceptor.java:198)
... 18 more

It is looks like you write before "The DFDL and EDI unparsers in the 2.0.0-M3 release will operate slightly differently to take advantage of the new streaming capabilities in Apache Daffodil 3." 

I update 'smooks-config.xml' like in the 'java-to-edifact' example. 

How can I write 'Inline replace' by code? How can I configure 'Pipeline' programmatically? 

The working 'smooks-config.xml' to Convert XML to EDI;

<smooks-resource-list xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"
  xmlns:edi="https://www.smooks.org/xsd/smooks/edi-2.0.xsd"

  xmlns:core="https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd">

  <core:smooks filterSourceOn="/Order">
    <core:action>
      <core:inline>
        <core:replace/>
      </core:inline>
    </core:action>
    <core:config>
      <smooks-resource-list>
        <edi:unparser unparseOnElement="*" schemaURI="/edi-to-xml-mapping.dfdl.xsd"
          segmentTerminator="%NL;" compositeDataElementSeparator="^" debugging="true"/>
      </smooks-resource-list>
    </core:config>
  </core:smooks>
</smooks-resource-list>

The working example  on version 2.0.0-M2 to configure convert XML to EDI programaticaly:

    protected static String runSmooksTransform() throws Throwable {
        Locale defaultLocale = Locale.getDefault();
        Locale.setDefault(new Locale("en", "IE"));

        Smooks smooks = new Smooks();

        DfdlSchema dfdlSchema = new DfdlSchema(new URI("/edi-to-xml-mapping.dfdl.xsd"),
            new HashMap<String, String>() {{

                this.put("{http://www.ibm.com/dfdl/EDI/Format}SegmentTerm", "%NL;");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}FieldSep", "+");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}CompositeSep", "^");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}EscapeChar", "?");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}RepeatSep", "*");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}DecimalSep", ".");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}GroupingSep", ",");
            }}, ValidationMode.Full, false, false);

        DfdlUnparser dfdlUnparser = new DfdlUnparser(dfdlSchema.compile());

// this is required because of a bug in DfdlUnparser

        ResourceConfig resourceConfig = new ResourceConfig();

        resourceConfig.setParameter("schemaURI", "");
        smooks.getApplicationContext().getRegistry().lookup(new LifecycleManagerLookup())
            .applyPhase(dfdlUnparser, new PostConstructLifecyclePhase(
                new Scope(smooks.getApplicationContext().getRegistry(), resourceConfig,
                    dfdlUnparser)));
///////////////////////////////////////////////////

        smooks.addVisitor(dfdlUnparser, "/Order");

        try {
            StringResult result = new StringResult();
            // Filter the input message to the outputWriter, using the execution context...
            smooks.filterSource(new StreamSource(new ByteArrayInputStream(messageIn)), result);

            Locale.setDefault(defaultLocale);
            return result.getResult();
        } finally {
            smooks.close();
        }
    }

--

Regards,

Ivan Metla

AgileVision sp. z o.o.

Software Development Services

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAuhFN7HSknJDsYiXBc485Ks83YXgfD0jHcrNO6fO8Qs0Zw%40mail.gmail.com.

[Claude Mamo]

The error happens on M3 because edi:unparser is now expecting a stream of partial nodes rather than a full node tree as in M2. Setting the selector to "*" will allow edi:unparser to visit each partial node.

How can I write 'Inline replace' by code? How can I configure 'Pipeline' programmatically?

You don't need to configure a pipeline in your application. A pipeline is used in the java-to-edifact example to replace the default output but it would be easier for you to just disable the default output with:

smooks.setFilterSettings(FilterSettings.newSaxNgSettings().setDefaultSerializationOn(false));

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CABuL3Dj53DdX0GDfU67HFgFNvjXCB3BKJW5rn2OC5AoCWwAKeg%40mail.gmail.com.

[Ivan Metla]

Awesome. It is working now.

The working example  on version 2.0.0-M3 to configure convert XML to EDI programmatically:

    protected static String runSmooksTransform() throws Throwable {
        Locale defaultLocale = Locale.getDefault();
        Locale.setDefault(new Locale("en", "IE"));

        Smooks smooks = new Smooks();

        DfdlSchema dfdlSchema = new DfdlSchema(new URI("/edi-to-xml-mapping.dfdl.xsd"),
            new HashMap<String, String>() {{
                this.put("{http://www.ibm.com/dfdl/EDI/Format}SegmentTerm", "%NL;");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}FieldSep", "+");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}CompositeSep", "^");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}EscapeChar", "?");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}RepeatSep", "*");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}DecimalSep", ".");
                this.put("{http://www.ibm.com/dfdl/EDI/Format}GroupingSep", ",");

            }}, ValidationMode.Full, false, false, null);
        DfdlUnparser dfdlUnparser = new DfdlUnparser(dfdlSchema.compile());

// this is required because of a bug in DfdlUnparser
        ResourceConfig resourceConfig = new DefaultResourceConfig();
        resourceConfig.setParameter("schemaURI", "");
        smooks.getApplicationContext().getRegistry().lookup(new LifecycleManagerLookup())
            .applyPhase(dfdlUnparser, new PostConstructLifecyclePhase(
                new Scope(smooks.getApplicationContext().getRegistry(), resourceConfig,
                    dfdlUnparser)));
///////////////////////////////////////////////////

        smooks.addVisitor(dfdlUnparser, "*");
        smooks.setFilterSettings(FilterSettings.newSaxNgSettings().setDefaultSerializationOn(false));

        try {
            StringResult result = new StringResult();
            // Filter the input message to the outputWriter, using the execution context...
            smooks.filterSource(new StreamSource(new ByteArrayInputStream(messageIn)), result);

            Locale.setDefault(defaultLocale);
            return result.getResult();
        } finally {
            smooks.close();
        }
    }

--

Regards,

Ivan Metla

AgileVision sp. z o.o.

Software Development Services

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujAs0Yi6UsMNkJA%3DzG_Cph5R80nyVZF%2BcK2r7NCi0jo3w%40mail.gmail.com.

[Claude Mamo]

Out of curiosity, why are you setting the locale before running Smooks?

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CABuL3DjUmJ4N_57OpVMuR1qYr_Qh4ewOpq%3Dpw81ro88Lcdru4A%40mail.gmail.com.

[Ivan Metla]

I modified a code of the example https://github.com/smooks/smooks-examples/blob/master/edi-to-xml/src/main/java/org/smooks/examples/edi2xml/Main.java to test approach.

--

Regards,

Ivan Metla

AgileVision sp. z o.o.

Software Development Services

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujTjQfncCdPv%2B3f_g6FO6nUxW1qk%2Bt_gubK-GMCXJynjA%40mail.gmail.com.

[Claude Mamo]

Yeah, that part might not be needed. I'll probably remove it from the example.

Claude.

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CABuL3Dijgo2Ghhihs0GGSsPJkrUWrGYzH53iMpq4%2BwSHsmmLnw%40mail.gmail.com.

[ime...@agilevision.io]

Yes. When first seen the example, I thought so but decided to leave it to be sure the problem, not on locale.