Multiple migration problems

[Martin]

Hi everyone,

we try to migrate an application using smooks 1.7.0 to 2.0.0-RC3. This app runs in the Payara application server and uses now camel 3.22.0. Unfortunately we run into several problems.

Our dependencies are as follows:

• smooks-bom (as pom import)
• smooks-core
• smooks-camel-cartridge
• smooks-rules-cartridge
• smooks-commons
• smooks-edi-cartridge
• smooks-scripting-cartridge
• smooks-edifact-cartridge
• edi-schemas
• edifact-schemas (classifier: d96a)
• d96a-edifact-binding
• edifact-schemas (classifier: d01b)
• d01b-edifact-binding

We have several camel routes which each initialise a SmooksDataFormat (our SmooksConverterRoutes). Therefor we have multiple SmooksDataFormats running.

public class SmooksConverterRoute extends RouteBuilder {
...
@Override
public void configure() throws Exception {
SmooksDataFormat smooks;
try {
getContext().getRegistry().bind(SmooksFactory.class.getName(), new SmooksProducer());

smooks = new SmooksDataFormat(getSmooksConfigFile());
smooks.setCamelContext(getContext());
smooks.start();
} catch (Exception e) {
String route = getInConsumer() + ":" + getOutProducer();
logger.error("Exception when creating or starting smooks context with route " + route + " with config file: " + getSmooksConfigFile(), e);
return;
}
from(getInConsumer()).routeId(getRouteId(getInConsumer())).convertBodyTo(String.class)
.unmarshal(smooks)
.process(smooksProcessor)
.to(getOutProducer());
}
...
}

As one can see we use a SmooksProducer which is a SmooksFactory. Which is implemented as follows
public class SmooksProducer implements SmooksFactory {
    @Override
    public Smooks createInstance() {
        throw new UnsupportedOperationException("Not supported yet.");
    }

    @Override
    public Smooks createInstance(InputStream config) throws IOException, SAXException {
        throw new UnsupportedOperationException("Not supported yet.");
    }

    @Override
    public Smooks createInstance(String config) throws IOException, SAXException {
        URIResourceLocator resourceLocator = new URIResourceLocator();
        resourceLocator.setBaseURI(URIResourceLocator.extractBaseURI(config));
        ApplicationContext applicationContext = new DefaultApplicationContextBuilder()
            .setClassLoader(this.getClass().getClassLoader())
            .setResourceLocator(resourceLocator).build();

        Smooks smooks = new Smooks(applicationContext);
        addTypeConverterFactories(smooks.getApplicationContext());

        smooks.addConfigurations(config);
        return smooks;
    }

    private void addTypeConverterFactories(ApplicationContext applicationContext) {
        Set<TypeConverterFactory<?, ?>> converterReg = applicationContext.getRegistry().lookup(TypeConverterFactoryLookup.TYPE_CONVERTER_FACTORY_REGISTRY_KEY);
        converterReg.add(new DateDecoderFactory());
        converterReg.add(new FileNameDecoder());
    }
}

We also have a couple of custom TypeConverterFactories which we register at runtime (because using the file registration using the file org.smooks.api.converter.TypeConverterFactory does not work for unknown reasons).

When a smooks config references a confgureable TypeConverterFactory as in
       <jb:value property="date" data="DueDateDateTime/DateTimeString" decoder="de.app.edi.process.decoder.DateDecoderFactory" >
          <jb:decodeParam name="allowEmpty">true</jb:decodeParam>
       </jb:value>
an exception occurs when camel tries to configure the route.
org.smooks.api.SmooksException: Failed to apply processing unit [org.smooks.cartridges.javabean.ext.DecodeParamResolver] to [org:smooks:unknowndoc:/smooks-resource-list/jb:bean[4]/jb:value[2]].
at org.smooks.engine.delivery.dom.SmooksDOMFilter.processVisitorException(SmooksDOMFilter.java:885)
at org.smooks.engine.delivery.dom.SmooksDOMFilter.access$700(SmooksDOMFilter.java:181)
at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.processMapping(SmooksDOMFilter.java:827)
at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.process(SmooksDOMFilter.java:774)
at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.access$000(SmooksDOMFilter.java:732)
at org.smooks.engine.delivery.dom.SmooksDOMFilter.filter(SmooksDOMFilter.java:457)
at org.smooks.engine.delivery.dom.SmooksDOMFilter.doFilter(SmooksDOMFilter.java:291)
at org.smooks.engine.delivery.dom.SmooksDOMFilter.doFilter(SmooksDOMFilter.java:269)
at org.smooks.Smooks._filter(Smooks.java:546)
at org.smooks.Smooks.filterSource(Smooks.java:499)
at org.smooks.engine.resource.config.XMLConfigDigester.digestExtendedResourceConfig(XMLConfigDigester.java:533)
at org.smooks.engine.resource.config.XMLConfigDigester.digestV20XSDValidatedConfig(XMLConfigDigester.java:328)
at org.smooks.engine.resource.config.XMLConfigDigester.digestConfigRecursively(XMLConfigDigester.java:243)
at org.smooks.engine.resource.config.XMLConfigDigester.digestConfig(XMLConfigDigester.java:210)
at org.smooks.engine.DefaultRegistry.registerResources(DefaultRegistry.java:197)
at org.smooks.Smooks.addConfigurations(Smooks.java:368)
at org.smooks.Smooks.addConfigurations(Smooks.java:340)
at de.app.edi.process.routes.invoice.SmooksProducer.createInstance(SmooksProducer.java:42)
at org.smooks.cartridges.camel.dataformat.SmooksDataFormat.start(SmooksDataFormat.java:155)
at de.app.edi.process.routes.SmooksConverterRoute.configure(SmooksConverterRoute.java:42)
at org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:723)
at org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:668)
at org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:600)
at org.apache.camel.impl.engine.AbstractCamelContext.addRoutes(AbstractCamelContext.java:1213)
...
Caused by: java.lang.NullPointerException: Cannot invoke "org.smooks.api.converter.TypeConverterFactory.createTypeConverter()" because "typeConverterFactory" is null
at org.smooks.cartridges.javabean.ext.DecodeParamResolver.visitBefore(DecodeParamResolver.java:98)
at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.processMapping(SmooksDOMFilter.java:821)
... 109 common frames omitted

Debugging showed that although we initialise our ApplicationContext with the TypeConverterFactories, Smooks generates a new one (in XMLConfigDigester.digestExtendedResourceConfig()) which does not know about our registered TypeConverterFactories. How can we prevent this or tell smooks to use our application context we created in the SmooksProducer?

When routes do not have configurable TypeConverterFactories camel can configure the routes without problems. But when we try to process a file, smooks will run into a problem when there is an invocation of ANY TypeConverterFactories.

13:31:15.813| ERROR o.s.e.delivery.DefaultContentDeliveryConfigBuilder - ContentHandlerFactory [org.smooks.engine.delivery.JavaContentHandlerFactory] unable to create resource processing instance for resource [Target Profile: [[org.smooks.api.profile.Profile#default_profile]], Selector: [decoder:_684ee0f9-1032-4261-b94f-aefa5923f32d], Selector Namespace URI: [null], Resource: [org.smooks.engine.converter.StringToBigDecimalConverterFactory$1], Num Params: [1]]. Failed to create an instance of Java ContentHandler [org.smooks.engine.converter.StringToBigDecimalConverterFactory$1].  See exception cause...
org.smooks.api.SmooksException: Failed to create an instance of Java ContentHandler [org.smooks.engine.converter.StringToBigDecimalConverterFactory$1].  See exception cause...
at org.smooks.engine.delivery.JavaContentHandlerFactory.create(JavaContentHandlerFactory.java:97)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.addContentDeliveryUnit(DefaultContentDeliveryConfigBuilder.java:442)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyContentDeliveryUnitStrategy(DefaultContentDeliveryConfigBuilder.java:386)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyStrategy(DefaultContentDeliveryConfigBuilder.java:373)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.iterate(DefaultContentDeliveryConfigBuilder.java:524)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.access$200(DefaultContentDeliveryConfigBuilder.java:504)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.extractContentHandlers(DefaultContentDeliveryConfigBuilder.java:346)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.load(DefaultContentDeliveryConfigBuilder.java:238)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.build(DefaultContentDeliveryConfigBuilder.java:140)
at org.smooks.engine.delivery.DefaultContentDeliveryRuntimeFactory.create(DefaultContentDeliveryRuntimeFactory.java:86)
at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:117)
at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:94)
at org.smooks.Smooks.createExecutionContext(Smooks.java:438)
at org.smooks.Smooks.createExecutionContext(Smooks.java:404)
at org.smooks.cartridges.camel.dataformat.SmooksDataFormat.unmarshal(SmooksDataFormat.java:127)
...
Caused by: java.lang.InstantiationException: org.smooks.engine.converter.StringToBigDecimalConverterFactory$1
at java.base/java.lang.Class.newInstance(Class.java:639)
at org.smooks.engine.delivery.JavaContentHandlerFactory.create(JavaContentHandlerFactory.java:92)
... 103 common frames omitted
Caused by: java.lang.NoSuchMethodException: org.smooks.engine.converter.StringToBigDecimalConverterFactory$1.<init>()
at java.base/java.lang.Class.getConstructor0(Class.java:3585)
at java.base/java.lang.Class.newInstance(Class.java:626)
... 104 common frames omitted

I am not quite sure, but this sounds like a dependency problem. As far as I know the build-in TypeConverterFactories are located in smooks-core (which we depend on).

Additionally we are facing issue 636 (https://github.com/smooks/smooks/issues/636) is there any progress or recommendation to handle the issue?

We would really appreciate if someone can help us to solve the problems as these are blocking issues for the complete migration.

Thank you and best regards,

Martin

[Claude Mamo]

We also have a couple of custom TypeConverterFactories which we register at runtime (because using the file registration using the file org.smooks.api.converter.TypeConverterFactory does not work for unknown reasons).

Usually this means that the application context doesn't have the right class loader set.

When a smooks config references a confgureable TypeConverterFactory as in
       <jb:value property="date" data="DueDateDateTime/DateTimeString" decoder="de.app.edi.process.decoder.DateDecoderFactory" >
          <jb:decodeParam name="allowEmpty">true</jb:decodeParam>
       </jb:value>
an exception occurs when camel tries to configure the route.

 

Can you post the code for DateDecoderFactory? Is the class annotated with @jakarta.annotation.Resource and the name element set to de.app.edi.process.decoder.DateDecoderFactory?

Additionally we are facing issue 636 (https://github.com/smooks/smooks/issues/636) is there any progress or recommendation to handle the issue?

Is this error just a warning? Can you attach to the issue a minimal, reproducible sample that reproduces the problem?

Claude

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/7642b659-1db8-45a6-8f2e-0013321b9051n%40googlegroups.com.

[mast....@gmail.com]

Hello Claude,

 

thank you for replying!

 

Regarding the right classloader.

Our SmooksProducer sets the classloader explicitly. When we don’t do it, the following exception is raised telling me, that probably the wrong classloader was used.

 

28.02.2024 15:45:50.610  [payara-executor-service-scheduled-task] ERROR de.app.edi.process.routes.SmooksConverterRoute - Exception when creating or starting smooks context with route direct:zgfrd-20-facturx10-in:direct:zgfrd-20-facturx10-validate with config file: /mappings/UNEDIFACT_UNECE_CrossIndustryInvoice_SmooksConfig.xml

org.smooks.api.SmooksException: Error processing resource file '/system-interceptors.xml'.

         at org.smooks.engine.resource.config.SystemResourceConfigListFactory.create(SystemResourceConfigListFactory.java:77)

         at org.smooks.engine.DefaultApplicationContextBuilder.registerSystemResources(DefaultApplicationContextBuilder.java:144)

         at org.smooks.engine.DefaultApplicationContextBuilder.build(DefaultApplicationContextBuilder.java:112)

         at de.app.edi.process.routes.invoice.SmooksProducer.createInstance(SmooksProducer.java:45)

at org.smooks.cartridges.camel.dataformat.SmooksDataFormat.start(SmooksDataFormat.java:155)

…

Caused by: org.smooks.api.SmooksException: No [org.smooks.api.delivery.ContentHandlerFactory] found for content of type 'class'. Hint: ensure the Smooks application context has the correct class loader set

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyContentDeliveryUnitStrategy(DefaultContentDeliveryConfigBuilder.java:383)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyStrategy(DefaultContentDeliveryConfigBuilder.java:373)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.iterate(DefaultContentDeliveryConfigBuilder.java:524)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.access$200(DefaultContentDeliveryConfigBuilder.java:504)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.extractContentHandlers(DefaultContentDeliveryConfigBuilder.java:346)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.load(DefaultContentDeliveryConfigBuilder.java:238)

          at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.build(DefaultContentDeliveryConfigBuilder.java:140)

          at org.smooks.engine.delivery.DefaultContentDeliveryRuntimeFactory.create(DefaultContentDeliveryRuntimeFactory.java:86)

          at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:117)

          at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:94)

          at org.smooks.Smooks.createExecutionContext(Smooks.java:446)

          at org.smooks.Smooks.createExecutionContext(Smooks.java:404)

          at org.smooks.engine.resource.config.XMLConfigDigester.digestExtendedResourceConfig(XMLConfigDigester.java:518)

          at org.smooks.engine.resource.config.XMLConfigDigester.digestV20XSDValidatedConfig(XMLConfigDigester.java:328)

          at org.smooks.engine.resource.config.XMLConfigDigester.digestConfigRecursively(XMLConfigDigester.java:243)

          at org.smooks.engine.resource.config.XMLConfigDigester.digestConfig(XMLConfigDigester.java:210)

          at org.smooks.engine.resource.config.SystemResourceConfigListFactory.create(SystemResourceConfigListFactory.java:69)

          ... 80 common frames omitted

 

This exception will not be raised when we set the classloader explicitly. Looking at the classloader we set in our SmooksProducer I can see all jar files that should be there, indicating that this is the “right” classloader. For instance, I can see the following jars

• \WEB-INF\lib\smooks-api-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-camel-cartridge-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-commons-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-core-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-dfdl-cartridge-1.0.0-RC3.jar
• \WEB-INF\lib\smooks-edi-cartridge-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-edifact-cartridge-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-javabean-cartridge-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-rules-cartridge-2.0.0-RC3.jar
• \WEB-INF\lib\smooks-scripting-cartridge-2.0.0-RC3.jar

 

I am not sure how to determine what the right classloader is or how to obtain it other than what we do. Do you have any suggestions?

 

The DateDecoderFactory indeed missed the Resource annotation. But adding it with the name element did not solve the problem. The same exception is still present.

For issue 636 I am not sure if I can provide a minimal example but yes, it’s a warning. For some reasons it is not happening anymore. I am not quite sure, but I am still investigating.

 

Best regards,

Martin

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujR75ZLE%3DJ1STFY%2B3tnKH66c3So395GsEfhO6MhY0Jb9A%40mail.gmail.com.

[Claude Mamo]

I might have missed this but what's exception stack trace when you (1) set the class loader AND (2) try to load your custom type converter factories from META-INF/services/org.smooks.api.converter.TypeConverterFactory?

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/006301da6a66%24c56d1600%2450474200%24%40gmail.com.

[mast....@gmail.com]

Hello Claude,

 

it turned out that we had a typo in org.smooks.api.converter.TypeConverterFactory. Now this works.

 

We also figured out the problem which caused the exception:

Caused by: java.lang.NoSuchMethodException: org.smooks.engine.converter.StringToBigDecimalConverterFactory$1.<init>()
at java.base/java.lang.Class.getConstructor0(Class.java:3585)
at java.base/java.lang.Class.newInstance(Class.java:626)
... 104 common frames omitted

 

In our smooks config file we used

<jb:value property="gross_value_tax" data="SpecifiedLineTradeSettlement/SpecifiedTradeSettlementLineMonetarySummation/GrossLineTotalAmount" decoder="BigDecimal">

<jb:decodeParam name="locale">en_US</jb:decodeParam>

</jb:value>

Turns out the decodeParam causes trouble in smooks 2. Is this intentionally not supported anymore? StringToBigDecimalConverterFactory uses a NumberTypeConverter internally which is a LocaleAwareTypeConverter.

 

But we encountered a new problem. When smooks uses our classpath the scanner takes a very long time scanning the whole classpath for each camel route configuration.

29.02.2024 17:20:13.726  [payara-executor-service-scheduled-task] INFO  de.app.edi.process.routes.invoice.PInvoiceRoute - configuring PInvoiceRoute, docsource: $simple{docsource}

29.02.2024 17:22:06.310  [payara-executor-service-scheduled-task] WARN  org.smooks.classpath.Scanner - java.nio.file.NoSuchFileException: C:\Payara\glassfish\domains\EDI\generated\edi-app. Unable to scan [C:\Payara\glassfish\domains\EDI\generated\edi-app] for Smooks resources

29.02.2024 17:22:41.539  [payara-executor-service-scheduled-task] WARN  org.smooks.classpath.Scanner - java.nio.file.NoSuchFileException: C:\Payara\glassfish\domains\EDI\generated\edi-app. Unable to scan [C:\Payara\glassfish\domains\EDI\generated\edi-app] for Smooks resources

29.02.2024 17:22:42.493  [payara-executor-service-scheduled-task] WARN  org.smooks.classpath.Scanner - java.nio.file.NoSuchFileException: C:\Payara\glassfish\domains\EDI\generated\edi-app. Unable to scan [C:\Payara\glassfish\domains\EDI\generated\edi-app] for Smooks resources

29.02.2024 17:22:43.406  [payara-executor-service-scheduled-task] WARN  org.smooks.classpath.Scanner - java.nio.file.NoSuchFileException: C:\Payara\glassfish\domains\EDI\generated\edi-app. Unable to scan [C:\Payara\glassfish\domains\EDI\generated\edi-app] for Smooks resources

29.02.2024 17:22:44.350  [payara-executor-service-scheduled-task] WARN  org.smooks.classpath.Scanner - java.nio.file.NoSuchFileException: C:\Payara\glassfish\domains\EDI\generated\edi-app. Unable to scan [C:\Payara\glassfish\domains\EDI\generated\edi-app] for Smooks resources

 

This is an example from one of our routes. It took almost 2min30sec to scan the classpath. The warnings are correct because the path does not exist which is also correct. How can we speed this up? We have a lot of routes which result in a startup time of about 10 min.

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAugMrKaW-E4fY2EapxgoJJ-LqewAP4bxA3F-fSVLFkxtig%40mail.gmail.com.

[Claude Mamo]

Great to see that this now works.

In our smooks config file we used

<jb:value property="gross_value_tax" data="SpecifiedLineTradeSettlement/SpecifiedTradeSettlementLineMonetarySummation/GrossLineTotalAmount" decoder="BigDecimal">

      <jb:decodeParam name="locale">en_US</jb:decodeParam>

</jb:value>

Turns out the decodeParam causes trouble in smooks 2. Is this intentionally not supported anymore? StringToBigDecimalConverterFactory uses a NumberTypeConverter internally which is a LocaleAwareTypeConverter.

Probably a bug. The locale specified in the config should be used. Can you create an issue?

This is an example from one of our routes. It took almost 2min30sec to scan the classpath. The warnings are correct because the path does not exist which is also correct. How can we speed this up? We have a lot of routes which result in a startup time of about 10 min.

I reviewed the v2.0.0-RC3 smooks-core code and something definitely weird is going on which wasn't happening in v1.7.1. This scanning behaviour was added at some point in v2.0.0-M2 but it doesn't make sense and it was likely a mistake. Fortunately, I already removed this behaviour in the latest snapshot as part of my earlier commits. Until the next release, you may want to try extending the Smooks application context and remove the problematic code highlighted here: https://github.com/smooks/smooks/blob/v2.0.0-RC3/core/src/main/java/org/smooks/engine/DefaultApplicationContext.java#L88-L95 . You should also extend DefaultApplicationContextBuilder in order to build your custom application context instead of DefaultApplicationContext. This might not work though because from the code I can see that DefaultApplicationContextBuilder is hard-coded in some places (which should not be the case!).

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/00d801da6b2e%244cbaa920%24e62ffb60%24%40gmail.com.

[mast....@gmail.com]

Hi Claude,

 

We tried to switch to the SNAPSHOT, since you said, that extending might not work properly and you already changed the behavior.

Therefor we included smooks-core and smooks-common as SNAPSHOT versions.

 

We changed our SmooksProducer to

 

    @Override

    public Smooks createInstance(String config) throws IOException, SAXException {

        URIResourceLocator resourceLocator = new URIResourceLocator();

        resourceLocator.setBaseURI(URIResourceLocator.extractBaseURI(config));

 

        ApplicationContext applicationContext = new DefaultApplicationContextBuilder()

            .withClassLoader(this.getClass().getClassLoader())

            .withResourceLocator(resourceLocator)

            .build();

       

        Smooks smooks = new Smooks(applicationContext);

        addTypeConverterFactories(smooks.getApplicationContext());

        smooks.addResourceConfigs(config);

        return smooks;

    }

 

The long startup time is gone. Unfortunately, this leads to the following exception when we try to start the route:

Caused by: org.apache.camel.RuntimeCamelException: org.smooks.api.SmooksException: Failed to apply processing unit [org.smooks.cartridges.javabean.MapToSelectorFromContextAwareAttribute] to [org:smooks:unknowndoc:/smooks-resource-list/jb:bean].

                at org.apache.camel.RuntimeCamelException.wrapRuntimeCamelException(RuntimeCamelException.java:51)

                at org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:67)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:130)

                at org.apache.camel.impl.engine.DefaultChannel.doStart(DefaultChannel.java:126)

                at org.apache.camel.support.service.BaseService.start(BaseService.java:119)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:116)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:130)

                at org.apache.camel.processor.Pipeline.doStart(Pipeline.java:207)

                at org.apache.camel.support.service.BaseService.start(BaseService.java:119)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)

                at org.apache.camel.support.processor.DelegateAsyncProcessor.doStart(DelegateAsyncProcessor.java:89)

                at org.apache.camel.support.service.BaseService.start(BaseService.java:119)

                at org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:113)

                at org.apache.camel.impl.engine.RouteService.startChildServices(RouteService.java:396)

                at org.apache.camel.impl.engine.RouteService.doWarmUp(RouteService.java:193)

                at org.apache.camel.impl.engine.RouteService.warmUp(RouteService.java:121)

                ... 64 common frames omitted

Caused by: org.smooks.api.SmooksException: Failed to apply processing unit [org.smooks.cartridges.javabean.MapToSelectorFromContextAwareAttribute] to [org:smooks:unknowndoc:/smooks-resource-list/jb:bean].

                at org.smooks.engine.delivery.dom.SmooksDOMFilter.processVisitorException(SmooksDOMFilter.java:889)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter.access$700(SmooksDOMFilter.java:186)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.processMapping(SmooksDOMFilter.java:831)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.process(SmooksDOMFilter.java:778)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter$ElementProcessor.access$000(SmooksDOMFilter.java:736)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter.filter(SmooksDOMFilter.java:466)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter.doFilter(SmooksDOMFilter.java:296)

                at org.smooks.engine.delivery.dom.SmooksDOMFilter.doFilter(SmooksDOMFilter.java:274)

                at org.smooks.Smooks._filter(Smooks.java:551)

                at org.smooks.Smooks.filterSource(Smooks.java:510)

                at org.smooks.engine.resource.config.loader.xml.XmlResourceConfigLoader.digestExtendedResourceConfig(XmlResourceConfigLoader.java:415)

                at org.smooks.engine.resource.config.loader.xml.XmlResourceConfigLoader.digestV20XSDValidatedConfig(XmlResourceConfigLoader.java:210)

                at org.smooks.engine.resource.config.loader.xml.XmlResourceConfigLoader.digestConfigRecursively(XmlResourceConfigLoader.java:167)

                at org.smooks.engine.resource.config.loader.xml.XmlResourceConfigLoader.load(XmlResourceConfigLoader.java:138)

                at org.smooks.engine.DefaultRegistry.registerResources(DefaultRegistry.java:179)

                at org.smooks.Smooks.addResourceConfigs(Smooks.java:379)

                at org.smooks.Smooks.addResourceConfigs(Smooks.java:351)

 

Our smooks config looks something like this:

<smooks-resource-list xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"

xmlns:core="https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd"

                xmlns:g="https://www.smooks.org/xsd/smooks/groovy-2.0.xsd"

xmlns:jb="https://www.smooks.org/xsd/smooks/javabean-1.6.xsd">

 

   <core:namespaces>

      <core:namespace prefix="rsm" uri="urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100" />

      <core:namespace prefix="xsi" uri="http://www.w3.org/2001/XMLSchema-instance" />

      <core:namespace prefix="ram" uri="urn:un:unece:uncefact:data:standard:ReusableAggregateBusinessInformationEntity:100" />

      <core:namespace prefix="udt" uri="urn:un:unece:uncefact:data:standard:UnqualifiedDataType:100" />

   </core:namespaces>

 

   <core:filterSettings type="SAX NG" />

 

   <core:exports>

      <core:result type="org.smooks.io.payload.JavaResult"/>

   </core:exports>

 

   <jb:bean beanId="sap_Invoi_No_Fi" class="de.edi.ownactrelat.pinvoice.Invoice" createOnElement="/CrossIndustryInvoice">

                …

                   </jb:bean>

                    …

                </smooks-resource-list>

 

We have multiple jb:beans in the config.

 

In RC-3 we can use this configuration and transform the incoming xml into our Invoice class. Do we miss something, or do we have to register the config file differently than we do in our SmookProducer?

 

We will create an issue for the BigDecimal problem.

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujvir%3DE1xYVBLFb4GVtw%2BiFCQ2XawM7dpxLvwhdrk4tUQ%40mail.gmail.com.

[Claude Mamo]

Never a good idea to reference snapshot :). I've merged a no. of API breaking changes lately to smooks-core so I'm not surprised that you got an error. I haven't yet fixed the downstream modules. If you need this urgently, I guess we can publish another RC before the big release. We should have one anyway so that the community can give feedback on new management module.

Claude 

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/003601da7085%24fe24d940%24fa6e8bc0%24%40gmail.com.

[mast....@gmail.com]

Hehe - yes you are right. We only used the snapshot to verify that your changes solve the problem, and they do.

 

Unfortunately, it’s urgent for us. When we include all our routes, we see startup times of about 30 mins and more.

When you decide to release a new RC, can you provide a time frame?

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujw1e-H%2BaHmA0c9eVF_Q1%2Bad%3DzAL595CrRek86GeG9nCg%40mail.gmail.com.

[Claude Mamo]

I've created a GitHub issue for the following error:

Caused by: java.lang.NoSuchMethodException: org.smooks.engine.converter.StringToBigDecimalConverterFactory$1.<init>()
at java.base/java.lang.Class.getConstructor0(Class.java:3585)
at java.base/java.lang.Class.newInstance(Class.java:626)
... 104 common frames omitted

About v2.0.0-RC4, should be published sometime next week. I've started a discussion about the release in the dev mailing list.

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/004f01da7096%249a0dec60%24ce29c520%24%40gmail.com.

[mast....@gmail.com]

Hi Claude,

 

This is great news about RC-4. Thank you.

 

Unfortunately, we hit another problem. When the smooks config contains an xpath expression with positions declarations a java.util.ConcurrentModificationException will be thrown. We could reproduce the problem with a modified version of the xml2java example.

 

The smooks config <jb:bean beanId="header" class="org.smooks.examples.xml2java.model.Header" createOnElement="header[2]">

This will produce a

org.smooks.api.SmooksException: Failed to filter source

                at org.smooks.engine.delivery.sax.ng.SaxNgFilter.doFilter(SaxNgFilter.java:122)

                at org.smooks.engine.delivery.sax.ng.SaxNgFilter.doFilter(SaxNgFilter.java:94)

                at org.smooks.Smooks._filter(Smooks.java:546)

                at org.smooks.Smooks.filterSource(Smooks.java:499)

                at org.smooks.examples.xml2java.Main.runSmooks(Main.java:85)

                at org.smooks.examples.xml2java.JavabeanTest.test(JavabeanTest.java:62)

                at java.base/java.lang.reflect.Method.invoke(Method.java:568)

                at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

                at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Caused by: java.util.ConcurrentModificationException

                at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1013)

                at java.base/java.util.ArrayList$Itr.next(ArrayList.java:967)

                at org.smooks.engine.delivery.sax.ng.SaxNgContentDeliveryConfig.addPositionCounters(SaxNgContentDeliveryConfig.java:243)

                at org.smooks.engine.delivery.sax.ng.SaxNgContentDeliveryConfig.addPositionCounters(SaxNgContentDeliveryConfig.java:232)

                at org.smooks.engine.delivery.sax.ng.SaxNgContentDeliveryConfig.reduceIndex(SaxNgContentDeliveryConfig.java:222)

                at org.smooks.engine.delivery.sax.ng.SaxNgContentDeliveryConfig.get(SaxNgContentDeliveryConfig.java:151)

                at org.smooks.engine.delivery.sax.ng.SaxNgHandler.<init>(SaxNgHandler.java:118)

                at org.smooks.engine.delivery.sax.ng.SaxNgHandler.<init>(SaxNgHandler.java:105)

                at org.smooks.engine.delivery.sax.ng.SaxNgParser.parse(SaxNgParser.java:71)

                at org.smooks.engine.delivery.sax.ng.SaxNgFilter.doFilter(SaxNgFilter.java:110)

                ... 8 more

 

The complete smooks config looks like this:

<?xml version="1.0"?>

<smooks-resource-list xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd" xmlns:jb="https://www.smooks.org/xsd/smooks/javabean-1.6.xsd">

    <jb:bean beanId="order" class="org.smooks.examples.xml2java.model.Order" createOnElement="order">

        <jb:wiring property="header" beanIdRef="header" />

        <jb:wiring property="orderItems" beanIdRef="orderItems" />

    </jb:bean>

   

    <jb:bean beanId="orderItems" class="java.util.ArrayList" createOnElement="order">

        <jb:wiring beanIdRef="orderItem" />

    </jb:bean>

    <jb:bean beanId="header" class="org.smooks.examples.xml2java.model.Header" createOnElement="header[2]">

        <jb:value property="date" decoder="Date" data="header/date">

            <jb:decodeParam name="format">EEE MMM dd HH:mm:ss z yyyy</jb:decodeParam>

            <jb:decodeParam name="locale-language">en</jb:decodeParam>

            <jb:decodeParam name="locale-country">IE</jb:decodeParam>

        </jb:value>

        <jb:value property="customerNumber" decoder="Long" data="header/customer/@number" />

        <jb:value property="customerName" data="header/customer" />

    </jb:bean>

 

    <jb:bean beanId="orderItem" class="org.smooks.examples.xml2java.model.OrderItem" createOnElement="order-item">

        <jb:value property="productId" decoder="Long" data="order-item/product" />

        <jb:value property="quantity" decoder="Integer" data="order-item/quantity" />

        <jb:value property="price" decoder="Double" data="order-item/price" />

    </jb:bean>

</smooks-resource-list>

.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAuhfvDKWrdckknJYUxaGAg31qOtRqJSdR7wZ--pz3Tmxyg%40mail.gmail.com.

[mast....@gmail.com]

We hit another problem with case sensitivity in xpath expressions in the smooks config file.

 

Our expression is:

<jb:value property="cfdiEmisorRfc" data="Emisor/@Rfc"/>

 

Unfortunately, the xml files we need to process do not care about the case of the tag or attribute names. So, we are facing xml documents containing this:

• <cfdi:Emisor rfc="RIT080306K39" />
• <cfdi:Emisor Rfc="SME9502015F6" />

 

In smooks 1.7.0 this was not a problem, both rfc values were extracted. In 2.0.0-RC-3 however  the value of the first example is not extracted anymore.  How can we achieve that smooks ignores the case of an attribute or tag name?

 

Best regards,

Martin

 

Von: smook...@googlegroups.com <smook...@googlegroups.com> Im Auftrag von Claude Mamo

Gesendet: Donnerstag, 14. März 2024 08:21

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAuhfvDKWrdckknJYUxaGAg31qOtRqJSdR7wZ--pz3Tmxyg%40mail.gmail.com.

[Claude Mamo]

In smooks 1.7.0 this was not a problem, both rfc values were extracted. In 2.0.0-RC-3 however  the value of the first example is not extracted anymore.  How can we achieve that smooks ignores the case of an attribute or tag name?

Frankly, I don't know whether to call this a feature or a bug in v1.7. What's the input source? Is it XML?

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/00a801da75fb%249aa11520%24cfe33f60%24%40gmail.com.

[mast....@gmail.com]

Yes, the input is XML. It’s an invoice in CFDI (https://www.avalara.com/vatlive/en/country-guides/north-america/mexico/mexican-e-invoices.html).

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAug09s4GcewRPP%2B4XG7fu%3DMrE%2BAQ4-cHJmm5k0Sisu-%2BuA%40mail.gmail.com.

[Claude Mamo]

Try using delegate-reader which was introduced in v2 (and still needs to be documented). With this reader, you can rewrite the input XML from within Smooks in streaming fashion. The downstream visitors will visit the rewritten XML. In your config, add something like this:

<smooks-resource-list xmlns="https://www.smooks.org/xsd/smooks-2.0.xsd"
                      xmlns:jb="https://www.smooks.org/xsd/smooks/javabean-1.6.xsd"
                      xmlns:core="https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd">

    <core:delegate-reader>
        <resource-config selector="*">
            <resource>org.smooks.examples.xml2java.RewriteAttributesVisitor</resource>
        </resource-config>
    </core:delegate-reader>
   
    ...
    ...
 </smooks-resource-list>  

Then implement the above RewriteAttributesVisitor to turn all the attributes into lower case:

package org.smooks.examples.xml2java;

import org.smooks.api.ExecutionContext;
import org.smooks.api.SmooksException;
import org.smooks.engine.delivery.sax.ng.SimpleSerializerVisitor;
import org.smooks.io.Stream;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;

import java.io.IOException;

public class RewriteAttributesVisitor extends SimpleSerializerVisitor {
    @Override
    public void visitBefore(Element element, ExecutionContext executionContext) {
        NamedNodeMap attributes = element.getAttributes();

        for (int i = 0; i < attributes.getLength(); i++) {
            Node item = attributes.item(i);
            element.removeAttribute(item.getNodeName());
            element.setAttribute(item.getNodeName().toLowerCase(), item.getNodeValue());
        }

        try {
            writeStartElement(element, Stream.out(executionContext), executionContext);
        } catch (IOException e) {
            throw new SmooksException(e);
        }
    }
}

Of course, customise accordingly. I'm hoping that like this we don't need to re-introduce case-insensitive selectors in the jb:value resource which seems a bit dodgy to me.

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/00b101da762f%24393c8920%24abb59b60%24%40gmail.com.

[mast....@gmail.com]

Hi Claude,

 

this seems to work in principle. But one needs to be careful not to translate all attributes. At least xmlns and xsi attributes must be excluded otherwise the namespaces may not work anymore. Also, one must respect namespaces in attributes (like <tag myNs:attribute=”text” /> ).

Unfortunately, your implementation example will not work correctly, because when removing an attribute all the remaining attributes will move up one index. This is a problem because the attributes are lexicographically ordered. When the first attribute was starting with a capital letter, then it will be inserted at another index than its original index.

 

Coding this rewrite is surprisingly complex. I am not quite sure that the user of smooks should do it since it’s quite error prone. I’ve probably missed more hidden problems because I am not that deep into XML.

When we write it ourselves it means that all the mappings in our smooks-resource-list will not follow the specification on which the documents we want to process are based.

For instance, if the specification says that a value is located at <Invoice @invoiceNumber=”123456” /> the smooks mapping for the invoice number would look like this

<jb:value property="invoiceNumber" data="/Invoice/@invoiceNumber" />

When we want to (more precisely have to…) support all cases invoiceNumber can occur we must implement the rewrite. But then the smook-mapping needs to change to

<jb:value property="invoiceNumber" data="/Invoice/@invoicenumber" />

Now the mapping does not represent the specification anymore because the specification talks about /Invoice/@invoiceNumber. Now the xpath expressions only work for smooks.

 

But if one neglects all the for once we run into encoding problems. In one of our xml files the following is stated:

<cfdi:DomicilioFiscal pais="México" />

This fails with

Caused by: org.smooks.api.SmooksException: com.fasterxml.aalto.WFCException: Invalid UTF-8 middle byte 0x78

at [row,col {unknown-source}]: [3,175]

 

The xml file claims to be UTF-8.

.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAuigJw9RHykt%2BOcbLFkqiLwWaP4MFx4xjfxA34varownMQ%40mail.gmail.com.

[Claude Mamo]

That last error is a bug in the way the rewrite handles multi-byte characters. é is causing it. Thanks for identifying it. I've created an issue for it and the fix will be available in RC4. About whether rewriting is the right solution for you, another alternative is to extend the JavaBean cartridge so that it replicates the old behaviour. I discovered  the cartridge is not very extendable at the moment because most Java methods are private but this will change in RC4. In RC4, you'll only need to extend a method and create an extension config for the extended cartridge to obtain the desired behaviour. I'll try to document this before pushing the release out.

Claude

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/014a01da76b7%24cf9d5ba0%246ed812e0%24%40gmail.com.

[mast....@gmail.com]

Hi Claude,

 

is there any news about the release of RC-4?

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAujb_R_n5kJobzSnBayMFkL8RuC9A5zQO%2BNxn%3Dk0mjWc1g%40mail.gmail.com.

[Claude Mamo]

I was out for the Easter holidays. Have one last fix to write which is about the ConcurrentModificationException exception you are getting when using indexed selectors. Hopefully I'll get to it this week. It goes without saying that PRs are more than welcome :) Though if this error isn't a blocker for you, I can cut a release straightaway.

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/000001da8017%24a4d5a940%24ee80fbc0%24%40gmail.com.

[mast....@gmail.com]

Hi Claude,

 

this fix is also crucial for us, because otherwise one of our routes would not work properly anymore. We gladly wait for it to be included in RC4.

As you have probably heard a thousand times regarding PRs already: we would like to participate but we are short on manpower ourselves ☹I am very sorry.

To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/CADvAAugd2Lho2tOCwNdEnhSDKy%2BscRUB1-_PDX7XiYMYYW_0PA%40mail.gmail.com.