Issue when i migrated from smooks RC2 to RC3 version.

[Reshma Kumari]

Hi Team,

I am getting following exception when i updated smooks from RC2 to RC3 version. The conversion works fine if i use RC2.

Any leads please.

Caused by: java.lang.RuntimeException: org.apache.daffodil.lib.exceptions.Abort: Invariant broken: parts.length.==(2)

org.apache.daffodil.lib.exceptions.Assert$.abort(Assert.scala:159)

org.apache.daffodil.lib.util.Misc$.optRelativeJarFileURI(Misc.scala:183)

org.apache.daffodil.lib.util.Misc$.getResourceRelativeOnlyOption(Misc.scala:145)

org.apache.daffodil.lib.xml.XMLUtils$.$anonfun$resolveSchemaLocation$3(XMLUtils.scala:1492)

at org.smooks.cartridges.dfdl.DataProcessorFactory.lambda$compileOrGet$0(DataProcessorFactory.java:104)

at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)

at org.smooks.cartridges.dfdl.DataProcessorFactory.compileOrGet(DataProcessorFactory.java:99)

at org.smooks.cartridges.edifact.EdifactDataProcessorFactory.createDataProcessor(EdifactDataProcessorFactory.java:116)

... 207 more

Caused by: org.smooks.api.SmooksConfigException: org.smooks.api.SmooksConfigException: java.lang.RuntimeException: org.apache.daffodil.lib.exceptions.Abort: Invariant broken: parts.length.==(2)
org.apache.daffodil.lib.exceptions.Assert$.abort(Assert.scala:159)
org.apache.daffodil.lib.util.Misc$.optRelativeJarFileURI(Misc.scala:183)
org.apache.daffodil.lib.util.Misc$.getResourceRelativeOnlyOption(Misc.scala:145)
org.apache.daffodil.lib.xml.XMLUtils$.$anonfun$resolveSchemaLocation$3(XMLUtils.scala:1492)
at org.smooks.cartridges.dfdl.unparser.DfdlUnparserContentHandlerFactory.create(DfdlUnparserContentHandlerFactory.java:74)
at org.smooks.cartridges.dfdl.unparser.DfdlUnparserContentHandlerFactory.create(DfdlUnparserContentHandlerFactory.java:56)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.addContentDeliveryUnit(DefaultContentDeliveryConfigBuilder.java:442)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyContentDeliveryUnitStrategy(DefaultContentDeliveryConfigBuilder.java:399)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyStrategy(DefaultContentDeliveryConfigBuilder.java:373)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.iterate(DefaultContentDeliveryConfigBuilder.java:524)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.access$200(DefaultContentDeliveryConfigBuilder.java:504)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.extractContentHandlers(DefaultContentDeliveryConfigBuilder.java:346)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.load(DefaultContentDeliveryConfigBuilder.java:238)
at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.build(DefaultContentDeliveryConfigBuilder.java:140)
at org.smooks.engine.delivery.DefaultContentDeliveryRuntimeFactory.create(DefaultContentDeliveryRuntimeFactory.java:86)
at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:117)
at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:94)
at org.smooks.Smooks.createExecutionContext(Smooks.java:438)
at org.smooks.Smooks.createExecutionContext(Smooks.java:404)
at org.smooks.engine.resource.visitor.smooks.NestedSmooksVisitor.filterSource(NestedSmooksVisitor.java:398)
at org.smooks.engine.resource.visitor.smooks.NestedSmooksVisitor.replaceBefore(NestedSmooksVisitor.java:325)
at org.smooks.engine.resource.visitor.smooks.NestedSmooksVisitor.visitBefore(NestedSmooksVisitor.java:264)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:178)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:175)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.EventInterceptor.visitBefore(EventInterceptor.java:67)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:178)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:175)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.TextConsumerInterceptor.visitBefore(TextConsumerInterceptor.java:89)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:178)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:175)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.StreamResultWriterInterceptor.visitBefore(StreamResultWriterInterceptor.java:87)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:178)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:175)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.ExceptionInterceptor.intercept(ExceptionInterceptor.java:106)
at org.smooks.engine.delivery.interceptor.ExceptionInterceptor.visitBefore(ExceptionInterceptor.java:86)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:178)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor$VisitBeforeInvocation.invoke(AbstractInterceptorVisitor.java:175)
at org.smooks.engine.delivery.interceptor.AbstractInterceptorVisitor.intercept(AbstractInterceptorVisitor.java:122)
at org.smooks.engine.delivery.interceptor.StaticProxyInterceptor.visitBefore(StaticProxyInterceptor.java:120)
at org.smooks.engine.delivery.sax.ng.SaxNgHandler.visitBefore(SaxNgHandler.java:284)
at org.smooks.engine.delivery.sax.ng.SaxNgHandler.startElement(SaxNgHandler.java:186)
at org.smooks.engine.delivery.SmooksContentHandler.startElement(SmooksContentHandler.java:110)
at com.ctc.wstx.sr.BasicStreamReader.fireSaxStartElement(BasicStreamReader.java:1787)
at com.ctc.wstx.sax.WstxSAXParser.fireStartTag(WstxSAXParser.java:806)
at com.ctc.wstx.sax.WstxSAXParser.fireEvents(WstxSAXParser.java:691)
at com.ctc.wstx.sax.WstxSAXParser.parse(WstxSAXParser.java:623)
at org.smooks.engine.delivery.sax.ng.SaxNgParser.parse(SaxNgParser.java:86)
at org.smooks.engine.delivery.sax.ng.SaxNgFilter.doFilter(SaxNgFilter.java:110)
... 158 more

at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)

at org.apache.daffodil.lib.oolag.OOLAG$.keepGoing(OOLAG.scala:65)

at org.apache.daffodil.core.dsom.SchemaSet.isValid$lzycompute(SchemaSet.scala:177)

at org.apache.daffodil.core.dsom.SchemaSet.isValid(SchemaSet.scala:172)

at org.apache.daffodil.core.dsom.SchemaSet.isError(SchemaSet.scala:589)

at org.apache.daffodil.core.compiler.ProcessorFactory.isError(Compiler.scala:138)

at org.apache.daffodil.core.compiler.Compiler.org$apache$daffodil$core$compiler$Compiler$$compileSourceInternal(Compiler.scala:362)

at org.apache.daffodil.core.compiler.Compiler$.org$apache$daffodil$core$compiler$Compiler$$compileSourceSynchronizer(Compiler.scala:408)

at org.apache.daffodil.core.compiler.Compiler.compileSource(Compiler.scala:339)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:169)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:155)

at org.smooks.cartridges.dfdl.DfdlSchema.compileSource(DfdlSchema.java:156)

at org.smooks.cartridges.dfdl.DfdlSchema.compile(DfdlSchema.java:124)

at org.smooks.cartridges.dfdl.DataProcessorFactory.lambda$compileOrGet$0(DataProcessorFactory.java:102) at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)

at org.apache.daffodil.lib.oolag.OOLAG$.keepGoing(OOLAG.scala:65)

at org.apache.daffodil.core.dsom.SchemaSet.isValid$lzycompute(SchemaSet.scala:177)

at org.apache.daffodil.core.dsom.SchemaSet.isValid(SchemaSet.scala:172)

at org.apache.daffodil.core.dsom.SchemaSet.isError(SchemaSet.scala:589)

at org.apache.daffodil.core.compiler.ProcessorFactory.isError(Compiler.scala:138)

at org.apache.daffodil.core.compiler.Compiler.org$apache$daffodil$core$compiler$Compiler$$compileSourceInternal(Compiler.scala:362)

at org.apache.daffodil.core.compiler.Compiler$.org$apache$daffodil$core$compiler$Compiler$$compileSourceSynchronizer(Compiler.scala:408)

at org.apache.daffodil.core.compiler.Compiler.compileSource(Compiler.scala:339)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:169)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:155)

at org.smooks.cartridges.dfdl.DfdlSchema.compileSource(DfdlSchema.java:156)

at org.smooks.cartridges.dfdl.DfdlSchema.compile(DfdlSchema.java:124)

at org.smooks.cartridges.dfdl.DataProcessorFactory.lambda$compileOrGet$0(DataProcessorFactory.java:102)

at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)

at org.apache.daffodil.lib.oolag.OOLAG$.keepGoing(OOLAG.scala:65)

at org.apache.daffodil.core.dsom.SchemaSet.isValid$lzycompute(SchemaSet.scala:177)

at org.apache.daffodil.core.dsom.SchemaSet.isValid(SchemaSet.scala:172)

at org.apache.daffodil.core.dsom.SchemaSet.isError(SchemaSet.scala:589)

at org.apache.daffodil.core.compiler.ProcessorFactory.isError(Compiler.scala:138)

at org.apache.daffodil.core.compiler.Compiler.org$apache$daffodil$core$compiler$Compiler$$compileSourceInternal(Compiler.scala:362)

at org.apache.daffodil.core.compiler.Compiler$.org$apache$daffodil$core$compiler$Compiler$$compileSourceSynchronizer(Compiler.scala:408)

at org.apache.daffodil.core.compiler.Compiler.compileSource(Compiler.scala:339)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:169)

at org.apache.daffodil.japi.Compiler.compileSource(Daffodil.scala:155)

at org.smooks.cartridges.dfdl.DfdlSchema.compileSource(DfdlSchema.java:156)

at org.smooks.cartridges.dfdl.DfdlSchema.compile(DfdlSchema.java:124)

at org.smooks.cartridges.dfdl.DataProcessorFactory.lambda$compileOrGet$0(DataProcessorFactory.java:102)

[Reshma Kumari]

I get this exception when i run the application on docker. The code seems to be working fine on my localhost.

Any leads? Could this be because of classloading issues?

[Claude]

Which version of Java are you running? Could you try Java 11?

Claude

[Reshma Kumari]

I am using java 17 version.

[Reshma Kumari]

Hi Claude,

In debugging, i further found that its using temp file (file:/var/folders/hb/79fhtzkj5hn6b9ysd7tnpb840000gp/T/6192875898312577926/EDIFACT-Interchange-481df9de-b55e-3491-9c79-76b3f9e846c2.dfdl.xsd) that gets created which is used  for creating compiler

processorFactory = compiler.compileSource(this.uri); (Inside DfdlSchema.java) The scheme is file, but if i see the error in stacktrace its pointing to the place where its trying to read file using jar URI. I am unable to understand how this temp file is used inside daffodil and why its trying to parse the URI which is parsed as jar URI.

If you can provide me some inputs around why the temp file is created and why its parsing it as jar URi will be helpful for my debugging.

[Claude]

I'll have to run the EDI cartridge tests on Java 17. So far, we've been testing the cartridge on Java 8 and 11 so it's possible that there is an incompatibility. What's weird here is that the error is occurring in your Docker container but not locally. I seem to recall the error stack trace you posted when I attempted to upgrade to the latest version of Daffodil. I had fixed the error by prepending a slash (see https://github.com/smooks/smooks-edi-cartridge/commit/f2c3b1084ad51dd707fb1fd44cf84c454481cdd8#diff-464940ac0f01118a96b31fd79b8a08d7fbc6c0bcaac90564aa46ce7221a42287R99) but I'll have to dig deeper to understand whether this error is related.

Claude

[Reshma Kumari]

To add to my observation, i see that the issue is because of the newly introduced code in org.apache.daffodil:daffodil-japi_2.12 dependency- version 3.6.0 in daffodil-lib/src/main/scala/org/apache/daffodil/lib/util/Misc.scala.

But what i don't understand is that why the code flow in daffodil side entering into resolution of jar path. I printed the URI which we send for dfdlschema.compile(), its something like (file:/var/folders/hb/79fhtzkj5hn6b9ysd7tnpb840000gp/T/6192875898312577926/EDIFACT-Interchange-481df9de-b55e-3491-9c79-76b3f9e846c2.dfdl.xsd) which is file URI.

Can you help me understand why it might be getting into flow of jar file resolution? I am sorry but i am really trying hard to compile daffodil codebase at my local. But no luck. So posting this doubt here. 

I believe it is trying to resolve the xsd:import/ xsd:include mentioned in the schema file which we pass for compiler generation. 
Any pointers on how these path looks like.

--
You received this message because you are subscribed to a topic in the Google Groups "Smooks Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/smooks-user/RDmTPWh9z4o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/092a36a5-be4e-4518-86fb-a2889bd7341an%40googlegroups.com.

[Reshma Kumari]

I am now able to execute daffodil locally and I found the issue is with the recent changes in daffodil (version 3.6.0) for resolving jar file locations.

I ran my projects jar in the docker which made the schema filelocation as jar:file:/dpw-test-1.0.0-SNAPSHOT-exe.jar!/BOOT-INF/lib/daffodil-lib_2.12-3.6.0.jar!/org/apache/daffodil/xsd/DFDL_part3_model.xsd 

When daffodil tried to resolve this location, It lands to Misc.optRelativeJarFileURI where there is check to split the filelocation with !/ and the expected length of split is 2. 

val parts = contextURI.toString.split("\\!\\/")
Assert.invariant(parts.length == 2)

This causes failure. Any idea if we can control this through smooks? I plan to open the bug at daffodil, but just wanted to check if this can be resolved by tweaking something at smooks itself.

Some tweaking at daffodil has made some progress, but i now i see following exception

Caused by: org.smooks.api.SmooksConfigException: org.smooks.api.SmooksConfigException: java.lang.RuntimeException: Schema Definition Error: Error loading schema due to org.xml.sax.SAXParseException; systemId: jar:file:/dpw-bridge-service-1.0.0-SNAPSHOT-exe.jar!/BOOT-INF/lib/daffodil-lib_2.12-3.6.0.jar!/org/apache/daffodil/xsd/XMLSchema_for_DFDL.xsd; lineNumber: 577; columnNumber: 47; src-resolve: Cannot resolve the name 'dfdl:anyOther' to a(n) 'attribute group' component.

Schema context: file:/tmp/14557585994110024041/EDIFACT-Interchange-481df9de-b55e-3491-9c79-76b3f9e846c2.dfdl.xsd Location in file:/tmp/14557585994110024041/EDIFACT-Interchange-481df9de-b55e-3491-9c79-76b3f9e846c2.dfdl.xsd

at org.smooks.cartridges.dfdl.unparser.DfdlUnparserContentHandlerFactory.create(DfdlUnparserContentHandlerFactory.java:74)

at org.smooks.cartridges.dfdl.unparser.DfdlUnparserContentHandlerFactory.create(DfdlUnparserContentHandlerFactory.java:56)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.addContentDeliveryUnit(DefaultContentDeliveryConfigBuilder.java:442)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyContentDeliveryUnitStrategy(DefaultContentDeliveryConfigBuilder.java:399)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ContentHandlerExtractionStrategy.applyStrategy(DefaultContentDeliveryConfigBuilder.java:373)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.iterate(DefaultContentDeliveryConfigBuilder.java:524)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder$ResourceConfigTableIterator.access$200(DefaultContentDeliveryConfigBuilder.java:504)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.extractContentHandlers(DefaultContentDeliveryConfigBuilder.java:346)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.load(DefaultContentDeliveryConfigBuilder.java:238)

at org.smooks.engine.delivery.DefaultContentDeliveryConfigBuilder.build(DefaultContentDeliveryConfigBuilder.java:140)

at org.smooks.engine.delivery.DefaultContentDeliveryRuntimeFactory.create(DefaultContentDeliveryRuntimeFactory.java:86)

at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:117)

at org.smooks.engine.DefaultExecutionContext.<init>(DefaultExecutionContext.java:94)

at org.smooks.Smooks.createExecutionContext(Smooks.java:438)

Any leads how this can be resolved?

[Reshma Kumari]

With the above findings i believe this will be faced by everyone who is trying to run the application using jar.

I don't think its the issue with java version and i believe you are not seeing any issue while running tests because they are using paths with file scheme and not jar scheme.

[Claude]

Best is to create an issue for this with all relevant details and a minimal reproduceable example.

Claude

[Reshma Kumari]

Created an issue for smooks:
https://github.com/smooks/smooks-edi-cartridge/issues/317

The issue is acknowledged and fixed at apache daffodil end. Smooks will need to update the daffodil dependency to fix the issue. Following is the link for daffodil issue tracking:
https://issues.apache.org/jira/browse/DAFFODIL-2892

You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/smooks-user/ba249f97-8751-4419-bec2-f300ba337ee4n%40googlegroups.com.