Optimization of Smooks Initialization Time for EDIFACT

[Akash Verma]

Hi Claude,

I’ve noticed that the first initialization of Smooks takes around 33 seconds, whereas subsequent calls—without reinitializing Smooks, as implemented in my code—take an average of around 0.080 seconds or 80ms.

I have a query regarding optimization: Is there a way to reduce the initialization time itself? In my use case, I need to initialize Smooks for different schemas and plan to store them in a map, as demonstrated in my code. However, this approach could significantly increase pod memory usage due to multiple EDIFACT versions (both out-of-the-box and custom). Storing Smooks objects in memory might not be ideal.

Can we optimize Smooks initialization time so that I don’t need to persist Smooks objects in JVM/pod memory? I have attached the results and the code for your reference. The code I’m following is from this repository:
Smooks Examples - XML to EDI.
I have only tweaked it slightly to fit my use case.

Please let me know if you need any additional details.

SmooksFile - https://pastebin.com/rBYdKjjb
Results - https://pastebin.com/S8iPEZaV

[Mantra]

Hi Akash,

Sorry bit confused,  are you using Java spring application? The smooks.addResourceConfigs("smooks-config.xml") will cache this schema when you build the application and will hold it in memory and once application is up it should be fine .  How you are setting the smooks-config.xml in you application? Even you can store it on disk with setting the config setting. 

Regards,

Mantra 

--
You received this message because you are subscribed to the Google Groups "Smooks Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to smooks-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/smooks-user/2d7f10eb-1a3f-44a1-969d-d34265f0307dn%40googlegroups.com.

[Akash Verma]

Hi Mantra,

Thanks for the response. Yes, I’m using a Java application, and the structure of the project is the same as we have here Smooks Examples - XML to EDI.. I’m programmatically setting up the settings.xml, as shown in the code, because we don’t have a working example for XML-to-EDIFACT.

I'm using:

pipeline.setParameter("smooksResourceList", "<smooks-resource-list xmlns=\"https://www.smooks.org/xsd/smooks-2.0.xsd\">" + edifactUnparser.toXml() + "</smooks-resource-list>");

instead of

smooks.addResourceConfigs("smooks-config.xml");

I don’t think it’s a cache issue because, if you check this out-of-the-box example JavaToEdifact Smooks Example Repo , where we are using smooks.addResourceConfigs("smooks-config.xml"), the first call during initialization takes ~58 seconds for the transformation, while in subsequent calls inside the loop, it processes within milliseconds.

• Code: https://pastebin.com/3YFR5Rtm
• Results: https://pastebin.com/P4nq5tBk

However, when I call the run/Transform method multiple times, as shown here:
https://pastebin.com/TbK7MuyS,

We consistently get a cache miss, and the transformation takes an average of 30 seconds.

• Results: https://pastebin.com/HbhqQhV8

It seems we’re always getting a cache miss based on the logs.

I’m looking for ways to optimize the Smooks initialization during the first call itself, as I’ll be creating different Smooks instances based on different smooks-config.xml files.

Please let me know if you need any additional details.

Thanks,
Akash

[Claude Mamo]

According to the cache miss log entries, the cacheOnDisk attribute on your edifact:unparser is set to false. You should obtain better initialisation time after the first run if you set it to true.

Claude

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/4f14e337-9563-4499-9306-5ff7e2807efcn%40googlegroups.com.

[Akash Verma]

Hi Claude,

Thanks for the solution—it worked perfectly! I tried adding cacheOnDisk as shown in the code here for EdifactToXml(EdifactParser): https://pastebin.com/cbyEjPkx , but it didn’t work. Could you help me figure out what might be wrong?

Akash

[Claude Mamo]

Try:

smooks.setReaderConfig(new EdifactReaderConfigurator("/d03b/EDIFACT-Messages.dfdl.xsd").setCacheOnDisk(true));

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/c2f89959-0525-4560-86cb-1215054f957en%40googlegroups.com.

[Akash Verma]

Tried! Getting the error :  https://pastebin.com/nS1n3Kf8

[Claude Mamo]

You might have to debug the error. Works for me locally.

To view this discussion visit https://groups.google.com/d/msgid/smooks-user/a4fadf5e-da4f-4293-b5e1-81e2f74034a3n%40googlegroups.com.