Porting from JPMML 1.3.7 to 1.5.1

[Sashikanth Chandrasekaran]

Hello,

I am porting JPMML model and evaluator from 1.3.7 to 1.5.1. For the most part I could figure out the API changes. However, I am not able to find the following classes that were available in 1.3.7. Any suggestions? Thanks!

-sashi.

org.jpmml.model.visitors.ArrayListOptimizer

org.jpmml.model.visitors.DataDictionaryCleaner

org.jpmml.model.visitors.MiningSchemaCleaner

org.jpmml.model.visitors.TransformationDictionaryCleaner

[Villu Ruusmann]

Hi Sashi,

> I am porting JPMML model and evaluator from 1.3.7 to 1.5.1.

That's a rather major leap.

I'd suggest you to perform it in two stages:
1) Upgrade from 1.3.7 to 1.4.4, so that you can start using the
org.jpmml.evaluator.ModelEvaluatorBuilder approach. More technical
details can be found here:
https://openscoring.io/blog/2018/12/06/jpmml_evaluator_api_builder_pattern/
2) Upgrade from 1.4.4 to latest 1.5.X. Now you can update all the
renamed/relocated classes, and get rid of redundant code.

> For the most part I could figure out the API changes.
>

When performing major version upgrades, then it might be a good idea
to scrap your existing non-compileable code altogether, and get the
latest from the "Basic Usage" section of the README file:
https://github.com/jpmml/jpmml-evaluator#basic-usage

> However, I am not able to find the following classes that were available in 1.3.7.
>

After upgrading to 1.5.1, you can safely delete this part of your code
that is dealing with configuring and applying visitors. The
o.j.e.LoadingModelEvaluator does this all for you:
https://github.com/jpmml/jpmml-evaluator/blob/1.5.1/pmml-evaluator/src/main/java/org/jpmml/evaluator/LoadingModelEvaluatorBuilder.java#L57

Invoking LoadingModelEvaluator#setVisitors(VisitorBattery) with a
hand-built visitor battery may actually degrade the performance.

> org.jpmml.model.visitors.ArrayListOptimizer
>

I believe it was renamed to org.jpmml.model.visitors.ArrayListTransformers.

> org.jpmml.model.visitors.DataDictionaryCleaner
> org.jpmml.model.visitors.MiningSchemaCleaner
> org.jpmml.model.visitors.TransformationDictionaryCleaner
>

They were moved to the JPMML-Converter library:
https://github.com/jpmml/jpmml-converter/tree/master/src/main/java/org/jpmml/converter/visitors

They are very useful when creating PMML documents from scratch. They
are much less useful when applied to existing PMML documents.


VR

[Sashikanth Chandrasekaran]

Hi Villu,

Thank you for the detailed response. These steps mostly worked for me but I ran into one issue with MiningModel with the following stack trace. 

I see that 1.5.1 has a MiningModelEvaluator as well as the MiningModel class. Perhaps there are some types of MiningModel that 1.5.1 does not support?

Any recommendations?

Thank you,

-sashi.

org.jpmml.evaluator.UnsupportedElementException: Element MiningModel is not supported
	at org.jpmml.evaluator.ModelManagerFactory.newModelManager(ModelManagerFactory.java:82)
	at org.jpmml.evaluator.ModelEvaluatorFactory.newModelEvaluator(ModelEvaluatorFactory.java:31)
	at org.jpmml.evaluator.ModelEvaluatorBuilder.build(ModelEvaluatorBuilder.java:107)

--

-sashi.

[Villu Ruusmann]

Hi Sashi,

> I see that 1.5.1 has a MiningModelEvaluator as well as the MiningModel class.
>

The pattern is that for every <Model> element (backed by a PMML class
model object in the JPMML-Model library), there is a corresponding
<Model>Evaluator model evaluator class:

org.dmg.pmml.tree.TreeModel -> org.jpmml.evaluator.tree.TreeModelEvaluator
org.dmg.pmml.mining.MiningModel ->
org.jpmml.evaluator.mining.MiningModelEvaluator

Right now there is one-to-one correspondence. In the near future, the
JPMML-Evaluator library will start providing multiple model evaluator
classes, with the intent of making it possible to provide <Model>
element configuration-specific subclasses. For example, one subclass
for small numeric-only decision tree ensembles and another one for
huge sparse decision tree ensembles.

> Perhaps there are some types of MiningModel that 1.5.1 does not support?
>

Nope. The org.jpmml.evaluator.mining.MiningModelEvaluator class is a
general-purpose model evaluator that should be able to handle any
MiningModel element out there (therefore it's so big and bloated right
now).

> org.jpmml.evaluator.UnsupportedElementException: Element MiningModel is not supported
> at org.jpmml.evaluator.ModelManagerFactory.newModelManager(ModelManagerFactory.java:82)
> at org.jpmml.evaluator.ModelEvaluatorFactory.newModelEvaluator(ModelEvaluatorFactory.java:31)
> at org.jpmml.evaluator.ModelEvaluatorBuilder.build(ModelEvaluatorBuilder.java:107)
>

I suspect there's a class name/type mismatch in your application.

The second argument to the ModelManagerFactory#newModelManager(PMML,
Model) method invocation should be an instance of
org.dmg.pmml.mining.MiningModel:
https://github.com/jpmml/jpmml-evaluator/blob/1.5.1/pmml-evaluator/src/main/java/org/jpmml/evaluator/ModelManagerFactory.java#L56-L83

However, in your case, it's something else. Perhaps your application
is using shading, so that org.dmg.pmml.mining.MiningModel has been
renamed to something else?

The matching in ModelManagerFactory#getServiceProviderClasses(Class)
is class name based, so shading would break it:
https://github.com/jpmml/jpmml-evaluator/blob/1.5.1/pmml-evaluator/src/main/java/org/jpmml/evaluator/ModelManagerFactory.java#L85-L105

What does ModelManagerFactory#getServiceProviderClasses(MiningModel.class)
print out for you?


VR

[Sashikanth Chandrasekaran]

Hi Villu,

Thanks once again for the detailed response. You are absolutely right. A few of our applications had shaded the JPMML jar. We will figure out a way to unshade it.

-sashi.

--

-sashi.