Dealing with many models

[Saulo Gil]

Hello,

We're currently working on a project that has >200 custom model classes. Declaring all of them inside model providers seems a bit tedious,  made me wonder if any of you guys had to deal with a similar situation.

We'd like to exchange ideas with other developers that had faced a similar issue and perhaps know how they had sorted it out.

Some ideas we're currently considering are:

• Tag models using annotations. We haven't figured out yet an elegant way for dealing with X* classes that aren't extended yet.
• Auto-generate the code that declares/registers models (and callouts), perhaps as a contribution to idempiere-plugin-scaffold.

Any thoughts would be welcome.

Best regards,

--

Saulo Gil | Orbital Software  | +54 911 3049 4237

[reua...@gmail.com]

For model class definitions I have extended the Ingenit Modelgenerator plugin.
The original plugin generates the Interface and X-Model classes. My extension will also create the M-Model class definition. (If that is what you mean by declarations)

Ingenit Model Generator:
https://wiki.idempiere.org/en/Plugin:_Model_Generator

my fork:
https://github.com/Soundbytes/idempiere-model-generator-plugin

I have to admit though I have not been able to update the 2-pack to include my modifications to the UI.
The 2-pack.zip can not be edited manually so I would have to recreate it from scratch and until now I did not do that.

In short: while my fork is the best thing since sliced bread (At least IMO) it will take some effort to make it run out of the box. If Orlando is prepared to slightly modify the original packin I could provide a pull request fairly soon to get the M-Model generator included in the original plugin.

Model class registration requires a custom model factory.

I have written one that will derive the model class name from the table name and then instantiate the model for a given table name prefix. BTW: works with arbitrary prefix length. Just put it into the model class folder and add the modelfactory.xml to the OSGI-INF folder and you're ready to go.

import java.lang.reflect.Constructor;

import java.sql.ResultSet;

import java.util.Properties;

import org.adempiere.base.IModelFactory;

import org.compiere.model.PO;

import org.compiere.util.CLogger;

import org.compiere.util.Env;

import org.compiere.util.Util;

public class MYModelFactory implements IModelFactory{

private final static CLogger log = CLogger.getCLogger(MYModelFactory.class);

private final static String packageName = "my.organization.model.package.name";

private final static String prefix = "MY_"; // <- Insert your prefix here

@Override

public Class<?> getClass(String tableName) {

if (!tableName.startsWith(prefix))

return null;

String className = tableName.substring(4);

className = packageName + ".M" + Util.replace(className, "_", "");

Class<?> clazz = null;

try {

clazz = Class.forName(className);

} 

catch (ClassNotFoundException e) {

e.printStackTrace();

log.severe("Class " + className + " not found.");

}

return clazz;

}

@Override

public PO getPO(String tableName, int Record_ID, String trxName) {

if (!tableName.startsWith(prefix))

return null;

Class<?> clazz = getClass(tableName);

if (clazz == null)

return null;

try {

Constructor<?> constructor = null;

try {

constructor = clazz.getDeclaredConstructor(new Class[]{Properties.class, int.class, String.class});

}

catch (Exception e) {

String msg = e.getMessage();

if (msg == null)

msg = e.toString();

log.severe("No transaction Constructor for " + clazz + " (" + msg + ")");

}

PO po = constructor!=null ? (PO)constructor.newInstance(new Object[] {Env.getCtx(), Integer.valueOf(Record_ID), trxName}) : null;

return po;

}

catch (Exception e){

e.printStackTrace();

}

return null;

}

}

[reua...@gmail.com]

Note: MYModelfactory is based on the iDempiere DefaultModelFactory authored by Jörg Janke and Hengsin. 
So Kudos and credits to them. Thanks Guys!

Andreas

[Saulo Gil]

Hello Andreas, thank you for the quick response.

We've thought about a similar approach, resembling classic Compiere class model resolution. Although it's simple enough, we're supposed to be wary of using Class.forName(String) under OSGi's realm (see https://developer.ibm.com/articles/osgi-demystified-part-3-avoiding-dynamic-class-loading/). This is relevant for us since one of our projects have classes that clash with some iDempiere classes and we won't be able to change that for a while.

As a side note, we're a bit suprised that the core is still using Class.forName. Would it make sense to change it to something like the following?

	Class clazz = packageAdmin.getExportedPackage(packageName).getExportingBundle().loadClass(className);

At this moment we're considering auto-generating class model registration code, since Java annotations don't seem to go well with X* model classes (they're not supposed to be edited).

Best regards,

Saulo Gil | Orbital Software  | +54 911 3049 4237

--
You received this message because you are subscribed to the Google Groups "iDempiere" group.
To unsubscribe from this group and stop receiving emails from it, send an email to idempiere+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/0de9bee8-b955-45d6-98e5-d40a86ee8aa8n%40googlegroups.com.

[Heng Sin Low]

if you wants to goes the full annotation path, you should change the model generator to either:

1) generate annotation for x* classes

OR

2) always generate m* class together with the annotation

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/01000178fb4cd911-274c5296-127a-42f9-a2a9-94a07f36303d-000000%40email.amazonses.com.

[Saulo Gil]

I'd rather to mix both options a bit, since:

• If there's an M* class, that's what I'd like to get
• I think I get the idea, but forcing M* class creation might create a bit of noise

I really liked your idea of using a custom model generator, I'll come back to this thread with some code.

Thank you

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/CAC%3D6jXG97nDBZygAsoruRWxBc6vjko0EoBnAFNj9qBmTfzcXvA%40mail.gmail.com.

[reua...@gmail.com]

I did also consider that option but found that always having an auto generated M-Model class suits me best even if it is only an empty stub sometimes. Since I am mainly  dealing with customizations of limited size I found it preferable to have a lean api that does not allow me to introduce too many errors into the code. So I tend to limit the available options in favour of maintainability even if that means a bit of overhead sometimes. Of course your mileage may vary.

Note that my M-Model generator is rather foolproof in that it will never overwrite an existing M-Class and throw an error instead. 

Andreas

[Carlos Antonio Ruiz Gomez]

Generating M class is always recommended - is better when your code reference the M class instead of the X_ classes, otherwise you'll need to do the changes from X to M later when you implement it.

I think is worthy if you want to add it to core, with a third flag to generate the M class - so is user decision.

Also I think Heng Sin suggested to include in core the plugin from Ingeint, that would be really useful on development time.

Regards,

Carlos Ruiz

Am 26.04.21 um 14:03 schrieb reua...@gmail.com:

[reua...@gmail.com]

Thanks Carlos,

I am all for including the plugin into core because I find it much more convinient than the standalone Model generator.  I'll also be happy to share my additions upstream either to the ingenit plugin or directly to core once it has been integrated.

Andreas
 

[Saulo Gil]

We've created an experimental branch where we register model classes using annotations.

Here we rely on the ClassIndex library, which builds a list of all annotated classes at compile time. By using annotation processors we get faster and cleaner association between each table and its model class, since it avoids using Class.forName. Might be needless to say that we also avoid runtime class scanning, which is slow and error-prone. The one thing we didn't like about this approach is that Eclipse requires annotation processors to be configured manually.

Though we haven't tested this approach extensively, we haven't found issues consuming these annotations from plugins. Creating IModelFactory implementations should be trivial enough.

Best regards,

Saulo Gil | Orbital Software  | +54 911 3049 4237

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/010001790b7c3696-6d69de55-0eff-48b3-acfb-e3580e295db5-000000%40email.amazonses.com.

[Heng Sin Low]

Is this stable enough for integration to core ?

Would be a nice enhancement to core.

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/010001791ba6852b-1b5c77a2-6481-44e6-ba71-9889fc67b0e9-000000%40email.amazonses.com.

[Saulo Gil]

Hello,

These changes should be stable, although that branch was created as a proof of concept. If you core developers like this approach we could move ahead and follow the extra steps (FR, documentation, testing, etc) targeting production quality.

Please confirm.

Best regards,

Saulo Gil | Orbital Software  | +54 911 3049 4237

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/CAC%3D6jXH46SM0EqwMC%2B6XrLd-eQe13wBEd0DD8c9HmWCfrcxNmg%40mail.gmail.com.

[Heng Sin Low]

1. The approach looks good to me.

2. I think this should be created as a new model factory with higher service.ranking than the current default model factory. This way, it can fall back to the current default model factory.

3. Have you tested with the model class from the plugin ? I'm not sure whether there would  be class loader issues here. If there are, I think it should be easy for plugins to reuse the new model factory added at #2.

4. It seems at Eclipse, you have to configure the annotation processor by each individual project. Do you know whether there is a simpler approach ?

5. What's the impact on our maven tycho build ? just some pom changes ?

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/01000179a88f80f6-42a66c90-c3c4-4576-9cd3-e2428e9feb2d-000000%40email.amazonses.com.

[Saulo Gil]

Hello,

Please see my replies below. We're submitting a FR soon.

Best regards

On 26/5/21 09:19, Heng Sin Low wrote:

1. The approach looks good to me.

We're doing it then

2. I think this should be created as a new model factory with higher service.ranking than the current default model factory. This way, it can fall back to the current default model factory.

I like the idea, even tough our intervention to DefaultModelFactory was arguably minor, this is a cleaner approach

3. Have you tested with the model class from the plugin ? I'm not sure whether there would  be class loader issues here. If there are, I think it should be easy for plugins to reuse the new model factory added at #2.

Yes, it all worked with no issues

4. It seems at Eclipse, you have to configure the annotation processor by each individual project. Do you know whether there is a simpler approach ?

Not that I'm aware of, but we'll take a second look hoping we can at least inject this configuration on the project(s).

Please see https://github.com/atteo/classindex#eclipse

5. What's the impact on our maven tycho build ? just some pom changes ?

Just a pom.xml change (not sure about Tycho 2.4, we'll see). The ClassIndex lib has the Apache License 2.0.

To view this discussion on the web visit https://groups.google.com/d/msgid/idempiere/CAC%3D6jXGwhy1dNSeC%3DZS2zdoWFu2iJtJ0si5Sg9CYRa0a-chxqA%40mail.gmail.com.