Question regarding modular data model
38 views
Skip to first unread message
Jonas Pöhler
May 11, 2022, 2:14:55 AM5/11/22
to Ebean ORM
Hi all, hi Rob,
we are developing a company internal application framework which provides basic functionality for further customer-specific development.
The framework provides data structures like users, configuration capabilities and more.
When we want to develop further functionality for a specific customer we create another project, which depends on that framework and which can specify its own data model.
For the most part this works good, but we have a case, which is not obviously solvable with Ebean.
Let's say, the framework provides a user like this:
```
@Entity
public class User {
private String name;
private String email;
/*
...
*/
}
```
In the customer specific project the idea is, to have a another bean/table that could be joined to the user:
```
@Entity
public class AdditionalUserData {
private String datacenterId;
@OneToOne
private User user;
/*
...
*/
}
```
The issue now is, that the `AdditionalUserData` is directly dependent of `User` and under normal circumstances we would specify `orphanRemoval` from the user's side.
But here we cannot really do this, because the `User` class is effectively finite and cannot be changed at that point. We have this issue at multiple points in the application where there is a need
to store additional (searchable) for a framework bean.
This means, that additionally we would need to write queries in the style of:
```
DB.find(User.class).where()
.eq("additionalUserData.datacenterId", "foo")
.findOne()
```
How would one do something like this in Ebean? Is there another way to achieve something similar with the existing features of Ebean?
If I would try to implement something like this in Ebean, my approach would be to allow some kind of "reverse-relation-mapping" to be specified during the bean-scan (either via annotation or some more specific API),
but I don't know how well Ebean will behave without an actual property backing this relation. I'm also unsure if this would be the right approach and if such a feature would be appealing to the community.
All the best!
Jonas
we are developing a company internal application framework which provides basic functionality for further customer-specific development.
The framework provides data structures like users, configuration capabilities and more.
When we want to develop further functionality for a specific customer we create another project, which depends on that framework and which can specify its own data model.
For the most part this works good, but we have a case, which is not obviously solvable with Ebean.
Let's say, the framework provides a user like this:
```
@Entity
public class User {
private String name;
private String email;
/*
...
*/
}
```
In the customer specific project the idea is, to have a another bean/table that could be joined to the user:
```
@Entity
public class AdditionalUserData {
private String datacenterId;
@OneToOne
private User user;
/*
...
*/
}
```
The issue now is, that the `AdditionalUserData` is directly dependent of `User` and under normal circumstances we would specify `orphanRemoval` from the user's side.
But here we cannot really do this, because the `User` class is effectively finite and cannot be changed at that point. We have this issue at multiple points in the application where there is a need
to store additional (searchable) for a framework bean.
This means, that additionally we would need to write queries in the style of:
```
DB.find(User.class).where()
.eq("additionalUserData.datacenterId", "foo")
.findOne()
```
How would one do something like this in Ebean? Is there another way to achieve something similar with the existing features of Ebean?
If I would try to implement something like this in Ebean, my approach would be to allow some kind of "reverse-relation-mapping" to be specified during the bean-scan (either via annotation or some more specific API),
but I don't know how well Ebean will behave without an actual property backing this relation. I'm also unsure if this would be the right approach and if such a feature would be appealing to the community.
All the best!
Jonas
Jens
May 14, 2022, 11:13:31 AM5/14/22
to Ebean ORM
To me it doesn't really make sense to have a locked entity provided by a library/framework because sooner or later this restriction will cause trouble. I can see only three solutions:
- All entity related classes in your library/framework should not be @Entity but instead @MappedSuperclass. Any application then needs to extend them if they want to use that common logic in that class as part of their own class. Yes each application then needs to define their own entities and can only opt-in to use something predefined.
- In your library/framework entity class have some properties to store generic data. In your User example you could have a JSON field in the User class to hold possible additional information. Or you use a Map<String, String>.
- Use JPA inheritance so you can extend your User class. However each inheritance strategy has its own set of problems you need to be aware of. JPA inheritance can also lead to performance issues.
IMHO using @MappedSuperclass and maybe some interfaces in case an application wants to use logic of your library/framework but not extend the MappedSuperclass.
If your goal is to free developers from recreating the same entities again and again then you might want a code generator that creates the desired entities and possibly make these entities already implement some interface so they can be used with your library/framework code. Once entities have been generated for an application, the developer is free to change them. The library/framework would then only define interfaces.
-- J.
Rob Bygrave
May 15, 2022, 5:11:35 PM5/15/22
to Ebean ORM
Side note: Ok weird, I didn't get an email from this thread? I only came to google groups to check something and stumbled across this ...
---
Nice answers/options Jens.
Note that IF this was a case of the EAV (Entity Attribute Value) pattern, it is my opinion that the best approach to that is a JSON field assuming that there is good JSON support in the database(s). For example, Postrges JSONB. For myself, I'd be very hesitant to go to the older EAV table approach.
What I'd add, is that a while back if I recall correctly, I was talking to Roland (and others?) about the possibility of entities extended entities without JPA inheritance (which we don't do and isn't strictly valid in JPA). The thought was, would it be good/useful to be able to do:
@Entity
public class CustomUser extends User {
public class CustomUser extends User {
private String datacenterId;
...
}
... which maybe thought of as a variation on the JPA standard @MappedSuperclass approach (a variation that allows us to have more than 1 concrete implementation mapped to the same base table without using JPA inheritance).
So why would we do this rather than the standard JPA @MappedSuperclass (and probably some interface(s))? When we build/design User we might not absolutely know that something later wants to extend it in such a way. So supporting this means we don't need to redesign/retrofit back in some @MappedSuperclass later. For example, we have 20 customers and only 1 of them needs a customization of only 1 extra column on only 1 entity ... (if ebean supported this) this would be a smaller change to make to accommodate/retrofit that need.
So, if we supported this we'd have application code that could use User or CustomUser and they happen to point to the same table. Internally it would be similar to an Ebean @View in terms of invalidating L2 caching appropriately.
// common code ...
@Entity
@Table(name="my_user")
public class User { ... }
public class User { ... }
// custom code ...
// The Ebean @View mechanism can be used in terms of L2 caching
@Entity
// @View(name="my_user", dependantTables="my_user")
public class CustomUser extends User {
public class CustomUser extends User {
private String datacenterId;
...
}
In case it's not super obvious, this is really for applications that are built by a company and deployed for each customer, and where there is some potential for customization specific to that customer.
Hmmm.
Rob Bygrave
May 16, 2022, 7:19:01 AM5/16/22
to Ebean ORM
Ok, hacking around with the idea of allowing an entity can extend another entity ...
What that means to Ebean internals is approximately:
1. We currently detect an entity extends an entity and fail in DeployBeanDescriptor.checkInheritance() ... change this to not fail, remember the "originalType" that is going to be "overridden / customized"
2. BeanDescriptorManager
- Early ... we treat the "originalType" as if it were a MappedSuperclass and remove it from deployInfoMap (DDL will be driven off the custom type, original type is treated like a "mapped supertype")
- In readEntityBeanTable() ... we register the BeanTable to both the new customised type (as per normal) and additionally the original type (effectively 2 types point to the same BeanTable)
- In registerDescriptor() ... we register the BeanDescriptor to the new customized type (as per normal) and additionally the original type (effectively 2 types point to the same BeanDescriptor)
So, with say MyBean as the original and MyBeanCustom extends MyBean as the custom type
MyBeanCustom custom1 = DB.find(MyBeanCustom.class, custom.getId());
MyBean original1 = DB.find(MyBean.class, custom.getId());
MyBean original1 = DB.find(MyBean.class, custom.getId());
Both these queries execute and return instances of MyBeanCustom. In fact, all use of MyBean should actually build MyBeanCustom instances.
This is interesting and simple enough, I'll look to push up a PR tomorrow and people can have a look and play around if they like.
Jonas Pöhler
May 25, 2022, 11:01:16 AM5/25/22
to Ebean ORM
Sorry for blanking on this. Been out-of-office and didn't realize, that commenting here apparently doesn't work in my installation of firefox, so doing this again.
First: Thank you both for your thoughts and ideas. It gave me the opportunity to think about other options, but in the end, I came back around to my initial idea, so let me explain.
1. We are unfortunately limited to database systems, which don't really have great json support and querying for entries in "additional data" is a necessity for us, so json is not our way to solve this problem (we use it for data we really don't need to search through)
2. The thing with inheritance is the following: We are working towards a platform system where you can add functionality by effectively "installing" a function package (including a JAR file). There might be multiple function packages in one installation and all of those would extend the common entity. Currently this would either lead to a table with a lot of columns (single-table inheritance) or multiple tables with basically the same columns plus some extra (mapped superclasses). Both of these solution work, but they have issues like searching for common data is harder (would need a union) and limits in column counts on some database systems. The overwriting approach we had in mind before has the issue, that it limits us to one "implementation" per JVM, which is not very flexible.
If there wasn't a border between the code modules, this data would usually just be referenced using a OneToOne or OneToMany from the main data entity anyway and we can almost model this with existing Ebean functionality. The one problem is that we cannot tell the main model to cascade it's deletion, resulting in a foreign key error.
In my head this would need some way of mapping this on the entities and additionally some logic when parsing annotations.
I hope, that I could make my concerns with these ideas clear. If you see something I didn't consider just tell me, maybe this still isn't our way to go.
First: Thank you both for your thoughts and ideas. It gave me the opportunity to think about other options, but in the end, I came back around to my initial idea, so let me explain.
1. We are unfortunately limited to database systems, which don't really have great json support and querying for entries in "additional data" is a necessity for us, so json is not our way to solve this problem (we use it for data we really don't need to search through)
2. The thing with inheritance is the following: We are working towards a platform system where you can add functionality by effectively "installing" a function package (including a JAR file). There might be multiple function packages in one installation and all of those would extend the common entity. Currently this would either lead to a table with a lot of columns (single-table inheritance) or multiple tables with basically the same columns plus some extra (mapped superclasses). Both of these solution work, but they have issues like searching for common data is harder (would need a union) and limits in column counts on some database systems. The overwriting approach we had in mind before has the issue, that it limits us to one "implementation" per JVM, which is not very flexible.
If there wasn't a border between the code modules, this data would usually just be referenced using a OneToOne or OneToMany from the main data entity anyway and we can almost model this with existing Ebean functionality. The one problem is that we cannot tell the main model to cascade it's deletion, resulting in a foreign key error.
In my head this would need some way of mapping this on the entities and additionally some logic when parsing annotations.
I hope, that I could make my concerns with these ideas clear. If you see something I didn't consider just tell me, maybe this still isn't our way to go.
Reply all
Reply to author
Forward
0 new messages