Ebean and Scala

[Madoc]

Hello,

I used to have great expectations for using Hibernate with Scala.
Since Scala is a JVM language and highly Java compatible, using a Java
ORM appears to be a good choice.

However, I found one huge problem when trying to use that combination:
Scala has different collection types than Java. Hibernate expects
mapped members to be, for example, of type java.util.Set . It's not
really possible to introduce the Scala collection types as so-called
UserCollectionTypeinstances.

I summed up my problems and questions in this thread at the Hibernate
forums:
https://forum.hibernate.org/viewtopic.php?f=1&t=1003746
Please refer to that page; I would find it inappropriate to copy all
the text to this forum.

My impression is that no one at the Hibernate forums answers to Scala
related questions, and there is really no Scala related development
coming any time soon to Hibernate. So I looked around for other ORMs.
This is how I came across Ebean. I was delighted to see that there is
even a Scala example in the Ebean reference documentation. The problem
with the example is that it declares a mapped collection explicitly as
java.util.List .

In the Hibernate forum article above, I wrote why I think it's not
that great an idea to be forced to use Java collection types in ORM-
able objects. And I also wrote down which ideas I see for a possible
solution. Since I got no response from the Hibernate forum, I thought
maybe I could use Ebean.

So, with reference to the Hibernate forum thread linked above, my
question is: Do you see a possibility that the problems which I have
with Scala+Hibernate might be solvable with Scala+Ebean?

Thanks for your attention.
Madoc.

[Rob Bygrave]

I'm pretty busy today ... but I'll try to take a quick look tonight if I can.

Cheers, Rob.

--
To unsubscribe, reply using "remove me" as the subject.

[Rob Bygrave]

Ok, I have had a quick look at your posting and the Scala API.

My instinct is that the best approach would be to provide type conversion/adapters between the java collection types and scala collection types. ('collection' used loosely here as Map is not strictly a java collection).


The 3 main java 'collection' types are:

Interfaces:  List, Set, Map
Underlying implementations: ArrayList, LinkedHashSet and LinkedHashMap.

Note: that we used the LinkedHashSet and LinkedHashMap as we want to maintain the order these are populated.


These 3 underlying implementations are wrapped by Ebean objects BeanList, BeanSet and BeanMap (which implement Ebean's BeanCollection interface). The main purpose of these wrappers is to:
  a) Invoke lazy loading when required
  b) Add 'mutation' listening for ManyToMany and 'delete orphans'/'private owned'


I'd suggest what would be required to do would be:

1. Identify the Scala collection types (Traits proabably) that you want to use in your scala beans. I'd expect these to map to List, Set and Map. ( I'd expect these to be Scala Buffer, Set and Map respectively based on the scala JavaConversion object).

2. Use Scala's JavaConversions to wrap the java List, Set and Map types. (This means the lazy loading and mutation listening built into BeanList, BeanSet and BeanMap gets reused).


Looking at the Scala JavaConversions object suggests java.util.List converts to scala Buffer. Java's Set and Map convert over to scala's mutable Set and Map.

The interesting methods in JavaConversions to convert to/from the appropriate types are:

asBuffer()      // convert java list to scala buffer
toList();        // convert scala buffer to java list

toSet()
asSet();

toMap();
asSet();


So, I'd suggest the way to support this would be for Ebean to identify the 'scala collection types' (scala buffer, set and map) when detecting the bean properties and their types. For these properties we need to additionally wrap the BeanList, BeanSet and BeanMap (using scala JavaConversions) into the matching scala types.


Does that sound like what you are after?


Cheers, Rob.

[Matthias]

Hi Rob, that sounds perfect. And it was exactly what I was trying with Hibernate when I wanted to use Hibernate's UserCollectionType. Unfortunately, as I wrote it did not work out the way I expected.

There are two more things to take into account, maybe: One is the Scala Option type, and the other is immutable collections.

The Scala Option type wraps an object reference that is either given (called "Some"), or not given (called "None"). It is used where a Java developer would use a nullable value, except that Scala developers don't like null values. Conceptually, you can think of it as a list that can only have zero or one element.

Immutable collections are, I believe, used more often in Scala than in Java. In Java, you can easily convert a regular collection to an immutable collection by calling methods like Collections.unmodifiablList, for example. In Java, the immutable collection has exactly the same interface as the regular, mutable collection. But it will throw exceptions at runtime when a modifying method gets called. This is different in Scala: There, unmodifiable collections have different interfaces/traits. They don't have modifying methods at all.

It would be important to design the Scala-Ebean "bridge" such that immutable collections would also be supported. This would allow Scala developers to work with Ebean in a very natural, Scala-esque way. This would undoubtedly make Ebean the only JPA-compliant ORM for Scala at the current time that allows the Scala developer to develop in the Scala-typical programming style.

Rob, what would you think: How can we go on from here? I do not know the internals of Ebean at all, and I don't feel competent to make such deep changes to the system. However, I like Hibernate's idea of making custom collection types "pluggable", like with the UserCollectionType interface I mentioned. But UserCollectionType made the impression to me of being rather complex and hard to understand. Also, a UserCollectionType in Hibernate would require every type member that is of a Scala collection type to be annotated with at least one Hibernate annotation. My personal preference would look like this: Ebean would allow me to define my own collection types in a central place just once, and no further annotation/configuration would be required per member anymore. Through that feature, I could effectively enable Ebean to treat my custom collection types like the Java collection types it already knows. Then, not only Scala collection types could be added by the user, but also all other kinds of collection types from other frameworks and whatnot.

What do you think would make more sense: Such a general "pluggable" feature, or a feature that would be a special solution just for the Scala collection types?

2010/4/11 Rob Bygrave <robin....@gmail.com>

[Daryl Stultz]

On Apr 11, 10:53 am, Matthias <madocd...@gmail.com> wrote:
>This would undoubtedly make Ebean the only JPA-compliant ORM

This is a bit off topic, but I'm wondering what it means to call Ebean
"JPA compliant"?

/Daryl

[Rob Bygrave]

> I'm wondering what it means to call Ebean "JPA compliant"?

This is an important point. Ebean can't be called 'JPA compliant' and we don't want to give the impression we are or will be 'JPA compliant'.

Interestingly, this issue can up amongst the Ebean 'committers' just recently (whether to support a JPA EntityManager etc). I can't say it won't ever happen (that Ebean will become JPA compliant) but in my opinion there is too many fundamental differences in the architecture and query language for this to happen.

Ebean does want to use the same JPA Mapping and does want to be as similar as possible but at the same time the fundamental differences are the reason Ebean exists. So to be clear, Ebean is not trying to be a JPA implementation and probably never will be.

--

[Matthias]

Sorry about that, I wasn't aware of it when I wrote it. This is because JPA compliance is not really that important for me.

I really meant something like "an ORM that is roughly like Hibernate", as transparent as possible. I wanted to exclude ORMs like ActiveObject and thought that "JPA compliant" would do the job. But you are right, it made my statement wrong.

2010/4/11 Rob Bygrave <robin....@gmail.com>

[Rob Bygrave]

> Hi Rob, that sounds perfect.

That's good because I implemented it last night (Using Scala 2.8 due to significant changes to the collections).  Aka you can use Scala mutable Buffer, Set and Map (instead of java List, Set and Map).

I also added a findBuffer() method (to Query etc) that returns a Scala mutable Buffer.

You can get the latest 2.6.0-SNAPSHOT from the archiva repository to have a play.

http://www.avaje.org/archiva/repository/snapshots/org/avaje/ebean/2.6.0-SNAPSHOT/
http://www.avaje.org/archiva/repository/snapshots/org/avaje/ebean/2.6.0-SNAPSHOT/ebean-2.6.0-20100411.134743-2.jar

> Such a general "pluggable" feature, or a feature that would be a special solution just for the Scala collection types?

At the moment Ebean is not aware of the Scala collection types so not really pluggable at this point (but could be without too much effort). I think we can leave it like this for the moment and if we need this to be pluggable look at that later (we don't need this for Groovy right?).


> Option type

Yup, I'm aware of the option type - I'll have a look at how to support that.

> immutable collections would also be supported

Is there a good / best way to convert the scala mutable types to immutable ?

[Rob Bygrave]

So the current scala test code I'm using is:

package test.s.m


//import _root_.java.util.List
import javax.persistence._
import com.avaje.ebean._
import scala.reflect._
import scala.collection.mutable._

 /**
  * Customer entity bean.
  */ 
 @Entity 
 @Table(name="o_customer") 
 class SCustomer { 
  
   @Id  
   var id:Int = 0 
      
   var name:String = null   
  
   var smallnote:String = null   
     
   @OneToMany(mappedBy="customer")
   @JoinColumn(name="customer_id")
   var contacts:Set[SContact] = null;
   //var contacts:_root_.java.util.List[SContact] = null 

 }

 object SCustomer extends SDao(classOf[SCustomer]) {
                              
 }



// --------------------


package test.s.m

import javax.persistence._

@Entity
@Table(name="contact")
class SContact {

   @Id
   var id:Int = 0 
  
   var firstName:String = null
  
   var lastName:String = null
  
   @ManyToOne
   var customer:SCustomer = null;
}

object SContact extends SDao(classOf[SContact]) {
 
}


// ----------------------

package test.s.m

import java.util.logging.Logger
import com.avaje.ebean._

/**
 * Dao for a given Entity bean type.
 */
abstract class SDao[T](cls:Class[T]) {
 
  private val ebeanServer = Ebean.getServer(null);
 
  /**
   * Find by Id.
   */
  def find(id:Any):T = {
    return ebeanServer.find(cls, id)
  }
 
  /**
   * Find with expressions and joins etc.
   */
  def find():com.avaje.ebean.Query[T] = {
    return ebeanServer.find(cls)
  }

  /**
   * Return a reference.
   */
  def ref(id:Any):T = {
    return ebeanServer.getReference(cls, id)
  }
 
 
  /**
   * Save (insert or update).
   */
  def save(o:Any):Unit = {
    ebeanServer.save(o);
  }
 
  /**
   * Delete.
   */
  def delete(o:Any):Unit = {
    ebeanServer.delete(o);
  }
}


// ------------------------


package test.s.m

import com.avaje.ebean._
import com.avaje.ebean.config._
import scala.collection._
 
object STestApp {
 
  def main(args : Array[String]) : Unit = {
   
     
      //GlobalProperties.put("ebean.search.packages","test.s..m")
     
      val p = SCustomer.find(1);
      if (p != null){
          println("got "+p.id + " "+p.name)
          p.name = "X2 Changed Name";
          SCustomer.save(p);
      }

      val personList = SCustomer.find()
           .fetch("contacts")
           .where().gt("id",0).findBuffer();
     
      println("personList"+personList)
     
     
      val perBuf = personList;//JavaConversions.asBuffer(personList);
     
      for (p <- perBuf) {
          println(" "+p.id +" "+ p.name )
          for (c <- p.contacts) {
              println(" contact "+c.id+" "+c.firstName +" "+c.lastName );
          }
      }
     
     
      println("done")
   
  }
}



/// ------------

Using Scala 2.8 beta.  Probably not going to support 2.7.7 due to changes in the scala collections API.

On Mon, Apr 12, 2010 at 2:53 AM, Matthias <mado...@gmail.com> wrote:

[MadocDoyu]

Rob, unfortunately it is bed time over here. I'm really excited about your posting and can't wait to try it out tomorrow (as soon as my job will allow).

All in all, I'm very thankful for your quick response and even a first approach to a solution in such a short time. Wow!

---

From: Rob Bygrave <robin....@gmail.com>

Date: Mon, 12 Apr 2010 09:41:17 +1200

To: <eb...@googlegroups.com>

Subject: Re: [ebean] Ebean and Scala

[Rob Bygrave]

> Sorry about that, I wasn't aware of it when I wrote it.

No problem. I just don't want to give people the wrong impression (that Ebean is or will be a JPA compliant ORM). Aka, we have a 'different opinion' about what ORM should be like compared to the JPA expert group.  Unfortunately we have not done a good job explaining this :(

Ebean could be described as being "session-less" or having "automatic management of the persistence context" (the API looks/acts in a stateless way - no merge/persist/flush etc) so hopefully some people realise there are many reasons to use Ebean over session based ORM's like Hibernate and the other JPA implementations.

[Daryl Stultz]

On Apr 11, 5:26 pm, Rob Bygrave <robin.bygr...@gmail.com> wrote:

> Interestingly, this issue can up amongst the Ebean 'committers' just
> recently (whether to support a JPA EntityManager etc).

Noooooooooooooooooooo! If this comes up for a vote I'll use all my
credits against it! :-)

> Ebean does want to use the same JPA Mapping and does want to be as similar
> as possible but at the same time the fundamental differences are the reason
> Ebean exists.

Exactly. That's what drew me to Ebean. I hate "attached/detached"
entities. (And the many great features has me continuing to work with
it in the hopes I can prove everything out and "sell" it to my boss.)

/Daryl

[Rob Bygrave]

> Noooooooooooooooooooo! If this comes up for a vote I'll use all my credits against it! :-)

No need to worry :)  I think the "attached/detached" refugees are starting to find Ebean. In my opinion our marketing department (me) hasn't done a great job in getting the word out so if people can help there that would be great.

Cheers, Rob.

/Daryl

[Rob Bygrave]

BTW: I have added support for Scala Option types so now you can go:

  
   var smallnote:Option[String] = None;

rather than:

   var smallnote:String = null;


... so that pretty much leaves out ... scala immutable collections and there is no findScalaSet() or findScalaMap() methods yet but yes, should be enough to play with and test it out in a larger scala model.


Cheers, Rob.

[Rien Nentjes]

Btw, the following addition to the pom broke the build (for me at least):

        <dependency>
            <groupId>org.scala</groupId>
            <artifactId>library</artifactId>
            <version>2.8</version>
            <scope>compile</scope>
        </dependency>    

I changed it to this as that seems to fix it (don't know much about maven);

        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.8.0.Beta1-RC8</version>
            <scope>compile</scope>
        </dependency>  

Rien

[Rob Bygrave]

Thanks, yes - I cheated and installed the scala-library on my local maven as I couldn't find the maven dependancy. Obviously I didn't look hard enough.

Cheers, Rob.

[Rien Nentjes]

Latest version still broken, my lack of maven knowledge doesn't seem to be the problem:

Caused by: java.lang.NoClassDefFoundError: scala/Option
        at com.avaje.ebeaninternal.server.deploy.parse.DeployCreateProperties.createProp(DeployCreateProperties.java:296)
        at com.avaje.ebeaninternal.server.deploy.parse.DeployCreateProperties.createProp(DeployCreateProperties.java:361)
        at com.avaje.ebeaninternal.server.deploy.parse.DeployCreateProperties.createProperties(DeployCreateProperties.java:152)
        at com.avaje.ebeaninternal.server.deploy.parse.DeployCreateProperties.createProperties(DeployCreateProperties.java:78)


There now seems to be a runtime dependency on scala library...

I really liked the fact that Ebean was lean and mean and doesn't have a dozen dependencies. Might it be an idea to add this functionality as an add-on and do the type checks on fully qualified class names as strings or something (no idea if that will work)?

For now I added the scala-library.jar to my war file and that fixes it.

Rien

[Odd Möller]

The latest Scala Beta 1 release is 2.8.0.Beta1 (without an RC qualifier) released on 2010-01-27:

<dependency>

  <groupId>org.scala-lang</groupId>

  <artifactId>scala-library</artifactId>

  <version>2.8.0.Beta1</version>

</dependency>

You can find it here:

<repository>

  <id>releases.scala-tools.org</id>

  <name>Scala-Tools Maven2 Repository</name>

  <url>http://scala-tools.org/repo-releases</url>

</repository>

Regards

Odd Möller

On Mon, Apr 12, 2010 at 10:13 AM, Rien Nentjes <rnen...@gmail.com> wrote:

[Rob Bygrave]

My bad.

I don't think we need or want a runtime dependency on the scala library.

I'll fix that.

[Rob Bygrave]

> version>2.8.0.Beta1

Thanks - I have updated the pom. It seems to have got to the general maven repos.

[Rob Bygrave]

> Caused by: java.lang.NoClassDefFoundError: scala/Option

Ok, according to me I have fixed this now (in HEAD) .

Thanks, Rob.

On Mon, Apr 12, 2010 at 8:42 PM, Rien Nentjes <rnen...@gmail.com> wrote:

[Rien Nentjes]

Works for me without the scala lib, thx!

Rien

[Matthias]

Hi Rob,

I just tried your snapshot, and I'm really amazed that it works. I'm still very new to Ebean, so I had to fix some beginner's mistakes of mine, but after that it went really smooth. I managed to replicate your example.

At the moment Ebean is not aware of the Scala collection types so not really pluggable at this point (but could be without too much effort). I think we can leave it like this for the moment and if we need this to be pluggable look at that later (we don't need this for Groovy right?).

Agreed.

Is there a good / best way to convert the scala mutable types to immutable ?

You could of course easily convert a mutable set x to an immutable set y by calling:

val y = scala.collection.immutable.Set.empty ++ x

In this case, y would be a snapshot of the current state of x. Subsequent changes to x would not have an effect on y.

Would this be sufficient? Because if you want the immutable set to reflect subsequent changes on the orignal mutable set, things get confusing: What if you remove an element from the immutable set (or, more precisely, re-assign the reference to the immutable set to a new immutable set that has all the elements of the immutable set minus the removed element), and later on that element gets added to the underlying mutable set - what would you expect to happen?

Therefore, if Ebean detects a change to an immutable collection, it would be best to re-assign the member variable / property that contains that collection with the updated version instead of modifying the immutable set. Is this possible?

I am really amazed by Ebeans and by your support of Scala. I am looking forward to use Ebeans in my Scala project. Also, I will put a reference to this discussion in the discussions on the Hibernate forum and on the Scala mailing list, so that other people researching the same topic can find this thread. Everyone must make his own decision based on his preferences and requirements, but for me, it looks as if Ebeans is an excellent fit.

If there is anything I can do to help (except for answering questions and motivating), please tell me. I am looking forward to the support for immutable collections.

Madoc.

2010/4/11 Rob Bygrave <robin....@gmail.com>

So the current scala test code I'm using is:

[...]

[Rob Bygrave]

> You could of course easily convert a mutable set x to an immutable set y by calling:

   val y = scala.collection.immutable.Set.empty ++ x

Ebean is written in Java has to do the same thing using Java code. The question is, how do you do that in Java? Scala has provided the JavaConversions object to help but it doesn't cover the immutable types.

Note that with the mutable types they are actually wrappers around the Java ones so Ebean is literally wrapping them and then later from the scala types getting the "underlying()" java List, Set or Map. It doesn't look like similar wrapping support is available for the scala immutable collections.  To me that could be a enhancement request to scala - for JavaConversion to have methods to wrap as immutable collections (and where Java code can later get back the underlying java list, set or map).

> I am really amazed by Ebeans and by your support of Scala.

I think it suggests Ebean's type system is in reasonable shape which is good.

[Matthias]

> You could of course easily convert a mutable set x to an immutable set y by calling:

   val y = scala.collection.immutable.Set.empty ++ x

Ebean is written in Java has to do the same thing using Java code. The question is, how do you do that in Java? Scala has provided the JavaConversions object to help but it doesn't cover the immutable types.

Note that with the mutable types they are actually wrappers around the Java ones so Ebean is literally wrapping them and then later from the scala types getting the "underlying()" java List, Set or Map. It doesn't look like similar wrapping support is available for the scala immutable collections.  To me that could be a enhancement request to scala - for JavaConversion to have methods to wrap as immutable collections (and where Java code can later get back the underlying java list, set or map).

I played around with the Scala library today, trying to create a Scala Set subclass in Java. However, I got confusing compiler errors. (For example, when trying to subclass the Scala HashSet in Java, you will find that Java does not like inheriting two interfaces that both define a method "apply()", where in one interface its return type is boolean and in the other interface, it is java.lang.Object. Scala seems to be okay with that.) I'm not sure if that's even possible. But even if it were possible, for example, to create a scala.collection.immutable.Set implementation in Java, you would need to forward all methods that the Scala trait implements to Set$class, following the pattern:

  public X someMethod() { return Set$class.someMethod(this); }

That's of course because Java can't handle Scala's multiple implementation inheritance via traits, therefore Scala provides for each Scala trait T the Java class T$class that contains all the trait's methods as static Java methods that receive the trait instance as first argument.

When one would do so, you would get a lot of methods that only delegate, and when a new Scala version would add one or to methods to the Set trait (or one of its super-traits or interfaces), you would have to manually add those to the Java classes as well. This would make it hard on the long run for Ebean to stay compatible to different Scala versions, unless you split up the Ebean release into different sub-versions for different Scala versions.

It might be that I'm wrong here, because I am still pretty new to Scala (who is not?). But this is the impression that I currently get. I would assume, no matter which way you put it, trying to implement or subclass Scala collection types in Java will open the door to some trouble. It seems much easier to me to subclass those collection types in Scala itself and then simply add the generated "*.class" files to the Ebean release. But maybe that's not as easy as it sounds to me because I don't know about all the details that this might imply for Ebean. What would you think, Rob?

> I am really amazed by Ebeans and by your support of Scala.

I think it suggests Ebean's type system is in reasonable shape which is good.

Totally true.

[Rien]

Hi,

Still problems on Ebean.save():

java.lang.NoClassDefFoundError: scala/Option
at
com.avaje.ebeaninternal.server.persist.DmlUtil.isNullOrZero(DmlUtil.java:
20)
at
com.avaje.ebeaninternal.server.persist.dml.InsertHandler.bind(InsertHandler.java:
90)
at
com.avaje.ebeaninternal.server.persist.dml.DmlBeanPersister.execute(DmlBeanPersister.java:
97)
at
com.avaje.ebeaninternal.server.persist.dml.DmlBeanPersister.insert(DmlBeanPersister.java:
75)
at
com.avaje.ebeaninternal.server.persist.DefaultPersistExecute.executeInsertBean(DefaultPersistExecute.java:
100)
at
com.avaje.ebeaninternal.server.core.PersistRequestBean.executeNow(PersistRequestBean.java:
400)
at
com.avaje.ebeaninternal.server.core.PersistRequestBean.executeOrQueue(PersistRequestBean.java:
430)
at
com.avaje.ebeaninternal.server.persist.DefaultPersister.insert(DefaultPersister.java:
369)
at
com.avaje.ebeaninternal.server.persist.DefaultPersister.saveEnhanced(DefaultPersister.java:
313)
at
com.avaje.ebeaninternal.server.persist.DefaultPersister.saveRecurse(DefaultPersister.java:
283)
at
com.avaje.ebeaninternal.server.persist.DefaultPersister.save(DefaultPersister.java:
267)
at
com.avaje.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:
1480)
at
com.avaje.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:
1470)
at com.avaje.ebean.Ebean.save(Ebean.java:535)

Rien

[Rob Bygrave]

Thanks, that should be fixed in HEAD now.

[bootjack2k]

Did this make it into trunk? I'm using 2.7.4 and I don't see findBuffer in Query.

Thanks!

- Henry

[Rob Bygrave]

Not sure what you mean by "Did this make it into trunk" - can you expand on what you mean.

No, there is no findBuffer method on Query - and I don't really intend to add one as that would add a non-optional dependency to Scala.

Cheers, Rob.