scala integeration

[Florian Gutmann]

Hello!

I'm currently experimenting with jooq together with scala.

Unfortunately there seem to be many rough edges in everyday usage.

Just some that I stumpled upon:

- DSL.row function not usable from scala due to a known limitation of java interop with Object... vararg in scala.

- Missing first class support for scala types makes certain situations quite cumbersome.

- ResultQuery fetchInto and scala classes don't work together.

Is there anything other integration than jooq-scala that I've just not discovered?

Are there any further plans to improve scala support?

Best regards,

Florian

[Lukas Eder]

Hello,

Like any other API, jOOQ can be used in Scala with the known interoperability limitations. Some comments inline:

2014-02-28 15:40 GMT+01:00 Florian Gutmann <flo...@gmail.com>:

Hello!

I'm currently experimenting with jooq together with scala.

Unfortunately there seem to be many rough edges in everyday usage.

Just some that I stumpled upon:

- DSL.row function not usable from scala due to a known limitation of java interop with Object... vararg in scala.

Can you explain this further?

 

- Missing first class support for scala types makes certain situations quite cumbersome.

Yes. This cannot be fixed without implementing a first class implementation.

 

- ResultQuery fetchInto and scala classes don't work together.

Can you explain this further?

 

Is there anything other integration than jooq-scala that I've just not discovered?

Not yet.

 

Are there any further plans to improve scala support?

There are plans for late 2014, depending on the amount of traction we get for such an integration. By improving Scala support, I mean creating something like "SOOQ". In the mean time, jOOQ will work at most as well as any other Java library in Scala.

Cheers,

Lukas

[Florian Gutmann]

Hi Lukas!

When using DSL.row with anything different then Field parameters, compilation fails with an "ambiguous reference to overloaded definition" error.

For example this code fails:

    row(1, 2)

The reason seems to be that the scala compiler can't find out which of the row methods is most specific.

http://stackoverflow.com/questions/3313929/how-do-i-disambiguate-in-scala-between-methods-with-vararg-and-without

http://stackoverflow.com/questions/2159248/spurious-ambiguous-reference-error-in-scala-2-7-7-compiler-interpreter/2172619

https://issues.scala-lang.org/browse/SI-2991

Sorry, saying ResultQuery fetchInto doesn't work with scala classes was too generic.

It doesn't work with case classes, which was the paradigm I wanted to use.

--
You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/IQ4j8glws9s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Lukas Eder]

2014-02-28 19:02 GMT+01:00 Florian Gutmann <f.gu...@chronimo.com>:

Hi Lukas!

When using DSL.row with anything different then Field parameters, compilation fails with an "ambiguous reference to overloaded definition" error.

For example this code fails:

    row(1, 2)

The reason seems to be that the scala compiler can't find out which of the row methods is most specific.

http://stackoverflow.com/questions/3313929/how-do-i-disambiguate-in-scala-between-methods-with-vararg-and-without

http://stackoverflow.com/questions/2159248/spurious-ambiguous-reference-error-in-scala-2-7-7-compiler-interpreter/2172619

https://issues.scala-lang.org/browse/SI-2991

Thanks for the links. I have created an issue for this. Will investigate more later on:

https://github.com/jOOQ/jOOQ/issues/3087

But I remember that some implicit Scala conversions used for improved Java/Scala interoperability can lead to such issues.

Sorry, saying ResultQuery fetchInto doesn't work with scala classes was too generic.

It doesn't work with case classes, which was the paradigm I wanted to use.

Hmm, what do case classes look like in byte code? Is there any constructor that jOOQ could call? How does it fail, do you get compilation errors, or an exception? I've also created an issue to track this:

https://github.com/jOOQ/jOOQ/issues/3088

Cheers

Lukas

[Lukas Eder]

Hi Florian,

I've been looking into the mapping of case classes... In fact, this is already possible out of the box, if you 

- annotate those case classes with java.beans.ConstructorProperties

- have an exact match between fetched columns and case class fields

Consider the following two queries:

    val books1 =
    dsl().select(T_BOOK.ID, T_BOOK.AUTHOR_ID, T_BOOK.TITLE)
         .from(T_BOOK)
         .orderBy(T_BOOK.ID asc)
         .fetchInto(classOf[BookCase])

    println(books1);

    val books2 =
    dsl().select()
         .from(T_BOOK)
         .orderBy(T_BOOK.ID asc)
         .fetchInto(classOf[BookCaseWithConstructorProperties])

    println(books2);

And these case classes:

case class BookCase(

  id: Int,

  authorId: Int,

  title: String)

case class BookCaseWithConstructorProperties @ConstructorProperties(Array("id", "authorId", "title")) (

  id: Int,

  authorId: Int,

  title: String)

Cheers, Lukas

[Florian Gutmann]

Hi Lukas,

sorry for my late answer. Unfortunately I've been quite busy during the last days.

I suppose you already found out how the internals of case classes look like.

Thanks for your great efforts and investigations!

Hopefully I will find some time to get another look on scala and jOOQ the next days.

If so, I will keep you posted.

Maybe we we'll also see each other on the Vienna Scala & Database jOOQ special edition on April, 7th.

Best regards, Florian

--

You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/IQ4j8glws9s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Lukas Eder]

Hi Florian,

2014-03-10 19:33 GMT+01:00 Florian Gutmann <f.gu...@chronimo.com>:

Hi Lukas,

sorry for my late answer. Unfortunately I've been quite busy during the last days.

No worries!

 

I suppose you already found out how the internals of case classes look like.

Thanks for your great efforts and investigations!

Yes, they're just like any immutable class you'd write in Java, except for the fact that getters are not called get[PropertyName], but just [propertyName]. Luckily, jOOQ's DefaultRecordMpaper already supports such method naming conventions (as documented here http://www.jooq.org/javadoc/latest/org/jooq/impl/DefaultRecordMapper.html).

The problem you were running into is the fact that method / constructor argument names are not available to Java's reflection API. That's a bit of a pity and the reason why this @ConstructorProperties annotation even exists.

 

Hopefully I will find some time to get another look on scala and jOOQ the next days.

If so, I will keep you posted.

That's great! Any feedback is very welcome! I'm sure there's a lot of room for improvements even before we implement more formal Scala support.

 

Maybe we we'll also see each other on the Vienna Scala & Database jOOQ special edition on April, 7th.

Yes, be sure to join my talk!

Cheers

Lukas

[Josh Padnick]

Hello there. I'm chiming in on this thread 10 months later.

I'm about to begin a new project with Play Framework and Scala and would like to use jOOQ over Slick if I can.  Have there been any significant updates in either documentation or Scala integration that Scala users might find helpful since this was originally posted in March 20, 2014?  

Any success stories of Scala users going to production with jOOQ?  I prefer the jOOQ paradigm but am nervous about edge cases I may run into down the line.

Thanks!

[Lukas Eder]

2015-02-03 16:48 GMT+01:00 Josh Padnick <josh.p...@gmail.com>:

Hello there. I'm chiming in on this thread 10 months later.

I'm about to begin a new project with Play Framework and Scala and would like to use jOOQ over Slick if I can.  Have there been any significant updates in either documentation or Scala integration that Scala users might find helpful since this was originally posted in March 20, 2014?

No, it hasn't been a priority in 2014

 

Any success stories of Scala users going to production with jOOQ?

We have a couple of paying customers who have gone live with jOOQ and we haven't heard from them apart from subscription renewal, so I take that as a good sign.

 

I prefer the jOOQ paradigm but am nervous about edge cases I may run into down the line.

The only edge cases that I'm aware of are lack of support for scala.Option and a lack of support for scala.Int et al., although one of the more interesting driving forces behind jOOQ 3.5's org.jooq.Binding API is to enable support for these user types.

Hopefully, other users might see this discussion and chime in, too

[Lukas Eder]

I've just tried the Options again, and in fact the following converter will work:

class StringOptionConverter extends Converter[String, Option[String]] {
  override def from(t : String) : Option[String] =
    if (t == null)
      None
    else
      Some(t)

  override def to(u : Option[String]) : String =
    u match {
      case None => null
      case Some(s) => s
    }

  override def fromType : Class[String] = classOf[String]
  override def toType : Class[Option[String]] = classOf[Option[String]]
}

It can be configured like this:

<customTypes>
    <customType>
        <name>Option[String]</name>
        <type>scala.Option&lt;String></type>
        <converter>org.jooq.scala.StringOptionConverter</converter>
    </customType>
</customTypes>

<forcedTypes>
    <forcedType>
        <name>Option[String]</name>
        <expression>.*\.first_name</expression>
    </forcedType>
</forcedTypes>

This will work, but in my opinion, Option is the wrong tool to model SQL's NULL (or any language's NULL) as it abuses generics for the job. In SQL, and in most languages, it is OK to compare something that is semantically nullable with something that is not. In Java/Scala using generics, Option[T] cannot be compared with T, e.g. by writing NULLABLE_FIELD.eq(NON_NULLABLE_FIELD).

In other words, using generics to model a second-class piece of information such as SQL's NULL / NOT NULL is probably not good.

As far as Int is concerned...

While you could write an IntConverter like this:

class IntConverter extends Converter[java.lang.Integer, Int] {
  override def from(t : java.lang.Integer) : Int = t.intValue()
  override def to(u : Int) : java.lang.Integer = java.lang.Integer.valueOf(u)
  override def fromType : Class[java.lang.Integer] = classOf[java.lang.Integer]
  override def toType : Class[Int] = classOf[Int]
}

You might not be able to use it in the code generator, because the generated code is Java code, and the Converter[Integer, Int] type is translated to Converter<Integer, Object>. In order to be able to use Int and the other "primitive" types from Scala, you'd have to rewrite the code generator to produce actual scala code (https://github.com/jOOQ/jOOQ/issues/4021).

Long story short, this part of the interoperability might not yet be as lean as if jOOQ were written in Scala.

[Josh Padnick]

Lukas,

Thanks so much for your prompt and thorough response.

I agree with your position on Scala Options and SQL Nulls.  This isn't a big deal since my Scala case classes simply don't have to use Option[String] for their types.

The most important thing I care about is workflow and productivity.  My dream is to be able to follow the following workflow:

(1) Thoughtfully design my RDBMS and write out my DDL SQL for FlyWay to execute.

(2) Apply it to Postgres

(3) Run the jOOQ Code Generator

(4) All my POJOs, most of my data access code, and the corresponding unit tests are now written

I probably don't have the time but if I were to write the Scala generator, can you share any thoughts you have about the best approach?  Based on a cursory code review, it looks like this would be as simple as writing a ScalaGenerator.java class that's modeled on the JavaGenerator.java at https://github.com/jOOQ/jOOQ/blob/master/jOOQ-codegen/src/main/java/org/jooq/util/JavaGenerator.java.

Also, when I previously worked with jOOQ, I found that I would have liked to have had jOOQ auto-generate my unit tests for me as well (although in theory, why would you ever need auto-generated unit tests if the code itself is auto-generated).  I wound up spending a lot of my time writing DAOs and their corresponding unit tests, and it would have been nice if I could have just auto-generated a stock DAO and then modified it from there.  Do you see unit test generation as "in scope" for the code generator?  As a native enhancement or a "custom extension"?

Anyway, the good news is it sounds like full-blown Scala support in jOOQ isn't too far off (either in time or in amount of work to be done).  While Slick is the most popular Scala library for ORM, I find the "idiomatic Scala to SQL" thought process quite awkward and opaque in some cases, even if it's intuitively easy to pick up.

Josh

[Lukas Eder]

Hi Josh,

2015-02-04 16:03 GMT+01:00 Josh Padnick <josh.p...@gmail.com>:

Lukas,

Thanks so much for your prompt and thorough response.

I agree with your position on Scala Options and SQL Nulls.  This isn't a big deal since my Scala case classes simply don't have to use Option[String] for their types.

The most important thing I care about is workflow and productivity.  My dream is to be able to follow the following workflow:

(1) Thoughtfully design my RDBMS and write out my DDL SQL for FlyWay to execute.

(2) Apply it to Postgres

(3) Run the jOOQ Code Generator

(4) All my POJOs, most of my data access code, and the corresponding unit tests are now written

OK, so that already works, then :)

Just in case you or other readers weren't aware of this blog post and manual section:

- http://blog.jooq.org/2014/06/25/flyway-and-jooq-for-unbeatable-sql-development-productivity/

- http://www.jooq.org/doc/3.5/manual/getting-started/tutorials/jooq-with-flyway/

 

I probably don't have the time but if I were to write the Scala generator, can you share any thoughts you have about the best approach?  Based on a cursory code review, it looks like this would be as simple as writing a ScalaGenerator.java class that's modeled on the JavaGenerator.java at https://github.com/jOOQ/jOOQ/blob/master/jOOQ-codegen/src/main/java/org/jooq/util/JavaGenerator.java.

Yes, the JavaGenerator is pretty much all there is to it. It can be configured from the code generation configuration. Of course you could also write one completely from scratch, but it would quickly go out of sync with our existing generator, as its internals may change quite a bit between minor releases.

The advantage of a ScalaGenerator.scala would be that you could use Scala's string interpolation for templating. This will make generating Java code a lot easier.

Also, when I previously worked with jOOQ, I found that I would have liked to have had jOOQ auto-generate my unit tests for me as well (although in theory, why would you ever need auto-generated unit tests if the code itself is auto-generated).  I wound up spending a lot of my time writing DAOs and their corresponding unit tests,

Are you aware of:

http://www.jooq.org/doc/latest/manual/code-generation/codegen-daos

 

and it would have been nice if I could have just auto-generated a stock DAO and then modified it from there.  Do you see unit test generation as "in scope" for the code generator?

Well... Unit tests are battle grounds few have returned from and none of them victorious. I'd personally like to stay off that battle grounds for another while before someone can show me a very compelling solution to auto-generating unit tests (would they mock the DB? Or use H2? Or Use MockDataProvider? You'll have millions of flavours...)

Having said so, if you show me: Why not :)

As a native enhancement or a "custom extension"?

Anyway, the good news is it sounds like full-blown Scala support in jOOQ isn't too far off (either in time or in amount of work to be done).  While Slick is the most popular Scala library for ORM, I find the "idiomatic Scala to SQL" thought process quite awkward and opaque in some cases, even if it's intuitively easy to pick up.

I call that the Functional-Relational Impedance Mismatch

Cheers,
Lukas