Advice on Slick vs. Hibernate

[Peter Hancox]

Hoping for some high-level advice on Hibernate vs. Slick and current Slick capabilities.

We are currently looking at a revised technology stack for the next generation of our 

application framework.  Moving from Java to Scala seems to be almost a certainty, 

however, not quite as certain on the persistence approach.

Current application framework built approximately 10 years ago in Java includes a 

custom ORM that has served us well.  When looking at the new technology stack the 

first option was obviously Hibernate which based on a very basic proof of concept 

seems to be viable.  And research seems to indicate that for the most part, a 

Hibernate based solution would play reasonably well with Scala.

We then came across Slick which would appear to be much better integrated with 

Scala (obviously) and technically a solution which can only continue to get better.

Unfortunately my experience with Scala is less than a few weeks and similarly for 

Hibernate; only a few weeks including the architecture proof of concept.  Slick 

experience is only a few days and I'm continually looking for real world examples, 

particularly some more CRUD oriented code rather than looking at different variants 

on more complicated queries.

For example, our custom ORM framework understands relationships between objects e.g., 

person has a home address and work address, addresses have suburbs, etc.  So when you 

call "save" on a person object the executed SQL manages the multiple tables, foreign 

keys etc.  It also handles only updating changed columns.  It appears that Hibernate 

does the same but haven't played enough with it yet to be sure.

Slick on the other hand appears to take care of database portability, mapping complex 

queries directly to Scala collections etc.  However, I can't see that it currently 

supplies much at the higher level of automatically performing all the updates/inserts 

necessary for example when calling "person.save()"?  What about only updating changed 

columns? Would the application programmer / library developer have to write most of 

this code?

Is this a fair assessment or have I missed some of the current functionality?  

Unfortunately my rudimentary understanding of Scala at this point is hindering 

my understanding of what Slick is currently capable of.

Based on my current understanding I still look like using Slick for the persistence 

layer.  Hibernate may well be a closer fit to our current ORM but over time it's not 

going to integrate any better.  Slick will have a considerably longer learning curve, 

but the Scala we're going to have to learn anyway.  Slick will integrate better and 

over time will catch up in terms of functionality.  There's also the future back-end 

support for NoSQL and other data sources such as web services.  Worst case we're back 

to a custom ORM but built on top of Slick rather than Java which will offer us better 

ongoing integration in the long term.

As you can probably tell I'm trying to convince myself that Slick is the way to go :-) 

even though probably much more work in the short term.  Any comments / guidance would 

be much appreciated.  I'd be particularly keen to be pointed at some examples of a CRUD 

type framework e.g., support for code similar to 

val person = Persons.byId(1234).first

val suburb = Suburbs.byName("suburbname").first

val address = Address("Street Line 1",suburb)

person.homeAddess = address

person.save

The "person.save" should do an insert into address table followed by update of only the 

home_address column in the person table.

[virtualeyes]

Really you should be looking at Grails if you're in need of Hibernate-style persistence, dirty fields, etc.

Slick is not an ORM, thankfully ;-)

If your team is taking the Scala route then try to shoehorn Hibernate into the stack, or give it a go with Slick. With the latter you'll likely be taking a different approach, some might say a better approach, but YMMV.

Start with Scala and it will all fall into place.

[Peter Hancox]

Thanks very much for your insight, this is precisely the input I'm looking for at this stage.

Hadn't really investigated Grails but will do so now.

You say that "Slick is not an ORM" which is pretty much the general view that my research has shown.  So what is 

Slick positioned as?  "persistence layer"? some other term?

If I take the view that I need an ORM (ala Hibernate/Grails) is it a good idea to attempt to build it on top of Slick?  I have 

the background on what is required for the ORM from our current custom implementation, just lack the Scala skills for

a good implementation.  However, provided it's heading in the right direction, speed of delivery isn't that big a deal and

time can be taken to get it right.  Are you aware of any current initiatives to implement ORM on top of Slick?

Or would your view be that if we're looking at an ORM in the Scala (functional programming) world then we're probably 

headed down the wrong path anyway?  If so, can you point me toward some good reference material on persistence in

the world of functional programming?  Currently in the middle of the Coursera course on functional programming but

need to find some guidance on how best to do persistance.

[SK]

Here is my 2 cents.

I've worked with Scala and Hibernate for a couple of years. It does work but it is not a very pretty sight. There are several reasons:

* Java collections like List, ArrayList, (and hibernate collections inherently) feels "clunky" in Scala. In the beginning Java api's feels safe and good, but after a while the feels odd. At least if you turn purely functional. 

* Hibernate got no real concept of Option. You may get around it, but that tends to "clutter" your code.

* I love case classes. No vars. Just vals. No side effects. It's simple. It's thread safe. This isn't Hibernate. It's Slick.

We started with a team with limited Scala knowledge. The foundation was quite a lot of Java and Hibernate (and some other standard Java technologies like SpringFramework). To keep the number of new technologies to minimum we kept on going with Hibernate. Scala has scaled (...just couldn't resist...) the way we think. We do things differently today then two years ago. We have turned more functional. 

In the end it really depends on what you are trying to make and your team. 

Good luck! 

SK

[Peter Hancox]

For the moment we'll probably be trying out a combination of approaches.

I like the simplicity of using Hibernate for simple CRUD type updates.  e.g., update the object, tell it to persist, and have the framework automatically handle dirty fields, references to associated objects etc.  At this stage I'm working under the assumption that Hibernate is capable of doing everything our current custom Java ORM can do.  If it can't then we'll look to re-implement our current custom functionality in Scala and probably on top of Slick.

For more complicated queries (probably anything that returns multiple rows), the flexibility of Slick seems ideal.

My challenge at the moment is to come up with a standard approach/template for entity classes that can be used by both approaches with minimal boilerplate / duplicated code in the applications.

Gotta love Scala as a language for developing libraries/frameworks!!!  Wish it was publicly available 10 years ago when we were writing our current Java framework.

Is it worthwhile looking at Scala ORMs?  Any recommendations or comments based on past experience?

Thanks for your feedback.

Peter 

[Michael Slinn]

Apparently database portability is not a goal of the Slick project. I wish it was.

Mike

[nafg]

On Monday, May 6, 2013 10:05:33 AM UTC-4, Michael Slinn wrote:

Apparently database portability is not a goal of the Slick project. I wish it was.

Mike

Sorry but that just isn't true. As long as you don't need features particular to one database (which is means your app is not database-portable) you can just abstract over the database type.

[Mike Slinn]

This topic was recently discussed at length (italics are mine):
"We provide behavioral abstraction where we can and where we see demand (meaning missing abstraction is a feature request, not a bug). We doubt that complete abstraction can be realistically achieved for a library with powerful queries like Slick and would certainly consume significant human resources which we do not have. The same features sometimes have subtle differences in their behavior in different databases and often accordingly in our backends. The details of these differences partly depend on the configuration of the database server, which Slick doesn't know about."

Mike

[Naftoli Gugenheim]

Yes, I followed the thread. Cvogt was referring to those features that some database provide and others don't Anyone can see that you can write code to ExtendedDriver, and then pass in specific driver instances.

[virtualeyes]

They're basically saying, hey, we're just 2 or 3 developers here, full abstraction is not happening; if there's missing functionality (or a feature specific to a particular DBMS), provide it yourself.

Seems perfectly reasonable, limited resources, Rome will not be built in a day.

[Grant Beaty]

Peter,

You may want to look at Activate: http://activate-framework.org/

It has some rough edges (its a low newer than Slick/ScalaQuery, and relies more on run-time exceptions), but is very impressive in what it does. It is an ORM, with all the baggage that implies. You don't even need to call object.save(), all changes in a transaction are automatically persisted to memory and then the backing store. Being STM-based, concurrency "just works" at a much more granular level than the table-level locks of many RDBMSs. Most of the time it feels like you're just mutating regular old mutable objects in a normal OOP manner. Scala has better access controls than Java, so you can make good use of those and much of the OOP stuff you probably tried to forget when moving to Scala.

Of course this is a double-edged sword. I get to make persistence changes in a normal way using normal variable assignment, but I don't get immutable query results for free. I do a lot of handling of real-time updates, so I have to manually turn my mutable models into immutable case-classes that can be accessed non-transactionally, which adds some verbosity.

I tried both Slick and Activate in my project, and ended up going with the later due to a very high update/insert query volume. Hitting the database on every update just wasn't going to work for me. I think my domain model is considerably simpler with Activate than it was with Slick, partially because I don't have to think of how to combine and compose queries. However I do have less type-safety to deal with, which has made debugging more difficult. It also doesn't automatically generate schemas at this point.

Unsurprising given that it uses its STM layer as a cache, Activate is much faster than Slick (at least in this benchmark, which was admittedly written by the Activate author):

http://databen.ch/

My application is a bit unique, as I basically only operate on a small subset of my data on any given day. Its all kept in-memory by Activate until its no longer "live", so I don't really have to worry about things like the n+1 problem. I'm not sure if I'd prefer Activate in a more traditional app or not.

/Grant

[Peter Hancox]

Hi Grant and thanks very much for your feedback.

When I first came across Activate, I was a bit "frightened off" by having the STM as yet

another layer in the technology stack to deal with, particularly as it didn't seem to be 

offering me much.  However, although I haven't yet had much time to look closer, it seems,

from a somewhat superficial understanding, that this would be reasonably transparent

to the application code?

I'm currently putting together a proof of concept where we're using Slick to generate the

the DDL for creating the database schema.  And using MapperDao for the ORM layer.

Once I get a few sample entities in place, I'll attempt to integrate Activate into the stack.

I'll then attempt the same with Hibernate.  Once I've got them all playing nicely in the

framework technology stack, I'll see which fits best for our applications.  Chances are

we'll end up with a combination of products as so far none of them can meet all requirements.

The framework needs to support multiple deployment approaches.  Our main application 

is a Java desktop application with a rich domain model and the new technology stack is

going to have to support that deployment approach and most likely share a database.

Some new applications will be web-based.  However, in most cases the applications are

going to have rich/complex domain model but low volume and minimal concurrency

requirements.  i.e., STM probably isn't needed but no problems using it if it doesn't

overly complicate things.

It sounds as if you're pretty happy with Activate as an ORM solution?  Have you been using

it for long?  Come across many "gotchas"? Have they been difficult to resolve?  Is this a

web based application?  How large is the database and do you see Activate as potentially

having problems with larger databases?

REGARDS

Peter

[Andrew Gaydenko]

On Monday, May 13, 2013 11:18:35 PM UTC+4, Grant Beaty wrote:

You may want to look at Activate: http://activate-framework.org/

Indeed exciting framework! Unfortunately at that moment I was digging in it, it was impossible to use it for already existing data with predefined primary keys type (string at my case), but Slick has suited ideally here.

It would be also interesting to know something about activate benchmarking at the case when data size is many times exceeding RAM size.

[Grant Beaty]

Peter,

For the most part the STM is transparent. There are really only two things to keep in mind when using it: 1) Wrap all reads and writes on entities in transactions, and 2) Don't put any side-effects (such as logging statements) inside transactions, because they can be repeated when a rollback occurs.

The biggest "gotcha" I've run into is memory usage. I've not yet used it in production, but will have by the end of the month.

Anyway if you want to discuss Activate further, we should probably do so on its mailing list ;) It sounds as if your application would not take much advantage of Activate's strengths, though.

/Grant

[Christopher Vogt]

For completeness the code

val person = Persons.byId(1234).first

val suburb = Suburbs.byName("suburbname").first

val address = Address("Street Line 1",suburb)

person.homeAddess = address

person.save

would look something like this with Slick

db.withTransaction{ implicit session: Session =>

   val suburb_id = Suburbs.filter(_.name === “suburbname”).map(_.id).first

   val address_id = Addresses.insert(Address(None,“Street Line 1”,suburb_id))

   Persons.filter(_.name === 1234).map(_.home_address_id).update(address_id)

} // <- save happens here

You work on the database tables directly, not on a local cache which then needs to be saved back to the db (.save). The and re-usable parts of these queries like the .filter calls can be pulled out into methods, if you wish so (e.g. byId and byName, which do not come with Slick). Make sure you read http://slick.typesafe.com/doc/1.0.1/lifted-embedding.html when trying this code, especially the section about inserts.

More in our Slick vs. ORM talk at Scala Days next week :).

Chris

[Sam Mueller]

One problem we've run into when evaluating slick is how poorly it generates SQL queries.  Just try to left join more than one table and you'll see what I mean.  There's details here:

http://grokbase.com/t/gg/scalaquery/12b85es21e/slick-0-11-2-joins-result-in-nested-selects

To me, left joins are super-basic when it comes to querying a relational db, and it leaves me wondering what more complex queries have problems in slick.  Honestly, I was hoping that slick would have prioritized query generation higher, since most companies view this as the most important aspect when using a data abstraction (ORM or otherwise).  Now I'm forced to look at solutions that aren't idiomatic to scala (such as hibernate), or something like activate.  Coming from the .NET world, I'm really missing the elegance and power of LINQ [to SQL], which has been available and stable for many many years.

Will this improve with direct embedding?  I haven't checked out the improvements in 2.0.0 M1, but maybe there's hope?

--Sam

[virtualeyes]

Slick generates proper SQL for the general case; it's the multiple outer join and other less common scenarios where you may think about doing sql"" string interpolated queries if performance is critical.

Crazy idea, but if your background is .NET have you considered hitting a Mono + F#/C# + LINQ web service and sending JSON as the response?

I know with the new Scala Pickling (or Pickler?) library there is some potential around more or less easily marshaling one type (JSON) to another (Scala case class) .

Too bad that Scala .NET was dropped, could have done some interesting polyglot setups all on a single platform.

[Mike Slinn]

IKVM (http://www.ikvm.net/) works incredibly well for running JVM code on .NET and interoperating. I was astounded by how good it was.

Mike

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "Slick / ScalaQuery" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scalaquery/aOyzJ30Ja_0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scalaquery+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Sam Mueller]

Thanks for the suggestions, but .NET is not an option anymore due to company-wide standards.  In the JVM world, scala feels very much like c# with it's OO roots and powerful functional constructs.  However, we've had so much trouble getting data in and out on both sides of the stack with json [de]serialization and sql crud operations.  The libraries aren't very mature, and it's a shame because there's so much potential here.

[Mike Slinn]

Sam,

The standalone Json library from Play 2.1.1 is terrific. @mandubian has done a great job.

SQL CRUD I agree is not there. Not a word about it in the latest Play roadmap, either.

Mike

[Sam Mueller]

Again, I'm going to have to disagree with you on the json library.  Just to give you an example, let's say I have this case class:

case class Entity (id: Int, name: String)

And let's say my repository returns a list of these entities.  Now I want to wrap this in a clean generic way with another case class:

case class EntityWrapper[T](count: Int, records: List[T])

To get this serialized into json, I have to write some implicit conversions:

implicit val eFormat = Json.format[Entity]

implicit val ewFormat = Json.format[EntityWrapper]

It would be so great if this worked, but it doesnt.  So now what?  How do I figure out the proper way to create the implicit read/write formatters?  Any and all examples that I've tried so far don't work.  There's no way to find out the best practices in writing formatters.  I challenge you to figure it out!

Trust me, it should not be this hard for such a simple use case.  Just like it shouldn't be so hard to generate a few efficient left joins in slick.

--Sam

[Simon Ochsenreither]

Regarding serialization:

As mentioned, I suggest looking into http://lampwww.epfl.ch/~hmiller/pickling/. Imho, it improves the state of the art considerably.


Concerning query generation:

I agree that a lot of work is still needed, but I think the right people are aware of it (and in charge).
I'm not a huge user of Slick, but it has come a long way since ScalaQuery. The Slick team has to manage various deep architectural changes (lifted vs. direct embedding, macros, NoSQL support, distributed queries, ...) and I think they are doing it pretty well. I expect that as soon something stable including these changes is shipped, more people will hit similar issues and will focus on better query generation.

[virtualeyes]

On Thursday, June 27, 2013 9:49:36 PM UTC+2, Simon Ochsenreither wrote:

Regarding serialization:

As mentioned, I suggest looking into http://lampwww.epfl.ch/~hmiller/pickling/. Imho, it improves the state of the art considerably.


Concerning query generation:

I agree that a lot of work is still needed, but I think the right people are aware of it (and in charge).
I'm not a huge user of Slick, but it has come a long way since ScalaQuery.

Depends on which way you're talking about. For end users little has changed, usability-wise you could argue that Slick is to-date actually a step backward from ScalaQuery (for example, the move to Scala collection semantics makes for some pretty hideous syntax, certainly a far cry from comparative elegance of LINQ, and generated SQL leaves something to be desired, as in aforementioned cases).

This is not necessarily a bad thing as clearly good things are to come/in process, but the interim is challenging.

For example, really looking forward to Type providers in Q3, but who knows, maybe Scala 2.11 will be required for TP to become a reality, if at all.

General sense is that nothing is set in stone with Slick. Interesting year ahead for sure ;-)