future of jooq about nosql db (mongo, couchbase, ...)

[Denis Miorandi]

hi Lukas,

      I'm evaluating some document db like mongo and couchbase. I love to use jooq to query so I'm wondering what about future of jooq / new db supports.

I suspect jooq is coupled with jdbc, but jdbc drivers are available both on mongo and couchbase, so question is: do you think is possible 

to implement those databases like standards new vendor/engine in jooq, or are there some blocking issue to do this?

Is it in plan or in you mind?

Tks

Denis

[Lukas Eder]

Hi Denis,

Thank you very much for your very interesting question.

There are two main driving things that will influence the decision of supporting a new database

Technological influence

First off, there's no definite answer to this, as these databases move rather quickly, and sometimes, in surprising ways. In particular, I'm very intrigued by CouchBase's N1QL language (and a bit less so, but still intrigued by Neo4j's Cypher language).

As a general rule of thumb, jOOQ may, in the future, support any SQL based database, regardless if it is relational or not. Much like jOOQ supports (in principle) SQL's XML, Object, or JSON extensions, jOOQ might also go the other way round and support a JSON database that decides to build a SQL-style language on top of it - as long as the database exposes a JDBC API to abstract the protocol layer, of course. But most of them do. JDBC can handle any data type on a low level, through nested ResultSets.

Market influence

The second driving force behind adding support for a new database is market driven. Every new SQLDialect that we support adds considerable complexity to our integration tests, and thus adds risks in terms of quality, time to market, etc. For obvious reasons, we want to invest as much time as possible into improving Oracle, SQL Server, PostgreSQL and MySQL integrations (see http://db-engines.com/en/ranking).

This means that any such integration must be valuable not only for our target audiences, but also for ourselves as a vendor. That's where these NoSQL databases are in a much poorer position compared to their RDBMS counterparts (at least from our vendor perspective), because they're all non-standard and mostly OSS databases:

• Non-standard means they implemented their own proprietary query language that is largely incompatible with the SQL standard, and thus much much harder for us to integrate. In addition to that, if we did support CouchBase and N1QL for instance, we couldn't profit from this integration for any other database. It may even turn out that CouchBase single-handedly "shuts down" the N1QL language in favour of something else, and we'd be screwed. Conversely, adding support for CTE, for instance was good for the entire jOOQ ecosystem, because at least DB2, Firebird, HSQLDB, Oracle, PostgreSQL, SQL Server, Sybase SQL Anywhere, Vertica, and soon MySQL 8 support them. So, while jOOQ occasionally supports non-standard SQL extensions (e.g. MySQL's ON DUPLICATE KEY UPDATE), jOOQ really heavily embraces the SQL standard.
  
• OSS means that unless we change our business model (free for OSS databases, commercial for commercial databases), we'd have to support those "NoSQL SQL dialects" for free. So, we don't get any revenue from such an integration.
• In addition to the above, MongoDB is the only NoSQL database with a significant market share (and a non-trivial query language, like Cassandra's), such that we might get any benefit out of a jOOQ API support. But MongoDB has their own JSON-based query language, so jOOQ does not help there.

As a counter example, QueryDSL used to be an on a par with jOOQ, competitor API years ago (they have lost traction and won't keep up with jOOQ any longer for many reasons). QueryDSL always flirted with a LINQ style query language super set that works on any data store, including MongoDB. If you read their mailing list, you can immediately see that this was one of QueryDSL's biggest mistakes both in terms of quality and usability, much like it was a mistake for LINQ to engage in that abstraction stuff. The only really popular LINQ provider is LINQ-to-Objects, and even then, it would have been wiser (in my opinion) to design a fancy collection API like Scala has (or to a lesser extent, Java 8 streams). Neither LINQ-to-SQL nor LINQ-to-EF nor LINQ-to-XML or anything else ever really gained popularity. An interesting opinion on this can be found on the jOOQ blog, when I interviewed Ming-Yee Iu, the author of JINQ:

https://blog.jooq.org/2016/04/19/jooq-tuesdays-ming-yee-iu-gives-insight-into-language-integrated-querying

jOOQ will not make this mistake, even if at first, you may think it would be great to continue using jOOQ also for those databases. But trust me. You will keep jOOQ in good memory instead of starting to hate it, if it did support a NoSQL store.

But perhaps there's hope :) CouchBase have their own, jOOQ-inspired internal DSL API, see:

http://www.slideshare.net/Couchbase/native-ap-isqueryingcbn1qljeffmorrismichaelnitschinger/31

It's not as sophisticated as jOOQ, but maybe, you can contribute? Or, you use PostgreSQL, whose JSON(B) support is rather sophisticated, and perhaps good enough for your use-cases?

I hope this helps,

Let me know if you have any additional questions, and I'm very happy to discuss.

Lukas

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Denis Miorandi]

Hi Lukas, 

     Thank you for your complete response. This is very useful to me to have a clear scenario.

I understand perfectly the point, mongodb is ok about Market influence but is proprietary.

I'm also with you about querydsl comparision. I'm switched from querydsl-sql to jooq two years ago. I perfectly understand what you are trying to say, and your business point of view.

Actually I'm using also jsonb on postgres cause my rest-jackson object have become too complex to map it on releational tables (hard to tracking changes to save json=>db, hard to manual mapping json<==>relational).

Jsonb it's an excellent implementation, probably enough to me but it miss a crucial feature of mongo. There are no way to link a document inside another (afaik), 

so the only option is to embed, but this it's is a little hard to mantain (btw is possible to simulate this on java side).

So this is why I'm starting to check pure document oriented / json databases.

Denis

To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.

[Lukas Eder]

Hi Denis,

Hmm, linking a document inside another... That was already a huge mess with XML, which formally supported the feature in XSD, but JSON...? Are you sure you want to go down that rabbit hole? I mean, after all, the relational model solved all that mess in the 70s, when before, there were only hierarchial and networked databases (i.e. document databases).

It's up to you, but I'd say the time you save today is the trouble you'll have 2-3 years down the line.

I'll be curious to hear what you'll decide to do!

Cheers,

Lukas

To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+unsubscribe@googlegroups.com.

[Denis Miorandi]

I'm exploring, those link does not have reference integrity this is what I'm concerned about, but 

if you have i.e. a companies collection and an orders collection every order have to be connected (customer, supplier, mandant) to companies registry. Embedding is an option but this is difficoult to refresh....

Btw document have to be as autoconsistant as possible, but a few connection are neccessary.

Anyway I'll let you know...

Tks

Denis

--
You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/sZSaYChQJEI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dott. Denis Miorandi
via dei Vignai 27
38060 - Nogaredo (TN)

[Lukas Eder]

2017-01-16 15:39 GMT+01:00 Denis Miorandi <denis.m...@gmail.com>:

I'm exploring, those link does not have reference integrity this is what I'm concerned about,

Well ;)

I know that JCR does support referential integrity in their document database structure, but it's such a mess again, especially in terms of performance...

 

but if you have i.e. a companies collection and an orders collection every order have to be connected (customer, supplier, mandant) to companies registry.

Sounds like the perfect job for the relational model :)

 

Embedding is an option but this is difficoult to refresh....

Btw document have to be as autoconsistant as possible, but a few connection are neccessary.

Anyway I'll let you know...

Looking forward,

Lukas