can jooq generate code w/o connecting to a db

[Mohit Jaggi]

Hi Folks,

I am trying to find a way to run the jooq code generation in an automated fashion on a build server. I do not want the build to try and connect to a DB to get the schema. Is there a way to supply the schema to jooq code generator in a file or something?

Mohit.

[Rob Sargent]

At some point the developers need to check in the generated code, building against the schema built up by flyway, mybatis, ruby-activeRecord or whatever.  The build system then gets the generated code along with the rest of the code, no?

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lukas Eder]

Hi Mohit,

2014-04-08 23:57 GMT+02:00 Mohit Jaggi <mohit...@gmail.com>:

Hi Folks,

I am trying to find a way to run the jooq code generation in an automated fashion on a build server. I do not want the build to try and connect to a DB to get the schema. Is there a way to supply the schema to jooq code generator in a file or something?

There are a couple of feature request that go into this direction:

- Support generating code based on JPA annotations (https://github.com/jOOQ/jOOQ/issues/2899)

- Various ideas around a code generation plugin architecture (https://github.com/jOOQ/jOOQ/issues/1464)

Already today, you can implement your own jOOQ-Meta Database that supplies schemas, tables, columns, etc. to the code generator. This custom Database could read any sort of XML file, e.g. an external one by Flywaydb or Hibernate.

How you generate that external file would then be entirely up to you.

2014-04-09 0:59 GMT+02:00 Rob Sargent <robjs...@gmail.com>:

At some point the developers need to check in the generated code, building against the schema built up by flyway, mybatis, ruby-activeRecord or whatever.  The build system then gets the generated code along with the rest of the code, no?

That's another very good option, although many people would claim that generated code should not be checked into the version control system. This depends on your taste and your existing build strategies.

Cheers

Lukas

[Mohit Jaggi]

Already today, you can implement your own jOOQ-Meta Database that supplies schemas, tables, columns, etc. to the code generator. This custom Database could read any sort of XML file, e.g. an external one by Flywaydb or Hibernate.

How you generate that external file would then be entirely up to you.

Lukas, 

I did not understand. Could you please point me to an example or section of jooq documentation please? 

Rob,

For the time being I am adding generated code to git repo, which I don't like much.

Thanks,

Mohit. 

[Witold Szczerba]

In my previous project I was thinking about this and finally I have
came to conclusion that the best option is to let CI server access a
real database to build project. My build was prepared for a database
details to be provided easily, so CI builds had theirs own databases
(in fact two, one for deployment and one for integration tests).

So, the database part of build system worked like this:
1) destroy databases (let's call them DEV and DEV_TEST)
2) build database module, this is:
a) Liquibase, as part of mvn build, recreates databases from scratch,
b) JOOQ, as later part of mvn build, generates code,
c) mvn compiles everything,
d) mvn install
3) compile, run integration tests (using DEV_TEST),
4) deploy to JEE server,
5) run e2e tests.

Staging builds are almost the same, but they do not destroy the
database, Liquibase just upgrades schema to new version.

At the beginning I was not sure if this is OK to couple the project
building process to a running database, but it really worked well.
Every developer had it's own database (either locally, in virtual
machine or an account on our development server). What's important
here is that developers didn't have to build database module on each
code change, but only when they actually altered the schema. It was
separated from a web application maven project. Building it did not
take long though, I don't remember, but I think it was something like
less than 2 minutes.

I was thinking: what's wrong with this? And finally came to conclusion
that there is absolutely nothing wrong with this. In fact, it is even
better, because everyone, from developers, CI DEV builds to CI STAGING
and PRODUCTION builds work on the same database engine. There is no
excessive abstraction above database between Liquibase and JOOQ:
what's in there is what gets used to generate schema and it's what
gets used to run integration and e2e tests. I liked it.

You might think that tying building process to

Regards,
Witold Szczerba

[Witold Szczerba]

Ooops, looks like I did not clear the last paragraph before sending :)
BTW: I was one of the developers too, so I tested it all by myself in practice.

P.S.
One advice: if you plan unit/integration/e2e tests, stick to the most
important rule: do not duplicate unit tests in integration tests and
unit+integration tests in e2e tests. This is one rule I can see broken
very often and it really makes development hard and and it becomes
harder and harder after each sprint.
It's no wonder that after bad experience like that, teams come to
conclusion that all that testing thing is a waste of time.

[Rob Sargent]

Hm, following Lukas's suggestion you'll be writing code to generate and later read a file, checking that in and reading it on the build machine to generate the jooq classes to build your app. Doesn't seem worth it to me. Checking them in let's them evolve naturally. 

Sent from my iPhone

--

[Roger Thomas]

First off I'm making no claims to have done all the work you have already done, but I'm involved in a project where the final aim will be
to do the CI cycle that you describe and post the resulting JOOQ output into a Maven repository.

The logic being that the schema will be coded in Liquibase or Flyway and have its own unit test cycle, with the output being a JOOQ
library that can then be consumed by developers.

From my point of view this looks like a nice R&D project with the value add of a fully tested schema and JOOQ library that can be
by developers. Just to complicate things I also plan to use Gradle rather than Maven as Maven XML does my head in :( and Gradle
may remove this problem.

[Lukas Eder]

Mohit,

2014-04-12 22:23 GMT+02:00 Mohit Jaggi <mohit...@gmail.com>:

Already today, you can implement your own jOOQ-Meta Database that supplies schemas, tables, columns, etc. to the code generator. This custom Database could read any sort of XML file, e.g. an external one by Flywaydb or Hibernate.

How you generate that external file would then be entirely up to you.

Lukas, 

I did not understand. Could you please point me to an example or section of jooq documentation please? 

Unfortunately, we don't have any documentation in that area yet. But the idea is quite easy to understand. jOOQ-Meta has a variety of "Database" implementations, which deliver database meta information to jOOQ-Codegen. In your code generation configuration file, you can see the "Database" here:

<generator>
    <database>
      <!-- The database dialect from jooq-meta. Available dialects are
           named org.util.[database].[database]Database. Known values are: [...]

           You can also provide your own org.jooq.util.Database implementation
           here, if your database is currently not supported or if you wish to
           read the database schema from a file, such as a Hibernate .hbm.xml file -->
      <name>org.jooq.util.oracle.OracleDatabase</name>

See: http://www.jooq.org/doc/latest/manual/code-generation/codegen-configuration

Currently, all of these implementations read their meta information from an actual database connection. There is, however, no stopping you from implementing your own "Database" implementation (just about 8 methods), which provides meta information from your own XML / CSV / JSON file, etc.

Within less than a day, you *could* pull this information from a file, rather than from a running database.

I'll comment on Witold's excellent explanation in another message.

[Lukas Eder]

Hi Roger, Witold,

From our own experience with customers, integration testing will eventually require being run against an actual database anyway, so moving away from the previous decade's we-must-unit-test-everything / we-must-get-100%-coverage hype towards a more 80 / 20-ish strategy where CI and integration testing is in the center of QA might lead to what we really want in the end: Higher quality.

It's very nice to see that you have both found solutions to integrating this code generation step into your build / CI process. There isn't a one-size-fits-all solution, but it's good to see that working with actual databases makes sense to both of you :-)

Now, I don't fully want to exclude the usefulness of another, intermediary step where the database schema is duplicated (generated, hopefully!) into another external data format, which could be used to produce jOOQ code. Vlad Mihalcea - a very active blogger - has recently shown this by example:

http://vladmihalcea.com/2013/12/06/jooq-facts-from-jpa-annotations-to-jooq-table-mappings/

He's using JPA-annotated entities to generate a (temporary) HSQLDB schema, which he uses to generate jOOQ code. A clever solution, if you're building an CQRS-style application, where JPA is used for the Command (C) part and jOOQ for the Query (Q) part.

Another solution would have been to let Hibernate generate .hbm.xml files, which could be read from a custom jOOQ-Meta "Database" implementation.

The options are endless :-). I think that the most important aspect of all of this is to be able to automate each of these steps, without needing to manually patch different representations of your database schema. There should be only one "source of truth", and all other objects should be generated, derived objects, like jOOQ's generated tables. In more complex projects, the "source of truth" will inevitably be DDL as you're writing triggers, stored procedures, views, storage clauses, etc. So again, like Roger and Witold have shown, embracing database-centric designs will not be a bad choice in the long run, when your project evolves.

Cheers

Lukas

[Witold Szczerba]

Hi Lukas,
I think, maybe it would be worth mentioning the issue in the documentation or faq or somewhere else (best practices?) on the project page. It seems to be causing a lot of confusion.

Regards,
Witold Szczerba
---
Sent from my mobile phone.

[Lukas Eder]

Hi Witold,

Hmm, looks like I had removed that paragraph from my E-Mail... I was actually suggesting the same thing :-)

Regards,

Lukas

[Mohit Jaggi]

Thanks. I will just checkin the code for now.

--
You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/kQO757qJPbE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.