jOOQ+Liquibase+docker

[Pasha Finkelshtein]

Hi everybody!

I wanna do the following during my Gradle build:

1. Launch docker (test?) container with postgres
2. Run my Liquibase migrations on it
3. Regenerate jOOQ code from the database
4. Bring the container down

However, I didn't find a working way to launch a (test)container during the build. Did anybody succeed in doing this?

Best,

Pasha

[Lukas Eder]

Hi Pasha,

I haven't tried yet, but it seems that Task.doFirst() and Task.doLast() could help here?

E.g.

tasks.named("jooqCodegen") {

    doFirst {

        // Start testcontainers

    }

    doLast {

        // Stop testcontainers

    }

}

You might add some additional logic to coordinate liquibase with jooqCodegen

I hope this helps

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/3422abd2-8312-4fc4-8476-82b6bf3b25ebn%40googlegroups.com.

[Pasha Finkelshtein]

Certainly it should work, but I can't start test containers within the Gradle build :(

doLast should not be even needed — Ryuk will take care of shutting the container down

Also, we need to run Liquibase migrations, so I think bringing container up should be a separate task too

Pasha Finkelshteyn Developer Advocate for Data Engineering JetBrains asm...@jetbrains.com https://linktr.ee/asm0dey Find out more

You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/WYKlwguBHwE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO5wcrgPJj9vKNS4Pr33E-0xhnTDKAn7kWwir-ma7XM1eA%40mail.gmail.com.

[Lukas Eder]

Well, surely testscontainers folks will be able to help with testcontainers specific issues?

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAO4yE_bi%3DiPeDNSj0fgxc-mcSiUfxSUHpennN%3DuM7FKJZ%3Dj%3DWQ%40mail.gmail.com.

[Pasha Finkelshtein]

I tried, no luck yet :)

But I believe it's just something nobody did - launch a container for build, not for tests.

Also, LiquibaseDatabase won't work for me, because I use `CALL` in my migrations, which creates a lot of entities in the DB...

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO5LOu3f7h5s_vsmz59Vw4kbHwxbf656DK5d76fk%2BHZu3Q%40mail.gmail.com.

[Simon Martinelli]

Hi Pasha,

I do this all the time, but with Maven.

Siva from the Testcontainers team even created a Maven Plugin:

https://github.com/testcontainers/testcontainers-jooq-codegen-maven-plugin

Unfortunately, I don’t know Gradle to help you.

— Simon

[Lukas Eder]

Maybe, if you ask more specific questions? It's hard to see what exactly didn't work on your end, and I doubt that:

- Nobody does this

- It doesn't work in general

I recommend doing this all the time for Maven and Flyway:

https://blog.jooq.org/using-testcontainers-to-generate-jooq-code/

And I'm sure it'll work for Gradle and Liquibase too. I have tasks open to do this myself for jOOQ's MCVE and demo projects, e.g.

https://github.com/jOOQ/jOOQ-mcve/issues/41

But didn't get around to it yet.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAO4yE_ZD0i7L8YxLt_EEMYkEpT2r_a73OvHczKUHRxyAjdMxKQ%40mail.gmail.com.

[Per Lundberg]

Hi Pasha,

Not _exactly_ identical to your use case, but what we do in our Gradle build is to launch a test container for our code generation, then run the migrations (with Flyway), then shut the container down. This works for us, but we've put the whole "launch test container" and run codegen etc. in a dedicated Java class instead of managing it purely via Gradle.

I can share some more details as needed, but you could first look at this which shows what our Gradle task looks like for this: https://groups.google.com/g/jooq-user/c/PxxcBaB8gxA/m/hxDKw3SvAgAJ

Best regards,

Per Lundberg

---

From: jooq...@googlegroups.com <jooq...@googlegroups.com> on behalf of Pasha Finkelshtein <pavel.fin...@gmail.com>
Sent: Thursday, February 22, 2024 11:01
To: jooq...@googlegroups.com <jooq...@googlegroups.com>
Subject: Re: jOOQ+Liquibase+docker

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAO4yE_ZD0i7L8YxLt_EEMYkEpT2r_a73OvHczKUHRxyAjdMxKQ%40mail.gmail.com.

[Pasha Finkelshtein]

This obviously SHOULD work, but it doesn't. And I was hoping to get a concrete example to understand what I'm doing wrong.

It seems to me that test containers just don't work during the gradle build, but maybe I'm wrong. And since nobody in the Testcontainers slack answer I thought that maybe here somebody has a working example.

Sorry for bothering you

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO42j5qVaW9yW%3DE-i_cEX2enxdnNns7umSgKQbx%3DidK6pA%40mail.gmail.com.

[Lukas Eder]

Well, what did you try?

[Pasha Finkelshtein]

This is what I have:

buildscript {
    dependencies {
        classpath("org.testcontainers:testcontainers:1.19.5")
        classpath("org.testcontainers:postgresql:1.19.5")
    }
    repositories {
        mavenCentral()
    }

}
....

val testContainer = org.testcontainers.containers.PostgreSQLContainer(DockerImageName.parse("paradedb/paradedb:latest").asCompatibleSubstituteFor("postgres"))

val startContainer by tasks.registering {
    testContainer.start()
}

Hypothetically, it should to the trick - when I call gradlew startContainer or when Liquibase's update tsk depends on it and I call update - it should spin up the container and stop it only when the gradle is done.

Or, for example, if I will put jooqCodegen.finalizedBy(stopContainer) - the same should happen. However, I got the following error in the very beginning:

Can't instantiate a strategy from org.testcontainers.dockerclient.UnixSocketClientProviderStrategy (ClassNotFoundException). This probably means that cached configuration refers to a client provider class that is not available in this version of Testcontainers. Other strategies will be tried instead.
Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
As no valid configuration was found, execution cannot continue.

I am not sure what is the issue here. Obviously, testcontainers work on my machine; I have native Docker installed. I have a dependency on testcontainers on my script's classpath. At this point I do not know what I am doing wrong :(

[HOMERMAN]

I do this exact thing in one of my Gradle builds. Runnig this task with -Penvironment=test will:

1. Create a Postgres Docker container
2. Initialize it with your Liquibase changelogs
3. Generate your jOOQ artifacts, and 
4. Shut down the container afterwards

You can see if(environment == "test") then Docker + Liquibase operations are invoked.

There's some hard-coded concepts (like environment names, Docker image version, etc), but you should be able to adapt it to your needs.

###################################

import liquibase.Liquibase
import liquibase.database.DatabaseFactory
import liquibase.database.jvm.JdbcConnection
import liquibase.resource.DirectoryResourceAccessor
import java.sql.DriverManager

tasks.register("jooqGenerate") {
  group = "jooq"
  description = "Runs the jOOQ code generator against the schema defined in the Liquibase changelogs"

  doLast {
    val environment = findProperty("environment")?.toString() ?: "local"

    var postgresContainer: PostgreSQLContainer<Nothing>? = null
    val schemaName: String
    val driverClassName: String
    val jdbcUrl: String
    val username: String
    val password: String

    if(environment == "test") {
      postgresContainer = PostgreSQLContainer<Nothing>("postgres:16.1").apply { start() }

      schemaName = "public"

      driverClassName = postgresContainer!!.driverClassName
      jdbcUrl = postgresContainer.jdbcUrl
      username = postgresContainer.username
      password = postgresContainer.password

      Class.forName(driverClassName)

      DriverManager.getConnection(jdbcUrl, username, password)
        .use { connection ->
          connection.autoCommit = true

          connection.createStatement().use { statement ->
            statement.executeUpdate("create schema if not exists $schemaName")
          }

          connection.schema = schemaName

          JdbcConnection(connection).use { liquibaseJdbcConnection ->
            val liquibaseDatabase = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(liquibaseJdbcConnection)

            val liquibase = Liquibase("main.yaml", DirectoryResourceAccessor(File("...")), liquibaseDatabase)
            liquibase.update("")
            liquibase.forceReleaseLocks()
          }
        }

    } else {
      schemaName = if(environment == "local") "public" else environment

      driverClassName = findProperty(...)!!.toString()
      jdbcUrl = findProperty(...)!!.toString()
      username = findProperty(...)!!.toString()
      password = findProperty(...)!!.toString()
    }

    GenerationTool.generate(
      Configuration()
        .withJdbc(
          Jdbc()
            .withDriver(org.postgresql.Driver::class.java.name)
            .withUrl(jdbcUrl)
            .withUsername(username)
            .withPassword(password)
        )
        .withGenerator(
          Generator()
            .withName(...)
            .withStrategy(...)
            .withDatabase(...)
            .withGenerate(...)
            .withTarget(...)
        )
    )

    postgresContainer?.stop()

[Per Lundberg]

Hi,

This seems to me like these dependencies are being added to the wrong classpath somehow. Are you sure they should go to the buildscript dependencies?

Here's what we have, but as mentioned, we have our codegen running somewhat differently (in a separate Gradle configuration/source set). Also note that we are using jOOQ Pro in our case; IIRC the artifact names have to be edited if you're using jOOQ Open Source (but the Gradle plugin probably takes care of the jOOQ stuff anyway in your case).

dependencies {​

    // for running DB-based codegen in isolated Docker container

    codegenImplementation( "org.testcontainers:testcontainers:${testcontainersVersion}" ) {

        exclude group: 'javax.xml.bind', module: 'jaxb-api'

    }

    codegenImplementation( "org.testcontainers:postgresql:${testcontainersVersion}" ) {

        exclude group: 'javax.xml.bind', module: 'jaxb-api'

    }

    codegenImplementation "org.postgresql:postgresql:${postgresqlJdbcVersion}"
    codegenImplementation "${jooqGroupId}:jooq:${jooqVersion}"

    codegenImplementation "${jooqGroupId}:jooq-codegen:${jooqVersion}"

}

Best regards,

Per

---

From: jooq...@googlegroups.com <jooq...@googlegroups.com> on behalf of Pasha Finkelshtein <pavel.fin...@gmail.com>

Sent: Thursday, February 22, 2024 12:08
To: jOOQ User Group <jooq...@googlegroups.com>
Subject: Re: jOOQ+Liquibase+docker

 

--

You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/c6de4831-14e6-4d64-b558-34b73b45a2ben%40googlegroups.com.

[Pasha Finkelshtein]

Thank Per,

I'm absolutely not sure. My way of thinking was that both liquibase and jooq need to have a running container. So it can't be neither jooqRuntime, nor liquibaseRuntime, so it should be like "gradleRuntime", but there is no such thing :)

How do you start the container?

Best,

Pasha

[Pasha Finkelshtein]

Thanks HOMERMAN,

Where does the dependency on PostgreSQLContainer come from? I don't see neither imports nor dependency declaration...

Best,

Pasha

[Per Lundberg]

Hi,

Yeah, understanding the source sets (or whatever the concept is properly called, not sure I have the right naming myself) can be challenging. This can be a bit of help: https://www.baeldung.com/gradle-source-sets

Our code for starting the container is pretty much like this:

        DockerImageName imageName = DockerImageName.parse( "timescale/timescaledb:2.14.2-pg13-oss" ).asCompatibleSubstituteFor( "postgres" );

        try ( JdbcDatabaseContainer<?> container = new PostgreSQLContainer<>( imageName ) ) {

            container.start();

            // Apply migrations, run jOOQ codegen

        }

Best regards,

Per

---

From: jooq...@googlegroups.com <jooq...@googlegroups.com> on behalf of Pasha Finkelshtein <pavel.fin...@gmail.com>

Sent: Thursday, February 22, 2024 15:03

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/398d1de6-b1f0-42a4-887c-5e17c5bbf251n%40googlegroups.com.

[Pasha Finkelshtein]

Thanks per,

Could you please show how you add a dependency on testcontainers to your script? 

Pasha Finkelshteyn Developer Advocate for Data Engineering JetBrains asm...@jetbrains.com https://linktr.ee/asm0dey Find out more

You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/WYKlwguBHwE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/VI0PR08MB10448C60863DA23CDCE8EF908EA562%40VI0PR08MB10448.eurprd08.prod.outlook.com.

[HOMERMAN]

@Pasha

Sorry - I copy/pasted/genericized before I shared, and I think inadvertently snipped the import out along the way:

import org.testcontainers.containers.PostgreSQLContainer

[Pasha Finkelshtein]

Thanks HOMERMAN,

But where does this import come from? Where is the dependency declared?

Is it in the usual dependencies section? With which scope?

[HOMERMAN]

It's on the buildscript's classpath. So from the very top of my build script it looks like this:

import liquibase.Liquibase
import liquibase.database.DatabaseFactory
import liquibase.database.jvm.JdbcConnection
import liquibase.resource.DirectoryResourceAccessor

import org.jooq.codegen.DefaultGeneratorStrategy
import org.jooq.codegen.GenerationTool
import org.jooq.meta.jaxb.Configuration
import org.jooq.meta.jaxb.Database
import org.jooq.meta.jaxb.Generate
import org.jooq.meta.jaxb.Generator
import org.jooq.meta.jaxb.Jdbc
import org.jooq.meta.jaxb.Strategy
import org.jooq.meta.jaxb.Target
import org.jooq.meta.postgres.PostgresDatabase
import org.testcontainers.containers.PostgreSQLContainer
import java.sql.DriverManager

buildscript {
   repositories {
     mavenCentral()
   }

  dependencies {
    classpath("org.testcontainers:testcontainers:${rootProject.extra["testcontainersBomVersion"]}")
    classpath("org.testcontainers:junit-jupiter:${rootProject.extra["testcontainersBomVersion"]}")
    classpath("org.testcontainers:postgresql:${rootProject.extra["testcontainersBomVersion"]}")
    classpath("org.liquibase:liquibase-core:${rootProject.extra["liquibaseVersion"]}")
    classpath("org.jooq:jooq:${rootProject.extra["jooqVersion"]}")
    classpath("org.jooq:jooq-meta:${rootProject.extra["jooqVersion"]}")
    classpath("org.jooq:jooq-codegen:${rootProject.extra["jooqVersion"]}")
    classpath("org.postgresql:postgresql:${rootProject.extra["postgresVersion"]}")
  }

}

tasks.register("jooqGenerate") {
   ...
}

[Pasha Finkelshtein]

Now I have a relatively full picture, thank you very much!

[Pasha Finkelshtein]

Hi @Lukas,

After some experimenting, I think that I hit an issue with the jOOQ Gradle plugin, or maybe it's lack of my understanding.

Thq jooq block requires jdbc URL to be known in advance, in the configuration phase. But in a perfect world, I need the docker to be run only in the execution phase.

For example, this code works:

val start by tasks.registering(Container::class) {
    println("START CONTAINER")
    container = parade
//    doLast {
        start()
//    }
}

abstract class Container : DefaultTask() {
    @get:Input
    abstract val container: Property<PostgreSQLContainer<*>>

    fun start() {
        container.get().start()
    }

    fun stop() {
        container.get().stop()
    }
}

But if I uncomment ths "doLast" line, I get an error

* What went wrong:
Mapped port can only be obtained after the container is started

I think that the jdbc block should be lazy and read its parameters only during execution. It might be true for other blocks too — hypothetically other tasks can augment reality with parts later used by jOOQ too.

[Lukas Eder]

Hi Pasha,

Yes, see this issue for lazy configuration evaluation:

https://github.com/jOOQ/jOOQ/issues/16188

--

You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/b901ed42-ecef-4738-8385-dacc6af3cc2bn%40googlegroups.com.

[Pasha Finkelshtein]

Beautiful!

Thanks everyone for the help!

You received this message because you are subscribed to a topic in the Google Groups "jOOQ User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jooq-user/WYKlwguBHwE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jooq-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO6JqDRmNgH92zdja4pGquGXQApH86uXT2k4w8fAs1CNeg%40mail.gmail.com.

[Lukas Eder]

Task configuration is now lazy with the next nightly builds of 3.20.0 and 3.19.6:

https://github.com/jOOQ/jOOQ/issues/16188

I'll probably make another set of patch releases next week.

[Lukas Eder]

3.19.6 has been released and the gradle plugin configuration should now be lazy.

Let me know if this fixes the remaining problems for you.

To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/f1e39661-1e4d-43af-b2ea-6156b8ba478bn%40googlegroups.com.

[Kevin Jones]

If I update to JOOQ 3.19.6 my IDE stops working properly

I have a multi-module Gradle project with the generated code in one module that's referenced by two others. My code is Kotlin. I've been running JOOQ 3.19.2 and everything is OK. if I upgrade to 3.19.5 then everything is still OK  however going to 3.19.6 causes the IDE (IntelliJ IDEA) to break. 

In the IDE I get hundreds of "Unresolved Reference" errors, one for each of the types that I generate with JOOQ.

I've tried deleting the generated code and re-generating it but I still get the same issue.

Nothing else has changed, just the update from 3.19.5 to 3.19.6

The project still builds and runs so it's only within the IDE that I'm seeing the issues,

I've tried invalidating IDEA's caches but still have the same issue.

Kevin 

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/f1e39661-1e4d-43af-b2ea-6156b8ba478bn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO5toY0oGeNLuAOHdSOz8S%3D2uzQGX%3Dk1zH3d71qkwxjzow%40mail.gmail.com.

--

Kevin Jones

KnowledgeSpike

[Lukas Eder]

Hi Kevin,

Thanks for your message.

How can this be reproduced, exactly?

Thanks,

Lukas

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAKT%3DYsMTGaKDwiBZeVLrADTgVTotPnnMQwPWF8T8EDLoXsqfRQ%40mail.gmail.com.

[Kevin Jones]

Yeah, I was wondering that :)

I've created this project (it's a cut down version of what I'm using)

https://github.com/kevinrjones/JOOQIssue

If you open this in the IDE then the current version of JOOQ is set to 3.19.5 and everything is loaded properly.

If you open the file "kotlin/com/knowledgespike/progressive/database/TeamRecords.kt" then you should see no errors

If you go into the gradle directory and open the libs.versions.toml file, 

    change the version to 3.19.6

    reload the gradle project

    open the file "kotlin/com/knowledgespike/progressive/database/TeamRecords.kt" then you should see the errors

If you have Idea's project window open, and expand the 'app_shared' project so that 'generated/java' is visible, then you can see the issue.

As you switch from JOOQ version 3.19.5 to 3.19.6 the generated code stops being a sourceSet, if you then switch back to 3.19.5 it becomes a sourceSet again

My JOOQ plugin configuration is in the build.gradle.kts file in the 'app_shared' project.

Hope that makes sense,

Kevin

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAB4ELO6TrnBxK1Z7_ceQxqTA%3Dj1eeG_eEZQggbYpD0cMCSfAxQ%40mail.gmail.com.

--

Kevin Jones

KnowledgeSpike

[Lukas Eder]

Hi Kevin,

Thanks for the example project. I can reproduce the issue with your instructions. Will look into it tomorrow.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAKT%3DYsO1SJO1RWA%2Bn9-F6JCVwco_%3DtvZRCjc9QM%2Bcgo936meQQ%40mail.gmail.com.

[Pasha Finkelshtein]

It seems to me that if the build and everything else work correctly from the command line, then it's a bug in IDEA and it should be reported. 

Please report at https://youtrack.jetbrains.com/issues/IDEA

Thank you in advance!

[Lukas Eder]

It does seem like something worth reporting to IntelliJ, though I recall how slippery the implementation of a lazy configuration was, so it won't hurt to investigate if the 3.19.6 plugin now does something "stupid." The current logic to register the code generation output directory with the source sets is this, in CodegenPlugin:

https://github.com/jOOQ/jOOQ/blob/7e22b9818e785c2c5fc199de34476d3ac2890dae/jOOQ-codegen-gradle/src/main/java/org/jooq/codegen/gradle/CodegenPlugin.java#L111

task.doFirst(t -> {
    SourceSetContainer source = project
        .getExtensions()
        .findByType(SourceSetContainer.class);

    if (source != null) {
        source.configureEach(sourceSet -> {
            if (sourceSet.getName().equals("main"))
                sourceSet.getJava().srcDir(task.getOutputDirectory());
        });
    }
});

Gradle has been an arcane mystery to me over these past months, so it may well be that I'm still missing something "obvious" about when to do what.

[Bernd Huber]

maybe a bit late,

but here is may take on using Liquibase Migrations in a Gradle Project together with Jooq Codegen:

- https://github.com/funkrusher/fk-framework-quarkus-jooq

i personally don't really need a plugin for the codegen, as i often want to customize many settings, so its

more conventient for me, to just call a Code-Builder Java-Class from a gradle-task that does everything.

see Task "GenerateJooqCode"

- https://github.com/funkrusher/fk-framework-quarkus-jooq/blob/main/fk_codegen/build.gradle

and the corresponding Java-Class that does it (with help of testcontainers):

- https://github.com/funkrusher/fk-framework-quarkus-jooq/blob/main/fk_codegen/src/main/java/org/fk/generator/JooqCodeGenerator.java

[Lukas Eder]

Not sure if this helps (or is really necessary), but I've seen this question recently:

https://intellij-support.jetbrains.com/hc/en-us/community/posts/18086275779730-IntelliJ-excludes-jooqCodegen-s-folder

[Kevin Jones]

The 3.19.7 version of the plugin still fails, I'm guessing you haven't had a chance to look at it yet.

I've now reported the issue to Jetbrains as well,

https://youtrack.jetbrains.com/issue/IDEA-350872/JOOQ-Plugin-No-Longer-Works-With-IntelliJ-Idea\

Kevin

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/4900698b-4f06-4b60-a04c-e995b120952en%40googlegroups.com.

--

Kevin Jones

KnowledgeSpike

[Kevin Jones]

For what it's worth, I have a workaround for this for the moment, I simply add a sourceSet definition to the grale file

sourceSets {
    this.main {
        val generatedDir= layout.projectDirectory.dir("generated/java")
        java.srcDir(generatedDir)
    }
}

--

Kevin Jones

KnowledgeSpike

[Lukas Eder]

Hi group,

Does anyone have insight into whether this issue has been resolved by IntelliJ? I vaguely recall having seen similar problems on some computer of mine, but I can't reproduce it with IntelliJ IDEA 2024.1.3 (Ultimate Edition), Build #IU-241.17890.1.

Cheers,

Lukas

[Pasha Finkelshtein]

It doesn't seem so

Pasha Finkelshteyn Developer Advocate
Mobile: +49 1525 981-7123 Email: pasha.fin...@bell-sw.com
bell-sw.com

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/9739967d-eb8c-46e3-9009-0e4a1bd48fd6n%40googlegroups.com.

[David Bullock]

Sheesh, I wrote some gorgeous comments on that issue, but they're all vanished.

Basically, if you ran 'gradle build', it would work.  But IDEA wasn't picking up the location of the generated files from the java/kotlin source-sets, and had 'good code is red' issues because of it.

Explicitly mentioning to Gradle that the jooq-codegen output dir was included in the java/kotlin sourceSets was a workaround.   Explicitly telling IDEA via the 'idea' Gradle plugin about the jooq-codegen output dir as a 'sources' folder also was a workaround.

I surmised that given the above, jooq-codegen was adding *a* sourceset, but not the one IDEA was looking for.

To fix the problem, jooq-codegen needs to declare its generated sources in the java/kotlin sourcesSet, not its own private sourceSet.  Note: kotlin, when the jooq generator is kotlin, and java when the jooq generator is Java.

Or, IDEA needs to be able to introspect ALL the source-sets and include them in its list of sources ... not just the Java/Kotlin ones.  But maybe that's a big ask, for it to know what to do with *every* random sourceSet.

HTH,

David.

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/9739967d-eb8c-46e3-9009-0e4a1bd48fd6n%40googlegroups.com.

[David Bullock]

Hi again,

So just to check, I took the sourceSet {} hack out of a working build.gradle.kts, reloaded it, and watched as IDEA un-marked the codegen output dir as a 'source' directory in the GUI.  This was on version 2024.1.1.

Then leaving it alone, I upgrade to 2024.1.3.  Upon load of the project, after it finished synching ('building model'), the codge output dir was still not marked as a 'source' dir.

Putting back the sourceSet {} hack and synching, IDEA 2024.1.3 marked the codgegen output dir as 'source' again in the GUI, while I watched.

Note sure if any of these are relevant, but:

Jooq: 3.19.5
Jooq codegen: default-generator (emitting .java files)
Gradle: 8.7
Gradle build file syntax: Kotlin

Project file loads the kotlin/JVM plugin

The gradle project experiencing the problem is part of a multi-project Gradle layout.

The larger gradle build uses 'convention plugins' via buildSrc

HTH,

David.

[Lukas Eder]

Thanks for your comments folks, and for documenting your workarounds here, David.

In fact, I can see the problem now in IntelliJ. And (much worse), I can see similar problems with ./gradlew test commands, when tampering with the build cache as part of this task:

https://github.com/jOOQ/jOOQ/issues/16318#issuecomment-2160651166

So, "clearly" (nothing is ever clear with gradle), the plugin must be doing something very subtly wrong. Seeing how the third party plugin by Etienne Studer does it, I doubt there's a "right" way:

https://github.com/etiennestuder/gradle-jooq-plugin/blob/v9.0/src/main/groovy/nu/studer/gradle/jooq/JooqPlugin.java#L63

But there must be a way that "happens to work" which I will continue to try to chase down.

Stay tuned folks

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/CAE14wDUPxagMjb23k9EN_1YXavT6ykaFKJey0EtjdjjQBnQh2Q%40mail.gmail.com.