DefaultExecutor design

[Raman Gupta]

Hello, I'm wondering about the logic in DefaultExecutor.

https://github.com/jOOQ/jOOQ/blob/main/jOOQ/src/main/java/org/jooq/impl/DefaultExecutor.java#L50-L53

When using a standard blocking JDBC driver, the bulk of the work done by a jOOQ thread will presumably be waiting on blocking network I/O. ForkJoinPool.commonPool() parallelism is set based on the number of processors available. This is appropriate for CPU-intensive tasks, but for JDBC threads spending most of their time waiting on network I/O shouldn't jOOQ leverage a much larger thread pool?

Example prior art: Kotlin Dispatchers.IO (designed for performing I/O operations like JDBC or other blocking calls) creates 64 threads or the number of cores (whichever is larger). See https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-i-o.html.

Thanks,
Raman

[Lukas Eder]

Hi Raman,

jOOQ is a SQL library. In all things concurrency, jOOQ aims for providing easy to reimplement SPIs, always with a "reasonable" default implementation. This is such a case. jOOQ needs some DefaultExecutor for all async tasks. Most of the JDK defaults to using ForkJoinPool.commonPool(), including Stream.parallel() and others. Is this a good choice for jOOQ, for the JDK? There's no sensible answer suited for everyone. jOOQ is not opinionated, it just does the same as the JDK.

The *worst* thing jOOQ could do is turn into a concurrency-opinionated library (e.g. are the typical concurrency questions the same for a H2/SQLite backed application as they are for Snowflake/BigQuery? Probably not). We'd be endlessly bikeshedding weird cases, not even taking into account all the reactive use-cases.

Hence the very simple to override SPIs, in this case, ExecutorProvider:

https://www.jooq.org/javadoc/latest/org.jooq/org/jooq/ExecutorProvider.html

Note, if you're using coroutines, why not use R2DBC and jOOQ's kotlin/coroutine extensions?

I hope this helps,

Lukas

--
You received this message because you are subscribed to the Google Groups "jOOQ User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jooq-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/48dc6586-97a9-4de1-b41c-4abfe0db52d5n%40googlegroups.com.

[Raman Gupta]

Thanks Lukas, that is fair. I've set the executor so that jOOQs I/O falls into the same bucket as the rest of the system by delegating to Kotlin coroutines IO thread dispatcher pool, and this seems to work well:

```

DefaultConfiguration()

  ...

  .set(ExecutorProvider { Dispatchers.IO.asExecutor() })

```

I'm already using jOOQ's kotlin/coroutine extensions and R2DBC was already on my list of things to look into. I just haven't had time yet as I've been happy with the upstream postgresql driver + HikariCP.

Thanks,
Raman

[Lukas Eder]

Thanks for documenting your solution, Raman.

This could perhaps also be useful to others, if you're interested, you could ask and answer your own question on Stack Overflow, which is something SO encourages:

https://stackoverflow.blog/2011/07/01/its-ok-to-ask-and-answer-your-own-questions/

Cheers,

Lukas

To view this discussion on the web visit https://groups.google.com/d/msgid/jooq-user/df780181-7bee-432c-b86c-dd5d5ae2b2c4n%40googlegroups.com.