Result not really serializable?

198 views
Skip to first unread message

Sander Plas

unread,
Apr 6, 2011, 3:50:48 PM4/6/11
to jOOQ User Group
Hello,

I'm trying to use jOOQ 1.5.6 together with Wicket.

I get the impression that although 'Result' implements 'Serializable',
it can not really be serialized because it has a reference to
org.jooq.impl.Factory?

When Wicket tries to serialize a Result i obtained with a selectFrom()
query, i get:

------------------------------------------------------------------------------------------------------------------------------------------------------------------
SEVERE: Error serializing object class my.prog.app.HomePage
[object=[Page class = my.prog.app.HomePage, id = 12, version = 0]]
org.apache.wicket.util.io.SerializableChecker
$WicketNotSerializableException: Unable to serialize class:
org.jooq.impl.Factory
Field hierarchy is:
12 [class=my.prog.app.HomePage, path=12]
org.jooq.Result my.prog.app.HomePage.res
[class=org.jooq.impl.ResultImpl]
private final org.jooq.FieldProvider
org.jooq.impl.ResultImpl.fields [class=org.jooq.impl.FieldList]
private final org.jooq.Configuration
org.jooq.impl.AbstractQueryPart.configuration
[class=org.jooq.impl.Factory] <----- field that is not serializable
at
org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:
346)
at
org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:
615)
at
org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:
538)
at
org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:
615)
at
org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:
538)
at
org.apache.wicket.util.io.SerializableChecker.checkFields(SerializableChecker.java:
615)
at
org.apache.wicket.util.io.SerializableChecker.check(SerializableChecker.java:
538)
------------------------------------------------------------------------------------------------------------------------------------------------------------------

Am i doing something wrong?

Lukas Eder

unread,
Apr 6, 2011, 5:26:48 PM4/6/11
to Sander Plas, jooq...@googlegroups.com
Hi Sander,

You're right, Factory was intentionally left "unserialisable" in the
first versions of jOOQ. Today, however, this is wrong. Serialisability
is being re-worked for jOOQ 1.5.7. I'll add some additional
integration tests to ensure your use case is working correctly.

Thanks for your feedback.
Lukas

2011/4/6 Sander Plas <sande...@gmail.com>:

Sander Plas

unread,
Apr 7, 2011, 7:24:33 AM4/7/11
to jOOQ User Group
Great! Could you give a bit more information on how serialization is
going to work in 1.5.7? Are you going to make Factory serializable,
thereby making everyting that's derived from it serializable? For
example, are things like individual result records and composed (but
unexecuted) queryies going to be serializable? This would really make
things a lot easier in web environments like Wicket.

thanks

Sander

On Apr 6, 11:26 pm, Lukas Eder <lukas.e...@gmail.com> wrote:
> Hi Sander,
>
> You're right, Factory was intentionally left "unserialisable" in the
> first versions of jOOQ. Today, however, this is wrong. Serialisability
> is being re-worked for jOOQ 1.5.7. I'll add some additional
> integration tests to ensure your use case is working correctly.
>
> Thanks for your feedback.
> Lukas
>
> 2011/4/6 Sander Plas <sander.p...@gmail.com>:

Lukas Eder

unread,
Apr 7, 2011, 7:43:38 AM4/7/11
to jOOQ User Group
Hello Sander

> Great! Could you give a bit more information on how serialization is
> going to work in 1.5.7? Are you going to make Factory serializable,
> thereby making everyting that's derived from it serializable?

I'm still working on the details of serialisability in jOOQ. I have
come to find it essential that every jOOQ object has access to a
org.jooq.Configuration with an underlying java.sql.Connection. This is
important for advanced data types in some RDBMS, as well as for a lot
of future improvements / bug fixes. So I'm adding a common interface
org.jooq.Attachable, which will be extended by the two main top level
types in jOOQ:

- org.jooq.Store: for data containers like Record, etc
- org.jooq.QueryPart: for unexecuted query elements

See the trunk version here:
http://jooq.svn.sourceforge.net/viewvc/jooq/jOOQ/src/org/jooq/Attachable.java?view=markup

The only reference that is going to be transient in jOOQ 1.5.7 is a
Configuration's reference to a java.sql.Connection. In order to
execute an org.jooq.Query after serialisation/deserialisation, you
will have to re-attach it to a Configuration (= Factory), so it can be
executed in the context of that Configuration.

In short: Your use case should be working out with version jOOQ 1.5.7

> This would really make
> things a lot easier in web environments like Wicket.

I'm curious about the details of that. How do you integrate jOOQ with
Wicket? If I know more about your use case, I could anticipate other
issues that might arise...

Cheers
Lukas

Sander Plas

unread,
Apr 7, 2011, 9:32:56 AM4/7/11
to jOOQ User Group
On Apr 7, 1:43 pm, Lukas Eder <lukas.e...@gmail.com> wrote:
>
> The only reference that is going to be transient in jOOQ 1.5.7 is a
> Configuration's reference to a java.sql.Connection. In order to
> execute an org.jooq.Query after serialisation/deserialisation, you
> will have to re-attach it to a Configuration (= Factory), so it can be
> executed in the context of that Configuration.
>
> In short: Your use case should be working out with version jOOQ 1.5.7

That's great!

> > This would really make
> > things a lot easier in web environments like Wicket.
>
> I'm curious about the details of that. How do you integrate jOOQ with
> Wicket? If I know more about your use case, I could anticipate other
> issues that might arise...

Well i'm currently just trying to write a very simple database driven
web application without too much complexity and/or "layers". It's just
plain Wicket java code, jOOQ, a PostgreSQL database and some simple
classes that i wrote to make coupling of Wicket & jOOQ less verbose -
but nothing "special" that couldn't be done without those classes. I'm
using a custom wicket request cycle that opens and closes database
connections from a connection pool and i use a jOOQ factory around
that connection.

jOOQ Record objects work great with Wicket's CompoundPropertyModel -
in short, if you have <span wicket:id="title"/> in your markup, and
some Record with a getTitle() method, you can simply wrap a
CompoundPropertyModel around the Record and associate that model with
the page and wicket Wicket will almost automatically fill in the value
of the "title" field in the "title" tag in the markup.

The problem with non-serializable records is that since Wicket is a
'stateful' web framework, it needs to store the 'state' of a page
using serialization between HTTP requests, because it often needs to
re-render a page after a link or button is clicked or something like
that. I could off course copy all primitive values from a Record in my
own serializable object, or make sure that only the primary key of a
record is serialized (if it has one!) and re-request the data from the
database on the next request, but it would be a lot easier if the
Record itself would be serializable. Another advantage of having a
serializable Record would be that i could in some situations simply
call delete() on a deserialized UpdatableRecord after a user clicked a
'delete' button or even just call store() on it after the user updated
values in a form - since the record is directly coupled to the form's
"Model", the record object already has the changed values in it.

The problem with non-serializable unexecuted queries is almost the
same: in some situations i want the data on the page to update
whenever the page is re-rendered, so i would like to attach a custom
model that holds the query itself to a page, so that the query is re-
executed every time the page is reloaded. I'm currently using my own
abstract wicket Model class for this on which i have to override a
"public Select getQuery()" method every time i use it, but if a query
would be serializable, i could just pass the query as an argument to a
non-abstract version of this class.

Lukas Eder

unread,
Apr 7, 2011, 9:44:00 AM4/7/11
to jooq...@googlegroups.com
Hi Sander,

That's a very nice use case for jOOQ! Is that project going to be
accessible publicly? I was recently thinking about documenting some
success stories of using jOOQ with other frameworks or in real-life
applications. On the other hand, such a nice jOOQ-Wicket integration
could even become part of jOOQ itself, as an optional jOOQ extension,
if you would be interested in sharing this? Of course, you would keep
the copy-right of this integration...

Cheers
Lukas

Sander Plas

unread,
Apr 8, 2011, 8:24:16 AM4/8/11
to jOOQ User Group
Hi Lukas,

The project i'm working on right now is a simple order page for a
local sandwich shop with a management interface for the shop owner.
The resulting application will be accessible on the shop's website.
I'm trying to use this simple project as a test case for the jOOQ/
Wicket combination. If this works out right, i plan to use this for
more (and bigger) projects.

I'd be happy to share code of the integration part or even of a
complete example application!

Do you have any idea when 1.5.7 will be available? Can i already test
the new 'Attachable' functionality in the trunk version?

- Sander

Lukas Eder

unread,
Apr 8, 2011, 12:13:57 PM4/8/11
to jooq...@googlegroups.com
Hi Sander,

> I'm trying to use this simple project as a test case for the jOOQ/
> Wicket combination. If this works out right, i plan to use this for
> more (and bigger) projects.

That would be very nice to see!

> I'd be happy to share code of the integration part or even of a
> complete example application!

Sure. Tell me whenever you're ready for such a project. By this month,
I will hopefully succeed with a Maven integration. I'm getting a hand
by another jOOQ user on that topic. So creating a dependency to Wicket
should be rather easy.

> Do you have any idea when 1.5.7 will be available? Can i already test
> the new 'Attachable' functionality in the trunk version?

I could release early, as your reported issues concerning
serialisability are rather urgent. Say, by the end of next week? In
the mean time, you can always download the jOOQ trunk from here:
http://sourceforge.net/projects/jooq/develop

The "Attachable API" is functional. Integration tests are running
smoothly, so far. So you could already test serialising and
re-attaching Query and Record objects (pre-release bug reports would
be very welcome). But be aware of the fact that I might change some
API elements before releasing 1.5.7

Cheers
Lukas

Sander Plas

unread,
Apr 8, 2011, 4:48:58 PM4/8/11
to jOOQ User Group
Hi Lukas,

On Apr 8, 6:13 pm, Lukas Eder <lukas.e...@gmail.com> wrote:
> I could release early, as your reported issues concerning
> serialisability are rather urgent. Say, by the end of next week? In
> the mean time, you can always download the jOOQ trunk from here:http://sourceforge.net/projects/jooq/develop

A quick release would be great off course but I don't want to rush
you :) I'm already quite happy if i can experiment with the trunk
version. Speaking of which, when i try to compile the current trunk
version of jOOQ on my home PC, i get:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------
Compiling 208 source files to /home/sander/NetBeansProjects/jOOQ/build/
classes
/home/sander/NetBeansProjects/jOOQ/src/org/jooq/impl/
AbstractStoredObject.java:113: warning: non-varargs call of varargs
method with inexact argument type for last parameter;
cast to java.lang.Object for a varargs call
cast to java.lang.Object[] for a non-varargs call and to suppress this
warning
setField(parameter, this.<T> constant(null));
/home/sander/NetBeansProjects/jOOQ/src/org/jooq/impl/
AbstractStoredObject.java:113:
<T>setField(org.jooq.Parameter<T>,org.jooq.Field<T>) in
org.jooq.impl.AbstractStoredObject cannot be applied to
(org.jooq.Parameter<T>,java.util.List<org.jooq.Field<?>>)
setField(parameter, this.<T> constant(null));
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error
1 warning
/home/sander/NetBeansProjects/jOOQ/nbproject/build-impl.xml:524: The
following error occurred while executing this line:
/home/sander/NetBeansProjects/jOOQ/nbproject/build-impl.xml:258:
Compile failed; see the compiler error output for details.
BUILD FAILED (total time: 5 seconds)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------

When i tried this earlier to day on my work PC, i saw the same error
in Netbeans, but for some reason it compiled anyway.

Do you have any idea what's going wrong here?

> The "Attachable API" is functional. Integration tests are running
> smoothly, so far. So you could already test serialising and
> re-attaching Query and Record objects (pre-release bug reports would
> be very welcome). But be aware of the fact that I might change some
> API elements before releasing 1.5.7

Just to make sure i understand how this is supposed to work - i need
to explicitly call [attachableobject].attach(newJooqFactory) before i
try to use a deserialized jOOQ object, right?

Would it be possible to add some kind of mechanism that
"automatically" re-attaches objects to a new factory whenever
necessary?

For instance, i can imagine that you could add some overridable method
to the Factory class (a "factories" factory) that allows the user to
define how a new factory should be created whenever needed so that
whenever a jOOQ object discovers that it's detached, it can call that
method to get a working factory. That way, a user could "ignore" the
whole serialization/deserialization process.

Lukas Eder

unread,
Apr 8, 2011, 5:03:54 PM4/8/11
to jooq...@googlegroups.com
Hello Sander,

> A quick release would be great off course but I don't want to rush
> you :) I'm already quite happy if i can experiment with the trunk
> version.

No worries. With the currently experienced trouble related to the
latest HSQLDB version, I'm pretty keen on releasing soon, myself:
http://groups.google.com/group/jooq-user/browse_thread/thread/a1e001b682d3c35

> Speaking of which, when i try to compile the current trunk

> version of jOOQ on my home PC, i get: [...]
>
> [... the warning ...]

Trunk is very hot at the moment :-)

I have experienced inconsistencies before, between compiling jOOQ with
Eclipse and with javac. I've come to the conclusion that either one
has some severe compiler bugs. I'm not sure which one actually does,
the issue is quite tricky. You can check out the bug details here, if
you're interested:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=340506

In any case, thanks for pointing that out to me. I was planning on
updating the wiki anyway, to document using jOOQ in a Netbeans
project. So I can fix this, soon:
https://sourceforge.net/apps/trac/jooq/ticket/389

> 1 error

I can reproduce the error as well. That shouldn't be. Thanks for the
hint. In the mean time, please use Eclipse on trunk. I'm not
supporting development with Netbeans, yet.

> Just to make sure i understand how this is supposed to work - i need
> to explicitly call [attachableobject].attach(newJooqFactory) before i
> try to use a deserialized jOOQ object, right?

The final API usage will include

- Factory.attach(Attachable... attachables).
- Factory.attach(Collection<Attachable> attachables).

> Would it be possible to add some kind of mechanism that
> "automatically" re-attaches objects to a new factory whenever
> necessary?

How would you do that? jOOQ doesn't know when / how objects are
serialised. I think that should be the client code's responsibility.
Unless you have an other idea?

> For instance, i can imagine that you could add some overridable method
> to the Factory class (a "factories" factory) that allows the user to
> define how a new factory should be created whenever needed so that
> whenever a jOOQ object discovers that it's detached, it can call that
> method to get a working factory. That way, a user could "ignore" the
> whole serialization/deserialization process.

You mean a central (static) Factory registry? jOOQ couldn't ensure
that all Factories in that registry hold valid (open, working) JDBC
Connections. I think, in your case, you will have to write a wrapper
around jOOQ for serialisation / deserialisation...

Cheers
Lukas

Lukas Eder

unread,
Apr 8, 2011, 5:27:24 PM4/8/11
to jooq...@googlegroups.com
>> 1 error
>
> I can reproduce the error as well. That shouldn't be. Thanks for the
> hint. In the mean time, please use Eclipse on trunk. I'm not
> supporting development with Netbeans, yet.

Fixed on trunk:
https://sourceforge.net/apps/trac/jooq/ticket/389

Sander Plas

unread,
May 3, 2011, 12:53:30 PM5/3/11
to jOOQ User Group
On Apr 8, 11:03 pm, Lukas Eder <lukas.e...@gmail.com> wrote:
> > Would it be possible to add some kind of mechanism that
> > "automatically" re-attaches objects to a new factory whenever
> > necessary?
>
> How would you do that? jOOQ doesn't know when / how objects are
> serialised. I think that should be the client code's responsibility.
> Unless you have an other idea?
>
> > For instance, i can imagine that you could add some overridable method
> > to the Factory class (a "factories" factory) that allows the user to
> > define how a new factory should be created whenever needed so that
> > whenever a jOOQ object discovers that it's detached, it can call that
> > method to get a working factory. That way, a user could "ignore" the
> > whole serialization/deserialization process.
>
> You mean a central (static) Factory registry? jOOQ couldn't ensure
> that all Factories in that registry hold valid (open, working) JDBC
> Connections. I think, in your case, you will have to write a wrapper
> around jOOQ for serialisation / deserialisation...

I wanted to reply to this earlier but i was too busy then and forgot
about it later.

I was thinking about some kind of "dynamic" Factory registry. For
example, you could define an interface or an abstract class called
FactoryRegistry that has a getFactory() method, and make it possible
to the user to associate a (serializable) object that implements/
extends FactoryRegistry with the initial jOOQ factory. If the factory
makes sure that all detachable/Attachable objects that it creates hold
a reference to that FactoryRegistry, they could call getFactory()
whenever they find out that they are detached and should be attached
to perform a task.

This way, the implementation of how jOOQ factories and/or JDBC
connections are created or fetched is up to the user of the library.
He has to make sure that his "registry" returns valid Factories with
working JDBC connections.

The advantage of such a solution would be that the user of the library
could simply define a FactoryRegistry once and from that point on
ignore the fact that serialization/deserialization goes on at all.
This would be very useful in environments like Wicket, where
serialization is not done explicitly by the user but "automagically"
in the background by the framework.

Please tell me if i'm missing something obvious that makes this
impossible ;)

Lukas Eder

unread,
May 3, 2011, 1:03:39 PM5/3/11
to jooq...@googlegroups.com
Sounds complicated... :-)

> Please tell me if i'm missing something obvious that makes this
> impossible ;)

What about various sessions? How does an Attachable know its session?
(both J2EE/HTTP/Wicket sessions AND database sessions)

Sander Plas

unread,
May 3, 2011, 4:42:07 PM5/3/11
to jOOQ User Group
I'm not really sure what you mean with 'sessions' and why it's
relevant in this context. How does an Attachable know its 'session' in
the current situation?

Do you perhaps mean that we need a reference to a J2EE/HTTP/Wicket
session to get a (pooled) JDBC connection? As far as i know that's not
strictly necessary. Wicket for instance has static methods to retrieve
things like the current session or the current request cycle.

All i'm proposing is to (optionally) replace the current "ad hoc" or
"user driven" factory.attach(AttachableObject) calls with something
that is driven by the Attachable. After all, the Attachable knows
whether it's detached or not and whether it needs to be attached for
the task that it is trying to perform, so it should be able to shout
"help, i need a factory!" when appropriate - for example when
record.store() is called after deserialization.

By delegating the re-attachment (and possibly recreation) of Factories
to a user definable FactoryRegistry or something like that, the whole
logic of how Factories are created or retrieved and what JDBC
connections are used is still the responsibility of the user - just as
it is now.

For example, if an application accesses two separate databases, it
will already have two separate JDBC connections and thus 2 separate
jOOQ factories in the current situation, so it's easy to pass separate
FactoryRegistries (either completely separate implementations or just
separate, differently configured instances - it's up to the user) to
those factories to ensure that the right database gets connected to
the right Attachable on deserialization.

BTW, i don't think your earlier remark "jOOQ doesn't know when / how
objects are
serialised." is entirely true. AFAIK you can hook into the
serialization/deserialization process by overriding the readObject()
and writeObject() methods. So, i guess it's probably also possible to
immediately re-attach attachables on deserialization.

Lukas Eder

unread,
May 3, 2011, 5:00:52 PM5/3/11
to jooq...@googlegroups.com
Hello Sander,

I understand what you intend to do. I just don't see (yet) how it's
possible. Before continuing, I'm sure we agree that providing
registries is optional, and in case jOOQ couldn't find a registry, it
will still throw the DetachedException when an unattached object is
stored/refreshed/deleted/executed, right?

Of course, the registry's implementation is up to the user. jOOQ will
only provide an interface. But how does the magic happen? Say you have
an attachable A. And you have two databases DB1 and DB2. Now A is
available in both DB1 and DB2 (by the SchemaMapping functionality). Of
course, you are right, the attachables can hook into deserialisation,
so once they are materialised again, they will ask the registry for a
Configuration or Factory. But what can they pass to the registry? A,
for instance doesn't know whether it belonged to DB1 or DB2 anymore,
as that information was session-dependent, or in the scope of jOOQ's
client code. So how can the registry provide the correct
Configuration? Or is that entirely up to the client code?

Maybe, you could provide me with a descriptive prototype? Is this what you mean?

interface FactoryRegistry {
void attachMe(Attachable attachable) throws NotAttachableException;
}

2011/5/3 Sander Plas <sande...@gmail.com>:

Lukas Eder

unread,
May 3, 2011, 5:17:49 PM5/3/11
to jooq...@googlegroups.com
But then again...

> interface FactoryRegistry {
>    void attachMe(Attachable attachable) throws NotAttachableException;
> }

How will the attachable find the correct registry? What does Wicket do
to discover these things? There are hundreds of possibilities, from
the simple JVM parameter, to JNDI, etc etc...

Sander Plas

unread,
May 3, 2011, 5:53:12 PM5/3/11
to jOOQ User Group
On May 3, 11:00 pm, Lukas Eder <lukas.e...@gmail.com> wrote:
> Hello Sander,
>
> I understand what you intend to do. I just don't see (yet) how it's
> possible. Before continuing, I'm sure we agree that providing
> registries is optional, and in case jOOQ couldn't find a registry, it
> will still throw the DetachedException when an unattached object is
> stored/refreshed/deleted/executed, right?

Agreed :)

> Of course, the registry's implementation is up to the user. jOOQ will
> only provide an interface. But how does the magic happen? Say you have
> an attachable A. And you have two databases DB1 and DB2. Now A is
> available in both DB1 and DB2 (by the SchemaMapping functionality). Of
> course, you are right, the attachables can hook into deserialisation,
> so once they are materialised again, they will ask the registry for a
> Configuration or Factory. But what can they pass to the registry? A,
> for instance doesn't know whether it belonged to DB1 or DB2 anymore,
> as that information was session-dependent, or in the scope of jOOQ's
> client code. So how can the registry provide the correct
> Configuration? Or is that entirely up to the client code?

I would say that it's entirely up to the client code. :)

I think that in most simple day-to-day scenarios (one database, one
schema) this won't be too complicated. But even in more complicated
situations, i think it's still manageable.

Let's take a look at your example. If i understand you correctly,
there is one database with two schemata (that's what the SchemaMapping
functionality does, right?). One schema is called DB1, the other one
DB2. Let's say Attachable A is a Record object that refers to a record
in DB1, but the same table exists in DB2. We know that A *could* refer
to DB1 as well as DB2, but before serialization happens we also know
that it *does* refer only to DB1. We know that because A's associated
Factory or Configuration is connected to DB1. A's Factory is connected
to DB1, because we passed a DB1 JDBC connection to that Factory when
we created it. Around that same time, we could have passed a DB1
associated registry to that same factory. By the time that the
deserialization happens, that FactoryRegistry still knows it should
provide a DB1 Factory.

What i'm trying to say: it should be entirely up to the client code :)
Even if you choose another approach than the FactoryRegistry one, i
think the general idea should still be to delegate everything to the
client code, for the simple reason that all those implementation
details should be outside the scope of the jOOQ project, just as the
creation of the initial JDBC connection is now.

> Maybe, you could provide me with a descriptive prototype? Is this what you mean?
>
> interface FactoryRegistry {
>     void attachMe(Attachable attachable) throws NotAttachableException;
>
> }

Yes, i think something like that would work. I was thinking the other
way around (let the registry provide a new Factory so that the
Attachable can re-attach itself) but this matches better with the
preferred jOOQ way of telling the Factory to re-attach attachables
(Factory.attach(Attachable... attachables)).

Lukas Eder

unread,
May 3, 2011, 6:11:35 PM5/3/11
to jooq...@googlegroups.com
> Let's take a look at your example. If i understand you correctly,
> there is one database with two schemata (that's what the SchemaMapping
> functionality does, right?).

That's exactly what it's for.

> Even if you choose another approach than the FactoryRegistry one, i
> think the general idea should still be to delegate everything to the
> client code, for the simple reason that all those implementation
> details should be outside the scope of the jOOQ project, just as the
> creation of the initial JDBC connection is now.

Yes, because I really have better things to do, too :-)
OK, I got it now. I think I may find some solution in the near future,
about how to find a relevant registry instance.

Lukas Eder

unread,
May 4, 2011, 4:32:03 AM5/4/11
to jooq...@googlegroups.com
> Yes, because I really have better things to do, too :-)
> OK, I got it now. I think I may find some solution in the near future,
> about how to find a relevant registry instance.

I have scheduled a change:
https://sourceforge.net/apps/trac/jooq/ticket/485

If I can come up with an acceptable solution how to "discover" the
relevant registry, then this can make it in 1.5.9.

Cheers
Lukas

Sander Plas

unread,
May 4, 2011, 6:07:56 AM5/4/11
to jOOQ User Group
That's great!

Thanks a lot for all the time & effort you put into this project!

Lukas Eder

unread,
May 5, 2011, 3:30:54 AM5/5/11
to jooq...@googlegroups.com
> Thanks a lot for all the time & effort you put into this project!

Well thanks for the interesting feedback!

Lukas Eder

unread,
May 16, 2011, 4:28:25 PM5/16/11
to jooq...@googlegroups.com
Hi Sander,

Feel free to test the ConfigurationRegistry and provide feedback, when
you have time
https://sourceforge.net/apps/trac/jooq/wiki/Manual/JOOQ/Serializability

Cheers
Lukas

Sander Plas

unread,
May 17, 2011, 12:05:04 PM5/17/11
to jOOQ User Group
Hi Lukas,

Thank you for adding this functionality to jOOQ!

I've been experimenting with it all day. Unfortunately, i haven't been
able to get this to work as intended with Wicket yet. As far as i can
see, the problem is not with jOOQ but with Wicket and my failure to
understand the exact way Wicket handles serialization.

It turns out that, although Wicket *does* serialize pages and their
associated objects after each http request, it does NOT always
deserialize them on the next request. In fact, it keeps an in-memory
cache of the last few pages.

Because i disconnect JDBC connections at the end of each HTTP request
cycle to prevent them from 'leaking', this means that i end up with
jOOQ objects that have references to disconnected JDBC connections.
Since the jOOQ objects do not get deserialized, ConfigurationRegistry
is not being called at all.

Unfortunately, there's no (easy) way to force Wicket to disable the in-
memory cache and to always use deserialization. I'm still trying to
find a way to do this for this, but i'm affraid that even if i find a
way, this will have a negative impact on the performance of the
application.

I was wondering if it would be possible to call the
ConfigurationProvider.provideFor method just before query execution
(when a jOOQ object is detached OR the associated JDBC connection is
disconnected) instead of in readObject?



slightly off-topic: i tried to download the jOOQ source myself (both
the current trunk version and r905, which is in fact release 1.5.9
according to the subversion comments) to experiment with these kind of
things, but i get this error when i try to compile it (maven in
Netbeans):

org/jooq/impl/InsertImpl.java:[109,20] reference to addValue is
ambiguous, both method <T>addValue(org.jooq.Field<T>,T) in
org.jooq.StoreQuery<R> and method
<T>addValue(org.jooq.Field<T>,org.jooq.Field<T>) in
org.jooq.StoreQuery<R> match

Sander

Lukas Eder

unread,
May 17, 2011, 12:15:32 PM5/17/11
to jooq...@googlegroups.com
Hi Sander

> Because i disconnect JDBC connections at the end of each HTTP request
> cycle to prevent them from 'leaking', this means that i end up with
> jOOQ objects that have references to disconnected JDBC connections.
> Since the jOOQ objects do not get deserialized, ConfigurationRegistry
> is not being called at all.

Does Wicket not manage your connections via DataSources, like a J2EE
container? Then you shouldn't need to worry about connections
yourself... J2EE containers usually have a shared connection pool for
these purposes, but I don't know Wicket.

> I was wondering if it would be possible to call the
> ConfigurationProvider.provideFor method just before query execution
> (when a jOOQ object is detached OR the associated JDBC connection is
> disconnected) instead of in readObject?

I'm sorry I forgot about that. I'll add a 1.6.0-SNAPSHOT version soon,
implementing this. I prefer attaching (if possible) at deserialisation
time, because some advanced data types (especially with Oracle) need
an open connection to be constructed. But there shouldn't be any
drawback in handling the other case, too.

https://sourceforge.net/apps/trac/jooq/ticket/535

> slightly off-topic: i tried to download the jOOQ source myself (both
> the current trunk version and r905, which is in fact release 1.5.9
> according to the subversion comments) to experiment with these kind of
> things, but i get this error when i try to compile it (maven in
> Netbeans):
>
> org/jooq/impl/InsertImpl.java:[109,20] reference to addValue is
> ambiguous, both method <T>addValue(org.jooq.Field<T>,T) in
> org.jooq.StoreQuery<R> and method
> <T>addValue(org.jooq.Field<T>,org.jooq.Field<T>) in
> org.jooq.StoreQuery<R> match

Thank you for mentioning this. I keep forgetting about Netbeans. It
seems that Eclipse, Netbeans and Maven don't have exactly the same
compiler behaviour when overloaded generic methods are involved, when
<T> binds to Object. This is a rather complex topic.

In the mean time, you could try with Eclipse...?

Cheers
Lukas

Sander Plas

unread,
May 17, 2011, 2:55:49 PM5/17/11
to jOOQ User Group
Hi Lukas,

> Does Wicket not manage your connections via DataSources, like a J2EE
> container? Then you shouldn't need to worry about connections
> yourself... J2EE containers usually have a shared connection pool for
> these purposes, but I don't know Wicket.

Wicket doesn't do anything with databases, but i do use Glassfish
connection pooling. However, as far as i'm aware, i still need to
"close" pooled connections to return them to the pool. Please correct
me if i'm wrong!

> I'm sorry I forgot about that. I'll add a 1.6.0-SNAPSHOT version soon,
> implementing this. I prefer attaching (if possible) at deserialisation
> time, because some advanced data types (especially with Oracle) need
> an open connection to be constructed. But there shouldn't be any
> drawback in handling the other case, too.
>
> https://sourceforge.net/apps/trac/jooq/ticket/535

Thanks. Could you also test whether the JDBC connection is still
connected and call provideFor() when the connection isn't?

In the current situation, i'm closing the (pooled) JDBC connection
without serializing/detaching any jOOQ objects at the end of the HTTP
request cycle.

> Thank you for mentioning this. I keep forgetting about Netbeans. It
> seems that Eclipse, Netbeans and Maven don't have exactly the same
> compiler behaviour when overloaded generic methods are involved, when
> <T> binds to Object. This is a rather complex topic.
>
> In the mean time, you could try with Eclipse...?

I will try that, thanks!

Sander

Lukas Eder

unread,
May 17, 2011, 5:20:19 PM5/17/11
to jooq...@googlegroups.com
> Wicket doesn't do anything with databases, but i do use Glassfish
> connection pooling. However, as far as i'm aware, i still need to
> "close" pooled connections to return them to the pool. Please correct
> me if i'm wrong!

You're right, that's how it's done. The close() method is intercepted
by the J2EE container.

> Thanks. Could you also test whether the JDBC connection is still
> connected and call provideFor() when the connection isn't?

That might work. On the other hand, I don't feel the opened/closed
state of a Connection should be in jOOQ's responsibility (I'm not even
sure whether isClosed() is implemented in all JDBC drivers). As an
alternative, check out line 69 in
http://jooq.svn.sourceforge.net/viewvc/jooq/jOOQ/src/main/java/org/jooq/impl/AbstractQuery.java?revision=908&view=markup

What about if jOOQ just always calls provideFor() before every query
execution, so you can run those (and additional) checks yourself? You
(client code) know more about what's a good Connection and what is
not. If I get a new Configuration through provideFor(), I'll use that.
Otherwise, the existing Configuration will have to do. And we'll leave
the original calls to provideFor() on deserialisation of a Connection.
I somehow feel, this will be the cleanest solution. What do you think?

Lukas

Sander Plas

unread,
May 18, 2011, 6:15:20 AM5/18/11
to jOOQ User Group
> What about if jOOQ just always calls provideFor() before every query
> execution, so you can run those (and additional) checks yourself? You
> (client code) know more about what's a good Connection and what is
> not. If I get a new Configuration through provideFor(), I'll use that.
> Otherwise, the existing Configuration will have to do. And we'll leave
> the original calls to provideFor() on deserialisation of a Connection.
> I somehow feel, this will be the cleanest solution. What do you think?

That sounds like an excellent solution!

Lukas Eder

unread,
May 18, 2011, 6:28:56 PM5/18/11
to jooq...@googlegroups.com
Hi Sander,

Feel free to check up on that solution using the latest SNAPSHOT:
https://oss.sonatype.org/content/repositories/snapshots/org/jooq/

Cheers
Lukas

Sander Plas

unread,
May 19, 2011, 6:34:47 AM5/19/11
to jOOQ User Group
Hi Lukas,
That works like a charm! This allows me to "ignore" the fact that all
the serialization/deserialization and JDBC connecting/disconnecting is
going on in the background. It also makes my code much cleaner.

Thanks!!

Lukas Eder

unread,
May 19, 2011, 7:23:39 AM5/19/11
to jooq...@googlegroups.com
> That works like a charm!  This allows me to "ignore" the fact that all
> the serialization/deserialization and JDBC connecting/disconnecting is
> going on in the background. It also makes my code much cleaner.

Nice!

Thanks for feedback.
Cheers Lukas

Reply all
Reply to author
Forward
0 new messages