Being able to load the embedded driver without having to put it in the classpath

[Pierre Vacher]

Hi all,

I'm trying to integrate Jaybird into LibreOffice Base in embedded mode (ie: all data is contained in an odb file and LibreOffice uses the integer engine provided by the driver).

First of all I want to thank Mark who made this possible.

This ticket is also a continuation of issue 629.

> That is only for loading the Firebird Embedded provider, which is an experimental feature to load the Firebird Embedded files from the classpath. Are you even using that?

I don't really know if I use it, does calling the driver with an url like jdbc:firebird:embedded:* call this service?

[Mark Rotteveel]

On 26/05/2024 13:16, Pierre Vacher wrote:
> I'm trying to integrate Jaybird into LibreOffice Base in embedded mode
> (ie: all data is contained in an odb file and LibreOffice uses the
> integer engine provided by the driver).

I highly doubt it. It might be able load the Firebird client included in
LibreOffice, though that depends on how it was modified, and it might
even be able to use Firebird Embedded that way with a normal file-based
Firebird database (though that is just a guess).

However, I think you'll need to do more to access the database embedded
in the ODB file, as that is a LibreOffice specific thing.

> > That is only for loading the Firebird Embedded provider, which is an
> experimental feature to load the Firebird Embedded files from the
> classpath. Are you even using that?
>
> I don't really know if I use it, does calling the driver with an url
> like jdbc:firebird:embedded:* call this service?

Then you're not using it. It is not a generic thing to locate Firebird
Embedded, it is a Jaybird specific thing to be able to load the embedded
files from the classpath.

That feature uses the ServiceLoader to locate a Firebird Embedded JAR
that has a specific descriptor class file and the embedded files, and if
it matches the platform, extracts it to a temporary folder (or otherwise
provides identifying information of the embedded engine location), so it
can load the Firebird Embedded engine from that temporary folder.

The only known implementation of this feature is
https://github.com/mrotteveel/jaybird-embedded-win32-x86-64

Mark
--
Mark Rotteveel

[Pierre Vacher]

Le dimanche 26 mai 2024 à 14:20:34 UTC+2, Mark Rotteveel a écrit :

I highly doubt it. It might be able load the Firebird client included in
LibreOffice, though that depends on how it was modified, and it might
even be able to use Firebird Embedded that way with a normal file-based
Firebird database (though that is just a guess).

However, I think you'll need to do more to access the database embedded
in the ODB file, as that is a LibreOffice specific thing.

 

This whole part is already functional, it is written in Python, its operation is very simple and that is why I was impatiently waiting for the possibility of creating databases without Java code. This is the same code that runs for all Database extensions in embedded mode (ie: HyperSQLOOo, SQLiteOOo, H2dbOOo and DerbyOOo). All these extensions call for help from the jdbcDriverOOo extension written in Java and which takes care of everything.

Then you're not using it. It is not a generic thing to locate Firebird
Embedded, it is a Jaybird specific thing to be able to load the embedded
files from the classpath.

If I don't use it then the problem doesn't come from there.

 

I cannot identify where and by whom the org.firebirdsql.gds.impl.BaseGDSFactoryPlugin class is loaded which is causing the problem.

[Pierre Vacher]

Well I think we have to admit that it is not possible to avoid adding the driver in the classpath if we want to use embedded mode.

The driver loads via java services a factory offered by the native archive, and this factory calls, in turn, java services present in the driver archive... The Java services are used recursively between these two archives: driver and native and therefore requires that the two archives be placed in the classpath.

It's a shame and it should be noted that if you don't use the embedded mode then the problem does not arise.

[Pierre Vacher]

In my opinion, the only implementation allowing dynamic loading of the driver would be to replace the java services offered by the driver archive with a factory offering access to the same resources and that the driver would give during the initialization of the native archive java services.

[Mark Rotteveel]

On 26/05/2024 18:13, Pierre Vacher wrote:
> Well I think we have to admit that it is not possible to avoid adding
> the driver in the classpath if we want to use embedded mode.

Maybe you're using terminology differently than I expect, but yes, the
driver must be on the classpath. Without the driver on the classpath,
there is no Jaybird.

> The driver loads via java services a factory offered by the native
> archive, and this factory calls, in turn, java services present in the
> driver archive... The Java services are used recursively between these
> two archives: driver and native and therefore requires that the two
> archives be placed in the classpath.

I am not sure what you mean with "The Java services are used recursively
between these two archives".

Mark
--
Mark Rotteveel

[Mark Rotteveel]

On 26/05/2024 15:06, Pierre Vacher wrote:
> If I don't use it then the problem doesn't come from there.
> I cannot identify where and by whom the

> *org.firebirdsql.gds.impl.BaseGDSFactoryPlugin* class is loaded which is
> causing the problem.

Jaybird itself loads classes implementing the interface
GDSFactoryPlugin, which are provided by WireGDSFactoryPlugin,
NativeGDSFactoryPlugin, and EmbeddedGDSFactoryPlugin, all three of which
extend the abstract class BaseGDSFactoryPlugin for their common
implementation. Those last two are in jaybird-native in Jaybird 6.

Mark
--
Mark Rotteveel

[Mark Rotteveel]

On 26/05/2024 18:41, Pierre Vacher wrote:
> In my opinion, the only implementation allowing dynamic loading of the
> driver would be to replace the java services offered by the driver
> archive with a factory offering access to the same resources and that
> the driver would give during the initialization of the native archive
> java services.

The problem you're having is unclear to me. Please provide more details,
and show how you load Jaybird and especially how you handle loading the
JAR files.

Mark
--
Mark Rotteveel

[Pierre Vacher]

Le dimanche 26 mai 2024 à 20:04:05 UTC+2, Mark Rotteveel a écrit :

Maybe you're using terminology differently than I expect, but yes, the
driver must be on the classpath. Without the driver on the classpath,
there is no Jaybird.

With the versions preceding version 6, if you dont use embedded mode, you dont need the native jar archive and you can dynamically load the driver of your choice at the time of connection. The jdbcDriverOOo driver allows you to specify the path of the jar archive and the class of the driver in the connection properties JavaDriverClassPath and JavaDriverClass respectively.

Loading drivers is done with Class.forName() with an URL classloader in DriverBase.registerDriver() method.

I am not sure what you mean with "The Java services are used recursively
between these two archives".

That's not really accurate. What it seems to me is that jaybird obtains the implementation of embedded mode in the native jar archive via java services, and that the native archive de facto considers that the jaybird archive is in the classpath while it would be more judicious to go through a factory.

Le dimanche 26 mai 2024 à 20:08:10 UTC+2, Mark Rotteveel a écrit :

Jaybird itself loads classes implementing the interface
GDSFactoryPlugin, which are provided by WireGDSFactoryPlugin,
NativeGDSFactoryPlugin, and EmbeddedGDSFactoryPlugin, all three of which
extend the abstract class BaseGDSFactoryPlugin for their common
implementation. Those last two are in jaybird-native in Jaybird 6.

 

With versions previous to version 6, if we do not use embedded mode, we only need the jaybird archive and we are able to load the driver with a Class.forName and a URL classloader and without the need to put the archive in the classpath. Using embedded mode requires the use of the native archive and using the native archive requires putting jaybird in the classpath.

Le dimanche 26 mai 2024 à 20:10:25 UTC+2, Mark Rotteveel a écrit :

The problem you're having is unclear to me. Please provide more details,
and show how you load Jaybird and especially how you handle loading the
JAR files.

If I can no longer use Class.forName with a URL classloader then the connection options allowing you to specify the driver (path and class) to load are no longer usable.

[Mark Rotteveel]

On 26/05/2024 21:35, Pierre Vacher wrote:
> With the versions preceding version 6, if you dont use embedded mode,
> you dont need the native jar archive and you can dynamically load the
> driver of your choice at the time of connection. The jdbcDriverOOo
> driver allows you to specify the path of the jar archive and the class

> of the driver in the connection properties *JavaDriverClassPath* and
> *JavaDriverClass* respectively.

> Loading drivers is done with Class.forName() with an URL classloader in
> DriverBase.registerDriver()

> <https://github.com/prrvchr/jdbcDriverOOo/blob/master/source/jdbcDriverOOo/source/io/github/prrvchr/uno/sdbc/DriverBase.java#L299> method.

I think you need to pass the URLs of both JARs and their dependencies
(like jna-jpms) to the `URLClassLoader`

> I am not sure what you mean with "The Java services are used
> recursively
> between these two archives".
>
>
> That's not really accurate. What it seems to me is that jaybird obtains
> the implementation of embedded mode in the native jar archive via java
> services, and that the native archive de facto considers that the
> jaybird archive is in the classpath while it would be more judicious to
> go through a factory.

This is the standard way of loading service providers in Java. The
native and embedded implementations are "plugins"/service providers.
They would always have a dependency on the main JAR because they extend
classes and implement interfaces from the main JAR.

> Le dimanche 26 mai 2024 à 20:08:10 UTC+2, Mark Rotteveel a écrit :
>
> Jaybird itself loads classes implementing the interface
> GDSFactoryPlugin, which are provided by WireGDSFactoryPlugin,
> NativeGDSFactoryPlugin, and EmbeddedGDSFactoryPlugin, all three of
> which
> extend the abstract class BaseGDSFactoryPlugin for their common
> implementation. Those last two are in jaybird-native in Jaybird 6.
>
> With versions previous to version 6, if we do not use embedded mode, we
> only need the jaybird archive and we are able to load the driver with a
> Class.forName and a URL classloader and without the need to put the
> archive in the classpath. Using embedded mode requires the use of the
> native archive and using the native archive requires putting jaybird in
> the classpath.
>
> Le dimanche 26 mai 2024 à 20:10:25 UTC+2, Mark Rotteveel a écrit :
>
> The problem you're having is unclear to me. Please provide more
> details,
> and show how you load Jaybird and especially how you handle loading the
> JAR files.
>
>
> If I can no longer use Class.forName with a URL classloader then the
> connection options allowing you to specify the driver (path and class)
> to load are no longer usable.

You can still do that, but - at a guess - you need pass *all*
dependencies to that URL classloader (that is why it accepts an array!).
That said, it has been a while since I worked with URLClassLoader and
composite class loader hierarchies, so I might be wrong there.

Alternatively, you need to build a fat JAR that merges both JARs into a
single JAR, but I would very much advise against that.

That said, I would expect you to have similar problems with Derby as
that driver is also split into multiple JARs?

Mark
--
Mark Rotteveel

[Pierre Vacher]

Le lundi 27 mai 2024 à 19:23:17 UTC+2, Mark Rotteveel a écrit :

That said, I would expect you to have similar problems with Derby as
that driver is also split into multiple JARs?

Indeed it is necessary to add additional archives in the classpath for embedded mode but I can load the driver without it being in the classpath itself.

On the other hand, I owe you thanks, because you gave me the solution:

Normally the JavaDriverClassPath parameter allows you to specify the jar archive to load (a file).

If, on the other hand, this parameter designates a folder, then I create as many classloaders as there are archives present in this folder.

It's very simple and very effective, thank you again.

Normally this will allow me to run jdbcDriver with Jaybird 5.0.4 and Java 11 while allowing me to run JaybirdOOo with Jaybird 6.0.0 and Java 17.

It is obvious that it is the first one that is launched which will define the version of the driver loaded for the LibreOffice session but it is really great..

[Pierre Vacher]

It's good JaybirdOOo is available.

This extension allows you to create a Firebird database in LibreOffice Base in two clicks.

Although this new implementation must still have many issue, I think that in terms of functionality the driver offered by LibreOffice with Firebird in embedded mode is largely outdated.

Thanks to the user for reporting any malfunctions and thanks to Mark for making this possible.