Is there a way to make it so plugins do not rely on File system paths?

[Nicholas DiPiazza]

I have a working example of using PF4J that loads a "plugin" from the local file system.

But I'm working on a cloud project now that makes it so we really need to load all plugins into the classloader using URLs to an HTTP network path.

I notice that PluginWrapper object only has a "java.nio.Path" object to represent the plugin. Is this going to be possible for us to use PF4J to support network resource plugins?

[Decebal Suiu]

Do you know about https://github.com/decebals/pf4j-update? I don't know if help you but you can take a look at https://s.apache.org/solr-plugin. The idea is to brings plugins from multiple repositories to your machine, It's similar with Eclipse Update Manager. 

If my information doesn't resolve your problem we can continue the discussion in other direction. I want to be sure that you know about PF4J-Update.

[Decebal Suiu]

I notice that PluginWrapper object only has a "java.nio.Path" object to represent the plugin. Is this going to be possible for us to use PF4J to support network resource plugins?

A Path is an object that may be used to locate a fine in a file system. Sure, the default file system is the local file system but they are available other implementations (providers):

- HDFS (https://github.com/damiencarol/jsr203-hadoop)

- Amazon S3 (https://github.com/Upplication/Amazon-S3-FileSystem-NIO2)

- In-memory (https://github.com/google/jimfs)

- Google cloud (https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google-cloud-contrib/google-cloud-nio)

The simple approach when you want to use plugins from a network (your network) is to put all your plugins in a shared directories. In this mode all your servers/applications use/share the same plugins.

On the other hand, PF4J works with any java.lang.ClassLoader as plugin class loader. The default plugin ClassLoader is PluginClassLoader (extends URLClassLoader) so you can use addURL(URL url) method to add a resource to the plugin class loader.

[Nicholas DiPiazza]

yeah we would have to implement our own file system for out "blob store" which is just a solr collection that stores our plugin files.

any easier ways that you can think of? let me know.

[Decebal Suiu]

Could you give me more details about what you try to achieve?

[Nicholas DiPiazza]

We have a web application that has plugins written in java. They are pretty simple in structure:

Plugin zip file
--> json descriptor file
--> main plugin Jar file
--> lib directory
     --> Dependency jar files

Currently when we install, we have our plugin zip file stored in a Solr instance.

When it's time to start the services, we check for plugins in the solr instance, and then download and extract the zip file to file system, then deploy into a separate class loader for each plugin.

We are looking to move to pf4j. One of the big things we want to achieve is to no longer to rely on a file system at all. So we'd like to be able to just grab the zip file from the Solr instance and add it directly to the classloader... having no interaction at all with the local file system. This makes things less complicated when working on multi-node server and being on a 3rd party hosting cloud provider.

[Decebal Suiu]

Definitely you can add resources in the plugin ClassLoader (PluginClassLoader by default) via addURL method.  

You also need to implement a PluginLoader that load plugin via Solr. Other possible custom implementation are for PluginDescriptorFinder (to read info from your json) and maybe for PluginRepository.

You can take a look at JarPluginManager to see a concrete custom PluginManager implementation.

I will go tomorrow in vacation (10 days) and it's possible to have limited access to internet.

[Decebal Suiu]

I came back from vacation. Did you make any progress? Can I help?

[Nicholas DiPiazza]

Hi,

No updates yet. We are still happily using the file system for now. And in all likelihood will be using the file system based for the next release. But we will end up revisiting the concept of zero-file system plugins eventually.

I'll reach out to you when we start trying to make this change. for now, the standard plugin maanger use case is suiting us pretty well.

On Tue, Aug 22, 2017 at 1:41 AM, Decebal Suiu <deceba...@gmail.com> wrote:

I came back from vacation. Did you make any progress? Can I help?

--
You received this message because you are subscribed to a topic in the Google Groups "pf4j" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pf4j/9WhJUIwjIzI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pf4j+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.