mvn shade (again), Could not resolve type id 'http' into a subtype of [simple type, class io.dropwizard.jetty.ConnectorFactory]

[Rudi Wirth]

We have a new project that uses Facebook Swift / Thrift, Guice, Airlift along with DW 0.9.3.
Running DW out of Eclipse or running tests with DW under junit all works fine.
Issues start with maven shade. First error was

ERROR [2016-07-18 12:24:16,937] RestRunnerImpl: failed to start
! javax.validation.ValidationException: Unable to find provider: class org.hibernate.validator.HibernateValidator
! at javax.validation.Validation$ProviderSpecificBootstrapImpl.configure(Validation.java:224) ~[nuoagent2.jar:na]
! at io.dropwizard.validation.BaseValidator.newConfiguration(BaseValidator.java:28) ~[nuoagent2.jar:na]
! at io.dropwizard.jersey.validation.Validators.newConfiguration(Validators.java:35) ~[nuoagent2.jar:na]
! at io.dropwizard.jersey.validation.Validators.newValidatorFactory(Validators.java:27) ~[nuoagent2.jar:na]
! at io.dropwizard.setup.Bootstrap.<init>(Bootstrap.java:68) ~[nuoagent2.jar:na]
! at io.dropwizard.Application.run(Application.java:72) ~[nuoagent2.jar:na]
! at com.nuodb.nagent.rest.RestRunnerImpl.start(RestRunnerImpl.java:43) ~[nuoagent2.jar:na]
! at com.nuodb.nagent.AgentMain.run(AgentMain.java:263) [nuoagent2.jar:na]
! at com.nuodb.nagent.AgentMain.main(AgentMain.java:153) [nuoagent2.jar:na]

which I was able to fix by editing and re-jar-ing META-INF/services/javax.validation.spi.ValidationProvider, with both providers (Apache came in via airlift):

org.apache.bval.jsr303.ApacheValidationProvider
org.hibernate.validator.HibernateValidator

Now I get a parsing error on the yml file. We test .yml files with both DefaultServerFactory and SimpleServerFactory.
Either stumbles over

/opt/nuodb/etc/nagentrest.yml has an error:
  * Failed to parse configuration at: server.applicationConnectors.[0]; Could not resolve type id 'http' into a subtype of [simple type, class io.dropwizard.jetty.ConnectorFactory]: known type ids = [ConnectorFactory]
 at [Source: N/A; line: -1, column: -1] (through reference chain: com.nuodb.nagent.rest.NewAgentRestConfig["server"]->io.dropwizard.server.DefaultServerFactory["applicationConnectors"]->java.util.ArrayList[0])

This is well-documented in other threads, and I've made sure that the uber-jar contains all the services.

<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

I even added some code stolen from DiscoverableSubtypeResolver which prints out various DW factories:

resource jar:file:/Users/rwirth/dev/nuodb_raft/nagent/target/nuoagent2-2.0-jar-with-dependencies.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.server.ServerFactory
resource jar:file:/Users/rwirth/dev/nuodb_raft/nagent/target/nuoagent2-2.0-jar-with-dependencies.jar!/META-INF/services/io.dropwizard.jetty.ConnectorFactory
 io.dropwizard.jetty.HttpConnectorFactory
 io.dropwizard.jetty.HttpsConnectorFactory
resource jar:file:/Users/rwirth/dev/nuodb_raft/nagent/target/nuoagent2-2.0-jar-with-dependencies.jar!/META-INF/services/io.dropwizard.logging.AppenderFactory
 io.dropwizard.logging.ConsoleAppenderFactory
 io.dropwizard.logging.FileAppenderFactory
 io.dropwizard.logging.SyslogAppenderFactory
resource jar:file:/Users/rwirth/dev/nuodb_raft/nagent/target/nuoagent2-2.0-jar-with-dependencies.jar!/META-INF/services/io.dropwizard.metrics.ReporterFactory
 io.dropwizard.metrics.ConsoleReporterFactory
 io.dropwizard.metrics.CsvReporterFactory
 io.dropwizard.metrics.Slf4jReporterFactory
resource jar:file:/Users/rwirth/dev/nuodb_raft/nagent/target/nuoagent2-2.0-jar-with-dependencies.jar!/META-INF/services/io.dropwizard.server.ServerFactory
 io.dropwizard.server.DefaultServerFactory
 io.dropwizard.server.SimpleServerFactory

The list looks correct, both running from java -jar uber-jar and running out of Eclipse.

I'm running out of ideas where else to look.  Perhaps a different class loader / service loader issue?

Any hints are much appreciated!

[Rudi Wirth]

On Tuesday, July 19, 2016 at 4:58:21 PM UTC-4, Rudi Wirth wrote:

I even added some code stolen from DiscoverableSubtypeResolver which prints out various DW factories:

..

The list looks correct, both running from java -jar uber-jar and running out of Eclipse

I should've looked harder. I only looked at the leave classes. I ended up debugging the discovery stuff, here:

Thread [main] (Suspended)   
    DiscoverableSubtypeResolver.<init>(Class<?>) line: 40   
    DiscoverableSubtypeResolver.<init>() line: 29   
    Jackson.configure(ObjectMapper) line: 61   
    Jackson.newObjectMapper() line: 23   
    Bootstrap<T>.<init>(Application<T>) line: 64   

Uber-jar:

resource jar:file:/opt/nuodb/jar/nuoagent2.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.server.ServerFactory
resource jar:file:/opt/nuodb/jar/nuoagent2.jar!/META-INF/services/io.dropwizard.jetty.ConnectorFactory
 io.dropwizard.jetty.HttpConnectorFactory
 io.dropwizard.jetty.HttpsConnectorFactory
resource jar:file:/opt/nuodb/jar/nuoagent2.jar!/META-INF/services/io.dropwizard.logging.AppenderFactory
 io.dropwizard.logging.ConsoleAppenderFactory
 io.dropwizard.logging.FileAppenderFactory
 io.dropwizard.logging.SyslogAppenderFactory
resource jar:file:/opt/nuodb/jar/nuoagent2.jar!/META-INF/services/io.dropwizard.metrics.ReporterFactory
 io.dropwizard.metrics.ConsoleReporterFactory
 io.dropwizard.metrics.CsvReporterFactory
 io.dropwizard.metrics.Slf4jReporterFactory
resource jar:file:/opt/nuodb/jar/nuoagent2.jar!/META-INF/services/io.dropwizard.server.ServerFactory
 io.dropwizard.server.DefaultServerFactory
 io.dropwizard.server.SimpleServerFactory

Eclipse or mvn exec:java:

resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-core/0.9.1/dropwizard-core-0.9.1.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.server.ServerFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-jetty/0.9.1/dropwizard-jetty-0.9.1.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.jetty.ConnectorFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-logging/0.9.1/dropwizard-logging-0.9.1.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.logging.AppenderFactory
 io.dropwizard.logging.LoggingFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-metrics/0.9.1/dropwizard-metrics-0.9.1.jar!/META-INF/services/io.dropwizard.jackson.Discoverable
 io.dropwizard.metrics.ReporterFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-jetty/0.9.1/dropwizard-jetty-0.9.1.jar!/META-INF/services/io.dropwizard.jetty.ConnectorFactory
 io.dropwizard.jetty.HttpConnectorFactory
 io.dropwizard.jetty.HttpsConnectorFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-logging/0.9.1/dropwizard-logging-0.9.1.jar!/META-INF/services/io.dropwizard.logging.AppenderFactory
 io.dropwizard.logging.ConsoleAppenderFactory
 io.dropwizard.logging.FileAppenderFactory
 io.dropwizard.logging.SyslogAppenderFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-metrics/0.9.1/dropwizard-metrics-0.9.1.jar!/META-INF/services/io.dropwizard.metrics.ReporterFactory
 io.dropwizard.metrics.ConsoleReporterFactory

 io.dropwizard.metrics.CsvReporterFactory
 io.dropwizard.metrics.Slf4jReporterFactory
resource jar:file:/Users/rwirth/.m2/repository/io/dropwizard/dropwizard-core/0.9.1/dropwizard-core-0.9.1.jar!/META-INF/services/io.dropwizard.server.ServerFactory
 io.dropwizard.server.DefaultServerFactory
 io.dropwizard.server.SimpleServerFactory

So for some reason, the shade plugin did not take care of multiple services for the same interface in 4 jars,
META-INF/services/io.dropwizard.jackson.Discoverable

I was able to jar xf the META-INF/services/io.dropwizard.jackson.Discoverable, edit it, then jar uf the uber-jar and DW is starting up. Same with

META-INF/services/javax.validation.spi.ValidationProvider per my initial post.

Once I know how to make the ServicesResourceTransformer behave, I'll post again. I have no idea why one of our rest apps works fine, and this one doesn't, when they both use the same maven shade step.

[Pushpendra Patel]

Hi,

    Did you find any way for ServiceResourceTransformer to work for multiple services for same interface ? I am facing same issue in year 2019 with latest versions of maven shade plugin and of Dropwizard. I am able to do same workaround as of yours. But not able to find what shade plugin can offer for this problem.

[Pushpendra Patel]

It is resolved for me. I was using wrong jar file to run out of multiple jars which were generated by other plugins. My bad.