Dropwizard 0.8.0-rc1 migration and testing

[Gene Trog]

I was recently tasked at my company to move one of our services that's not deployed to prod yet from dropwizard 0.7.1 to dropwizard 0.8.0-rc1.  It's a fairly basic service that takes a POST request and logs it to a file for logging and analytic purposes.

I just wanted to briefly share what my pain points were with the migration for others to be aware about.  In general, the migration was straight-forward on the dropwizard side, but, in migrating our dropwizard boilerplate libraries that had a lot of jersey specific functionality, I found myself having to change just about everything we do.  A lot of these changes are covered here https://jersey.java.net/documentation/latest/migration.html.  Here's the list of things I found having to change:

1.) Client filters:  It seemed that there was no longer a 'ClientFilter' class and ended up having to replace it with ClientRequestFilter.  The filter no longer seems to act as a passthrough where the handle() method returns a transformed/filtered request object, but I guess you're supposed to now change the request in place?

2.) In our boilerplate library, we also have a module for boilerplate client functionality.  I had to change all references of com.sun.jersey.Client to use javax.ws.rs.client.Client, which was pretty straight forward.

3.) In changing to Java's Client, some of the resource endpoints changed.  The resource() method now returns WebTarget rather than WebResource, and there is no longer an asyncResource() method because WebTarget allows you to invoke it asynchronously.  This allowed me to remove a lot of our 'async' boilerplate methods.

4.) In some instances, I had to use readEntity() instead of getEntity()

5.) HttpRequestContext seemed to no longer be available, so I found a suitable replacement using ContainerRequest

6.) In one of the unit tests for a filter, I was using the dropwizard resource test rule.  The filter itself depended on the injection of HttpServletRequest, which seemed to no longer be injected by default, so, I had to write my own HK2 injection binder to inject a mocked version of this value.

7.) The way basic authentication is done has changed.  On the service side, we would have to register a provider called BasicAuthProvider<>, and now instead, we register a BasicAuthFactory<> which is wrapped with AuthFactory.binder().  On the client side, to embed credentials, before we would register a HTTPBasicAuthFilter, and now we register an instance returned by HttpAuthenticationFeature.basic().

8.) In our boilerplate client code, I was receiving an exception when trying to call put() on Client with a null entity.  This was due to http validation, which can be turned off by adding the property ClientProperties.SUPPRESS_HTTP_CLIENT_VALIDATION to the client with a value of true.

I did like the fact that now I could easily inject our resources using HK2's AbstractBinder, something I've been wanting to do with Guice reliably for some time. Beyond all these changes I made, I also did some basic load testing to verify that there were no significant performance penalties in switching, since this is a high-throughput service.  I tested performance using siege running on my laptop, sending requests to an instance of the service running on a remote box.  I ran 625 concurrent requests, each with a 0-1 sec. delay and here are the results I got:

BEFORE (Dropwizard 0.7.1)

RUN 1

Transactions:       59299 hits

Availability:      100.00 %

Elapsed time:       59.68 secs

Data transferred:       16.57 MB

Response time:        0.12 secs

Transaction rate:      993.62 trans/sec

Throughput:        0.28 MB/sec

Concurrency:      118.86

Successful transactions:           0

Failed transactions:           0

Longest transaction:        1.48

Shortest transaction:        0.00

RUN 2

Transactions:       60339 hits

Availability:      100.00 %

Elapsed time:       59.14 secs

Data transferred:       16.86 MB

Response time:        0.10 secs

Transaction rate:     1020.27 trans/sec

Throughput:        0.29 MB/sec

Concurrency:      105.45

Successful transactions:           0

Failed transactions:           0

Longest transaction:        1.41

Shortest transaction:        0.00

AFTER (Dropwizard 0.8.0-rc1)

RUN 1

Transactions:       61320 hits

Availability:      100.00 %

Elapsed time:       59.79 secs

Data transferred:       17.13 MB

Response time:        0.10 secs

Transaction rate:     1025.59 trans/sec

Throughput:        0.29 MB/sec

Concurrency:      106.55

Successful transactions:           0

Failed transactions:           0

Longest transaction:        1.42

Shortest transaction:        0.00

RUN 2

Transactions:       60320 hits

Availability:      100.00 %

Elapsed time:       59.52 secs

Data transferred:       16.86 MB

Response time:        0.10 secs

Transaction rate:     1013.44 trans/sec

Throughput:        0.28 MB/sec

Concurrency:      106.30

Successful transactions:           0

Failed transactions:           0

Longest transaction:        1.61

Shortest transaction:        0.00

As the results show, switching over had no impact on overall performance.  So, in general, I didn't run into too many problems on the dropwizard side, albeit this service is fairly basic and doesn't use a lot of extended dropwizard functionality.  The bulk of the changes were really in the jersey filters and injections providers we use.

[Ryan Kennedy]

Thanks for the detailed writeup, Gene! Also, I'm happy to see a positive data point on the performance after the prior one we'd gotten on the list.

Ryan

--
You received this message because you are subscribed to the Google Groups "dropwizard-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dropwizard-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Damian Pawłowski]

Gene thanks for sharing. I'm also migrating our services from dropwizard 0.7.1 to 0.8. However I'm stuck rewriting unit tests that use dropwizard resource test rule (in 0.7 I used this solution https://github.com/dropwizard/dropwizard/issues/651).

Could you give an example how did you solve this problem? My previous attempts injecting HttpServletRequest failed.

Thanks

[Gene Trog]

So I ended up having to mock out and inject HttpServletRequest myself:

    @Rule

    public ResourceTestRule testRule;

    private HttpServletRequest request;

    public BasicAuthenticatorTest() {

        BasicAuthenticator auth = null;

        try {

            auth = BasicAuthenticator.fromFile(testJsonFile);

        } catch (IOException e) {

            Throwables.propagate(e);

        }

        request = mock(HttpServletRequest.class);

        testRule = ResourceTestRule

            .builder()

            .addProvider(AuthFactory.binder(new BasicAuthFactory<>(auth, "test", AuthUser.class)))

            .addResource(new TestAuthResource())

            .addProvider(new ContextInjectableProvider<>(HttpServletRequest.class, request))

            .build();

    }

And the implementation for ContextInjectableProvider:

package com.rtr.wizard.jersey;

import org.glassfish.hk2.utilities.binding.AbstractBinder;

public class ContextInjectableProvider<T> extends AbstractBinder {

    private final Class<T> clazz;

    private final T instance;

    public ContextInjectableProvider(Class<T> clazz, T instance) {

        this.clazz = clazz;

        this.instance = instance;

    }

    @Override

    protected void configure() {

        bind(instance).to(clazz);

    }

}

--
You received this message because you are subscribed to a topic in the Google Groups "dropwizard-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dropwizard-dev/VInOW_ebiAc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dropwizard-de...@googlegroups.com.

[Saem Ghani]

Thanks for documenting your experiences Gene, I'm glad to see most of them reflect my own, for the most part.

On Monday, November 17, 2014 10:56:48 AM UTC-8, Gene Trog wrote:

<snip>

6.) In one of the unit tests for a filter, I was using the dropwizard resource test rule.  The filter itself depended on the injection of HttpServletRequest, which seemed to no longer be injected by default, so, I had to write my own HK2 injection binder to inject a mocked version of this value.

I'm stuck with some of tests erroring out with an NPE, the trace is as follows -- the odd thing is that the debugger in Intellij won't even break at any breakpoint I set within the failing test :

java.lang.NullPointerException

at org.glassfish.jersey.test.JerseyTest.tearDown(JerseyTest.java:634)

at io.dropwizard.testing.junit.ResourceTestRule$1.evaluate(ResourceTestRule.java:144)

at org.junit.rules.RunRules.evaluate(RunRules.java:20)

at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

at org.junit.runners.Suite.runChild(Suite.java:127)

at org.junit.runners.Suite.runChild(Suite.java:26)

at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)

at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)

at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)

at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)

at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)

at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

at org.junit.runner.JUnitCore.run(JUnitCore.java:160)

at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)

I was wondering if this was similar to what you encountered knowing you had to create an HttpServletRequest injector, or an error you encountered that perhaps wasn't documented in the previous list?

<snip>

[Gene Trog]

I don't recall running into a NPE on Jersey's tearDown(), sorry.  So, if you go to the definition for JerseyTest.tearDown(), attach the source, putting a break point in it doesn't cause IntelliJ to stop?

--

[Saem Ghani]

Hi Gene,

It looks like it's the teardown of the @ClassRule annotation for the resource builder.

Yes, I can get the breakpoint at the teardown call, the container for the JerseyTest created by the rule is definitely null, I'm just not sure why. I'm trying to dig through it, I was hoping someone else might have run into this issue.

[Gene Trog]

The only strange thing I remember about class rule fields is they have to be public and static or else they don't get set properly at run-time.

[Saem Ghani]

In all the classes they're public static, this happens with or without the Mockito Test Runner, been following the test examples here in the current master: https://github.com/dropwizard/dropwizard/tree/master/dropwizard-example/src/test/java/com/example/helloworld/resources

[Saem Ghani]

On Wednesday, December 3, 2014 10:04:04 AM UTC-8, Saem Ghani wrote:

In all the classes they're public static, this happens with or without the Mockito Test Runner, been following the test examples here in the current master: https://github.com/dropwizard/dropwizard/tree/master/dropwizard-example/src/test/java/com/example/helloworld/resources

It seems like inside org.glassfish.jersey.test.JerseyTest:609, the createTestContainer is failing, I'm digging deeper in the stack traces -- still a Java noob, so it's slow.

I think one of the differences might be that I have a maven parent project with multiple dropwizard applications as sub-modules of it. I'm running all tests across the board, and if I run them individually, they work just fine. I'm home this last tid bit triggers something for someone as a possible cause of this.

[Saem Ghani]

Problem solved!

So the issue is that we're using the swagger-jaxrs, which was including javax.ws.core:jsr311-api. This overlaps and "hilarity" ensues.

I chased down a number of these and this is what my swagger dependency in maven looks like:

<dependency>

            <groupId>com.wordnik</groupId>

            <artifactId>swagger-jaxrs_2.10</artifactId>

            <version>1.3.10</version>

            <exclusions>

                <exclusion>

                    <groupId>javax.ws.rs</groupId>

                    <artifactId>jsr311-api</artifactId>

                </exclusion>

                <exclusion>

                    <groupId>joda-time</groupId>

                    <artifactId>joda-time</artifactId>

                </exclusion>

                <exclusion>

                    <groupId>org.javassist</groupId>

                    <artifactId>javassist</artifactId>

                </exclusion>

            </exclusions>

        </dependency>

Also note, that javassist version is slightly off, I currently include 3.18.2-GA and 3.18.1-GA, the latter is because of dropwizard, but the former are due to swagger and us including Hibernate's entity manager 4.7 Final. I haven't chased that down any further, after this, everything seems to be building fine. I'll post again, if there are any further migration issues.

[Valerii Pekarskyi]

Hi guys,

I've caught the same NPE during migrating our codebase to dropwizard 0.8.0-rc2 (jersey2 is crucially needed now). So yes, for the next generation who is going to have the same problem - check your 'mvn dependency:tree -Dverbose' to see why your jersey dependencies of io.dropwizard:dropwizard-jersey are omitted. Then check how many javax.ws.rs.core.Application you have in your code (at least one of them does not have getProperties() method).

And one small request to the dropwizard developers - please update the ResourceTestRule apply method because now it swallows any exceptions happened in setUp and evaluate if anything bad happened in tearDown (and it is painful to debug). If an issue should be created somewhere, please point me there.