Location for new "non-core" modules

[mza...@apple.com]

Hey dropwizard-dev,

We've got some modules we'd like to open source that we use internally at Apple. Some of those might make sense to be included in the core Dropwizard project, but others definitely don't, and it would make much more sense to have them live outside of the Dropwizard repo. 

We're hoping we might be able to contribute these to the Dropwizard organization (https://github.com/dropwizard) instead, as that seems like a place where it would make sense for Dropwizard extensions to live. However, we wanted to consult you guys to see if you had any strong preferences one way or another, or if there's any process we should be aware of in order to add a project to the Dropwizard GitHub organization.

It might be worth mentioning that it would be easier for us to open source them if they can live in the Dropwizard organization, rather than a brand new spot, due to how our corporate open source policy works. 

Thanks!  

[Jochen Schalanda]

Hi Michael,

Thanks for reaching out!

Depending on the specific modules, it might make sense to adopt them in the Dropwizard organization on GitHub. Alternatively, you can also host them in your own accounts and add them to the module directory at https://modules.dropwizard.io/, as Nick described in his last email.

Do you have a list and description of the modules you're planning to open-source?

Also, what license are they currently under or which licenses do you plan to use for them? All projects under the Dropwizard umbrella are using the Apache License 2.0.

Are there any other things to consider copyright- or license-wise when you're contributing these modules?

And finally, do you plan to keep contributing to the modules you're going to open-source or will it be more or less a "code drop" without any further involvement?

Best regards,

Jochen

[mza...@apple.com]

Hey Jochen! 

> Do you have a list and description of the modules you're planning to open-source?

We do! 

So we have three modules that we think make sense to live in the core Dropwizard project. Two of which are in a pretty ready state to be contributed, but because 2.0.0 is already in RC stage, might not make sense until that ships.

The ones we have are:

• A module containing a bundle that allows for configuration of various headers that are important for services that are accessed by web browsers. This includes: HTTP Strict Transport Security (HSTS), X-Frame-Options, X-Content-Type-Options, X-XSS-Protection (XSS), Content Security Policy (CSP), and Cross-Origin Resource Sharing (CORS).
• A module containing a more-feature rich and extendable admin page, in short. This would need a little bit more work before we could contribute it, because there’s some cleanup work we’d need to do, and some changes to make it more flexible in how it’s used.
• A bundle containing improved health check functionality. Provides functionality to allow a user to associate a schedule with any registered health checks to run on a recurring basis, and to allow marking which checks are critical to service health (i.e., if a dependency is down, it should bring the service down with it). The bundle then allows exposing another health check URL on the server port, which simply returns the aggregated view of the application’s health, rather than trigger checks on every health check.

We also have some additional potential contributions that we don’t think would make sense to live in the core Dropwizard project, but rather (hopefully) could live in the Dropwizard organization. Our reasoning for including these would be that we already maintain these internally, and keep them up to date with the underlying client libraries we’re wrapping, AND we do so in a consistent way, so we figured it might make sense to broaden the audience for these, as there’s no good reason not to. 

• A module providing integration with Kafka.
• A module providing integration with Redis.
• A module providing integration with Cassandra.
• A module providing integration with FoundationDB.

There may be some others I’m missing here, that we may propose down the line, but this should be the majority of what we’d like to publish initially. 

> Also, what license are they currently under or which licenses do you plan to use for them? All projects under the Dropwizard umbrella are using the Apache License 2.0.

Are there any other things to consider copyright- or license-wise when you're contributing these modules?

We would publish these apps using the Apache License 2.0, as all other Dropwizard projects do.

> And finally, do you plan to keep contributing to the modules you're going to open-source or will it be more or less a "code drop" without any further involvement?

We plan to keep contributing to the modules we contribute. We maintain these internally on an ongoing basis already, so we’d just continue to do so.

[mza...@apple.com]

> We would publish these apps using the Apache License 2.0, as all other Dropwizard projects do.

 

Not apps, libraries. :) 

[babc...@umich.edu]

While I'd prefer and wait until after 2.0 is out before including them into dropwizard, these modules seem well thought out: finer grained healthchecks with the ability to have them cached, and deeper support for HTTP headers seem great!

How backwards compatible are these changes?

Are you thinking submitting the healthcheck code against the metrics repo, the dropwizard repo, or a new repo under the dropwizard organization?

[jpl...@gmail.com]

I'd also like to see the HTTP header module brought into the core.  Palantir also has their own module that allows configuring the headers at https://github.com/palantir/dropwizard-web-security

-Justin

[mza...@apple.com]

> How backwards compatible are these changes?

They're totally backwards compatible. They're purely additive bundles.

If there's interest in pulling them into 2.0.0, we'd be happy to contribute them, but we're also fine holding off. 

[Jochen Schalanda]

Hi Michael,

I also think that all of the mentioned libraries sound very interesting. :-)

In order to move quickly on this issue and not delaying it unnecessary, I'd suggested putting all of the described libraries into the Dropwizard GitHub organization (http://github.com/dropwizard/) for now.

If you only want to push the Git repositories to GitHub, I will happily create the necessary repositories and add your GitHub accounts with write privileges to these new repos.

If you already have repositories for these projects in your own GitHub organization and would like to migrate issues, pull requests, projects, and wikis as well, we'll have to transfer the GitHub repositories as described at https://help.github.com/en/articles/transferring-a-repository. In this case, please reach out to me so that we can prepare everything.

Cheers,

Jochen

[mza...@apple.com]

Hey Jochen!

So let's just focus on the three probably most widely useful of the modules I mentioned so far, which would be the health, admin, and web header ones.

> If you only want to push the Git repositories to GitHub, I will happily create the necessary repositories and add your GitHub accounts with write privileges to these new repos.

Sure, if we think these belong in a separate repo under the Dropwizard org, we'll want the following repos created (subject to agreement on the naming):

1. dropwizard-health
   
2. dropwizard-admin
3. dropwizard-web

Otherwise, we can include them in the core repo as well, if that's preferable, in which case we can submit PRs against that. We were thinking they might make sense there, but I don't know that we feel super strongly about that.

I'll start prepping these for contribution, so that once we know where they should go, and potentially have the repos set up, we can get them contributed quickly.

Thanks!

[Michael Zamani]

So, big thanks to Jochen for creating the projects for us under the Dropwizard org.

I just pushed the first repo I've had a chance to clean up for upstreaming, dropwizard-kafka. More to come soon!

https://github.com/dropwizard/dropwizard-kafka

[Jeremiah Adams]

Can you share a high overview of use cases you h ave for the kafka consumers in dropwizard? We use dropwizard for our rest services,  and apache samza for kafka stream processing. I am interested to see how you are using kafka consumers in dropwizard.

[Michael Zamani]

Hey Jeremiah,

That’s a good question, and it ultimately will come down to what you need.

For instance, maybe you have an existing Dropwizard that you’d like to also consume messages in addition to whatever else it’s doing. Maybe having a http health check, and an admin page that’s familiar is beneficial. 

Spark streaming, and Samza (which sorry if this is wrong, not super familiar with it) would require you to set up a cluster, and maybe that’s not worth it in all cases, though I can definitely see them being useful pretty broadly.

Or maybe you’re just wanting to use the Kafka streams APIs (which granted, what I have now in dropwizard-Kafka doesn’t support), which again doesn’t require a cluster, but gives some of the features of spark streaming (and Samza?).

Just some thoughts, hopefully that clarifies.

[Jeremiah Adams]

Thanks for the info. Yes, Samza occupies the same space as Spark Streaming but requires no tuning (been a while since I've used spark so this may have changed.) 

I see the dropwizard consumer as more utility/administrative as well but was curious. I'll likely be using your project in admin/monitoring applications. Thanks for sharing it. I have more questions specific but will move it to Github issues and the like.

-jeremiah

[Michael Zamani]

Just another update, I pushed our code for https://github.com/dropwizard/dropwizard-foundationdb

The next one on my plate is the dropwizard-health module, which I'm hoping to push today or tomorrow.

[Michael Zamani]

(Sorry for the spam) 

dropwizard-health is available now as well: https://github.com/dropwizard/dropwizard-health/

[Michael Zamani]

Would it be possible for us to get another repo created in the Dropwizard org? Let's call this one dropwizard-kubernetes. It's a client integration for the fabric8 Kubernetes Client (https://github.com/fabric8io/kubernetes-client).

[Michael Zamani]

Bump on this:

Would it be possible for us to get another repo created in the Dropwizard org? Let's call this one dropwizard-kubernetes. It's a client integration for the fabric8 Kubernetes Client (https://github.com/fabric8io/kubernetes-client).

[Jochen Schalanda]

Hi Michael,

I've created the mentioned repository and added you and Peter to the team.

Cheers,

Jochen