Confluent Community License and Confluent Kafka Connectors

[Andrew Otto]

Hi,

We'd been preparing to use Kafka Connect at the Wikimedia Foundation, the non-profit that runs Wikipedia, but have run into a blocker.

Our first and main use case for Kafka Connect is to replace our usage of Camus to handle ingestion from Kafka into HDFS.  Camus has our only remaining old Kafka client. Replacing it would allow use to finally require authenticated and encrypted Kafka connections, better protecting user PII that flows through our Kafka clusters.  We use JSON and JSONSchema, so I had planned to finish implementing a community converter[1] to handle conversion to/from JSONSchemas to Kafka Connect.

The blocker is the Confluent Community License.  At the WMF, we "strive to use open source tools over proprietary ones."[2].  Wikimedia’s free cloud service has the rule:  "Proprietary software: Do not use or install any software unless the software is licensed under an Open Source license."[3]

This means that we have to make choices based on not only what is the best tool for the job, but also what is the best free/libre tool for the job. Kafka Connect HDFS is the best tool for the job of Kafka -> HDFS+Hive ingestion, but unfortunately as of December 2018 it is no longer free/libre.

I understand why Confluent chose to go with a non-free/libre 'source-available' license for its tools like KSQL and Schema Registry.  However, I don't see why that license had to be applied to all non-core components that Confluent develops, like the Kafka Connect HDFS Connector.

I don't personally have a stake in the recent controversy[4] and trend[5] of companies combating AWS-like use of their open source software with non-free/libre licenses.  I mostly just want to use Kafka Connect with Confluent connectors. Confluent hasn't changed licenses for librdkafka or confluent-kafka-python or Confluent Dockerfiles, so I don't see why other non-SaaS-like softwares like Kafka Connect plugins had to be changed.  I'm currently most interested in Kafka Connect HDFS, but I'm sure other Confluent developed connectors will be useful too.

So I have a request for Confluent:  For those softwares that are not relevant to the spirit of the CCL (i.e. combating SaaS competition), would it be possible to revert them to a free/libre license like Apache 2.0?

I know this request is a long shot, but I have to ask :)

If the license can’t be free/libre, then at WMF we’ll have to find a solution other than Kafka Connect, or we’ll have to fork the pre-CCL version of Kafka Connect HDFS.

Let me know if there is a better place for this request than the confluent-platform mailing list.

Thanks,

- Andrew Otto

  Senior Systems Engineer

  Wikimedia Foundation

[1] https://github.com/ottomata/kafka-connect-jsonschema

[2] https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Guiding_Principles#Freedom_and_open_source

[3] https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#What_uses_of_Cloud_Services_do_we_not_like?

[4] https://www.techrepublic.com/article/mongodb-ceo-tells-hard-truths-about-commercial-open-source/

[5] https://www.datacenterknowledge.com/open-source/confluent-creates-new-open-source-license-stop-cloud-poaching

[Sendoh Daten]

Hi Andrew,

I'm not representing confluent, and merely a Kafka-Connect lover. In the market lots of services selling kafka ecosystem. They earn benefit from kafka and the related frameworks which should be noticed.

The community license simply means the competitor must not use those frameworks. 

The definition of open-source varies, presto/elasticsearch as the other examples, I found them free to use(w/ some conditions). The code quality is very high and lots of tutorials and CI/CD friendly, and this comes with more cost and care comparing to some other Apache projects. That said after you deploy, you find it not straightforward to maintain and upgrade the version, and has to spend some time to implement the missing parts. 

The community license does not follow strictly open-source and I agree, but the contribution you make will be used freely by other people, if they don't compete with confluent, which makes sense to me. I would view it like this - the open-source definition evolves due to the reality.

Best,

Sendoh

Andrew Otto於 2019年7月9日星期二 UTC+2下午5時38分02秒寫道：

[Ben Stopford]

Hi Andrew

Thanks for the well-articulated request. Really, the vast majority of users should be unaffected by the license change as it is very specific about what purposes are excluded. I do appreciate that it doesn’t work well if your organization has a blanket ban on software that is not fully open source. Would Wikimedia consider an exception? 

Regarding your request, we don’t have any plans to change the license on server-side components, again, in the near future. I do take your point here though. 

Best regards

Ben

Office of the CTO

Confluent

[Andrew Otto]

Thanks for the response Ben!

FYI, I also emailed Neha, and she's put me in touch with Mike Agnich, who has said he's going to speak with his team about this and respond in a week or so.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/4aacf11d-cea8-4689-9a60-95ad1c28b42d%40googlegroups.com.