Support Hadoop 3

[Guillaume Bibens]

Hello,

I saw on github that a version 4 of the Cascading library was ongoing. Is support of Hadoop 3 in the scope of the new version? Any idea of when new version will be available?

Thanks,

Regards

Guillaume

[Chris K Wensel]

Hi

My plan is to release Cascading 4.0 for Hadoop 2 only.

When C4 will be released, I’m unsure. There are a few things that need to change overall, namely

- maven package name needs a new group name (we no longer own cascading.org)

- publish through jcenter (we no longer own conjars domain so may go away)

- migrate conjars to blob and make readonly (it is a security risk and may go away)

- setup new site on new domain (we don’t own cascading.org)

- probably need a new github org (we don’t own the github org either)

- look at merging cascading-local into the main project

- refresh the user guide

i’m sure there are more things, and many can be deferred, but just those few items are hard to think about let alone do during free time (there is no corporate sponsor for Cascading so community work is after hours).

It would be great if someone forked wip-4.0 and started on a 4.1 with tandem hadoop3 support (like we had 1 and 2 support in earlier Cascading releases). 

but it’s never easy having a shared code base across multiple compilation units.

also, Hadoop will be pinned to Java 8 for the foreseeable future https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions

but cascading local mode libraries would thrive under Java 11 (which is planned for Hadoop3). making dual H2 and H3 support a little harder to figure out, and less desirable.

if anyone wants to take this on, feel free to open an issue or post a new thread to discuss.

ckw

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/7d726fa6-124f-43b0-ac2c-2183bcc6ef8ao%40googlegroups.com.

—

Chris K Wensel

ch...@wensel.net