Ant, ZipScanner, and GWT

[Colin Alworth]

We have two issues tracking the dependency that GWT has on Ant, https://github.com/gwtproject/gwt/issues/9690 and https://github.com/gwtproject/gwt/issues/9677. I've taken a little time and produced a minimal set of classes copied from ant which appear to provide a working ZipScanner. For the moment, this lives in its own git repository at https://github.com/vertispan/ant-zipscanner, and is not deployed anywhere.

I spent a while testing with latest ant, 1.10.8, but semantics have changed with the zip/directory scanner API such that leading "/" characters no longer match - rather than break behavior, I've instead switched to simply using the 1.6.5 code, and modified that for our purposes. This has the advantage of being substantially smaller than 1.10.8 - 18 types instead of 46, and of those 18, 11 are required for ant's own implementation of the zip format, so the other 7 are needed to scan a zip and match contents. It is quite likely that this could be pruned further, but might come at a cost when updating to some later ant version (should we decide to do that).

This repository is arranged in three commits:

• Create a script to copy the classes we need from the ant repo, and a pom to ship these contents. The script is handy when iterating on a new version, but probably won't be needed unless we do attempt to use a new version
  
• Copy all required classes from the tag "rel/1.6.5" in the upstream ant repo
• Comment out anything which didn't compile due to missing dependencies (we have no use for Project or refs, we only call ZipScanner methods directly), or which modify the filesystem in some way (to limit risks when depending on this codebase

The resulting output is shaded into a new package by the pom so that it will not interfere with ant when GWT is compiled (or if GWT is invoked from ant).

When GWT is updated to both include this jar and also to reference these new shaded classes instead of the originals within ant, all tests in dev/ pass and all samples compile, though I haven't yet run all of the other tests.

So, my question is then how to proceed. First of all, is this approach acceptable in terms of maintenance, or would we prefer reimplementing the API from scratch in order to drop this dependency, or just updating to the latest ant version for this one API?

Assuming we like the approach, should we incorporate these copied/modified sources into GWT itself, or add an additional jar to gwtproject/tools, with instructions on how to update it, and from there include it into the project? A somewhat related option could be releasing this jar to maven central under something like org.gwtproject.ant:ant-zipscanner, and maintaining it in the github.com/gwtproject umbrella.

Finally, is there any objection to staying with 1.6.5, or should I see if we can use a later version, or update GWT's internals to use the new behavior around leading slashes, etc?

[Thomas Broyer]

To minimize the work, and because 1.6.5 works well for us without known vulnerability, I would either copy/paste the code into gwtproject/gwt or provide it in a JAR in gwtproject/tools and call it a day. We already have a copy of Rhino (with changes for JSNI), and a slimmed down version of jsilver containing only the streamhtmlparser.

No need to try to update and risk a breaking change, and no need to make it "more maintainable" as I'm sure it'll be a one-shot anyway (and that decision could be revisited in the future should we need to make changes to those Ant classes).

[Colin Alworth]

So, given either "make a git repo on gwtproject/ and add a jar to gwtproject/tools" with the minimal history, or a single commit adding all already-modified classes to gwt in one go? I should be able to turn out either change fairly quickly, once we decide.

Adding to GWT directly would be somewhat lower friction (no need to ship a jar to central, easier to further tweak if something is screwy), but as I said, loses that tiny bit of history/context. Like you said, a forked jar is not at all new for the project to have, and is a nice way to say "this is external, even if we tweaked it a bit". For the zip distribution I imagine we'd shade it in to the overall zip, but for the m2 release it would probably be an external jar (since it will hopefully never change).

-- 

  Colin Alworth

  co...@colinalworth.com

--

You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/7e0c1856-2835-4254-a8ed-b73a03de1ea7o%40googlegroups.com.

[Peter Donald]

Possibly a dumb question ... but isn't it possible to just manually
vendor in the source files to GWT? I can't imagine there is a whole
lot of dependencies that would need to be maintained. Just strip out
the gunk you don't need.

> To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/fd913f25-f6a0-4820-ad96-ff95a064cdb0%40www.fastmail.com.



--
Cheers,

Peter Donald

[Colin Alworth]

If I understand you, that is what I'm doing - I only the required 18 classes (with some members removed so that I don't need more than that), and am just seeking an opinion other than my own as to whether it is better to checked them straight into gwt as com.google.gwt.thirdparty.ant... classes, or into a standalone jar (to be clearer that this is external work+license, and have a pointer to where that work originated).

Ant works as a standalone jar as it is, so this would include no external dependencies, just these few classes either added directly to gwt, or to gwt's tools repo as a jar. The manual work is done, and it was fairly minimal (after i figured out the minimal set, and that latest ant doesnt work), just want to get thoughts on packaging/licensing, if there are any considerations.

--
Colin Alworth
co...@colinalworth.com

> https://groups.google.com/d/msgid/google-web-toolkit-contributors/CACiKNc5vrgR1wWz1kEn1HSU0O76ks7tr1BddMY6MfvMc47vnEQ%40mail.gmail.com.
>

[Peter Donald]

On Sat, Jun 13, 2020 at 11:11 AM Colin Alworth <co...@colinalworth.com> wrote:
> If I understand you, that is what I'm doing - I only the required 18 classes (with some members removed so that I don't need more than that), and am just seeking an opinion other than my own as to whether it is better to checked them straight into gwt as com.google.gwt.thirdparty.ant... classes, or into a standalone jar (to be clearer that this is external work+license, and have a pointer to where that work originated).

If it was me, I would just pull the source in and place it in
com.google.gwt.util.tools.zipscanner and flatten all the files into
that package. No need to mark it as vendored as you never intend to
pull from upstream again as they have moved in an incompatible
direction. They are license compatible as both Apache-2.0 and as long
as the GWT project does not have a policy against including code that
is copyright to another organization in source tree then that seems
like the easiest option going forward.

--
Cheers,

Peter Donald

[Thomas Broyer]

On Friday, June 12, 2020 at 11:05:34 PM UTC+2, Colin Alworth wrote:

So, given either "make a git repo on gwtproject/ and add a jar to gwtproject/tools" with the minimal history, or a single commit adding all already-modified classes to gwt in one go? I should be able to turn out either change fairly quickly, once we decide.

Adding to GWT directly would be somewhat lower friction (no need to ship a jar to central, easier to further tweak if something is screwy), but as I said, loses that tiny bit of history/context. Like you said, a forked jar is not at all new for the project to have, and is a nice way to say "this is external, even if we tweaked it a bit". For the zip distribution I imagine we'd shade it in to the overall zip, but for the m2 release it would probably be an external jar (since it will hopefully never change).

If you're worried about the size of the JAR, let's kill legacy DevMode instead ;-)

(Google possibly even has the patch ready, as IIRC they already removed it in their internal repository)

Let's keep it simple and just ship it inside the gwt-dev JAR; our other repackaged dependencies are in there already (protocol buffer, streamhtmlparser, GSS, Guava, jscomp –for sourcemaps–, etc.)