What is transpiler.js?

[Nate]

Hey Everyone!

I noticed a few changes to base.js (and the closure library)... One thing I found was a compiled blob (transpiler.js [1]). I noticed something similar in the closure-deps tool (jsfile_parser.js [2]). Code/messages in those both looks familiar... Digging around led me to this note among other references. [3]

I'm think what I'm seeing is impressive --- which, is a Java to Javascript source transformation of some part of a large compiler.

Is the closure compiler (or some subset) passed through J2CL [4] to produce transpiler.js (and jsfile_parser.js) in the closure library? Is there some build target to reproduce these files?

Thank You!

Nate

[1] https://github.com/google/closure-library/blob/master/closure/goog/transpile.js

[2] https://github.com/google/closure-library/blob/master/closure-deps/lib/jsfile_parser.js

[3] https://github.com/google/closure-compiler-npm/blob/master/packages/google-closure-compiler-js/readme.md

[4] https://github.com/google/j2cl

[Nick Reid]

Hey Nate,

You've correctly deduced that transpile.js is a portion of the Closure Compiler transpiled with J2CL and then run through the compiler.

This system currently exists to support an old style of JS tests that are embedded in HTML files. Dependencies of some of those tests have evolved since the tests were written, to use syntax features that aren't available in older browsers. Without some kind of intervention, tests in those browsers would suddenly have parse errors. Normally, we'd fix this by transpiling the test and it's deps on the server, before sending the JS to the browser; however, the HTML nature of these tests makes that difficult. transpile.js represents a workaround in which transpilation happens in the testing browser itself, after the test code is sent (as strings), but before it's evaled in the JSVM.

This ability is related to an older effort by Closure Compiler to work on systems that don't have a JVM. This is pretty common among frontend developers, but they do tend to have JSVMs. So there was a period where the entire Closure Compiler was available as a JS program.

Both of these artifacts are now something we eventually want to clean up. J2CL is not free; it demands compromises in Java code which make compiler development more costly. Additionally, we now support the no-JVM usecase using Graal native code images, which we've seen totally replace usage of the J2CL-ified compiler. transpile.js is one of the last remaining loose ends. The full J2CL compiler has been eliminated.

As such, there is no open source build target for creating transpile.js. We don't want it to grow additional dependents. Work is ongoing to eliminate the old tests that use it. When that work eventually finishes (no timeline presently) we will most probably delete transpile.js from the repo.

Nick

[Nate]

Hey Nick!

Thanks for the details. Good to know. If it's on the way out, we'll probably avoid looking at that code path further, and disable transpile.js via the goog.TRANSPILE "never" option.

I think I understand the function of this module, if I'm not mistaken: Parts of closure library adopted es6 syntax (made available via "goog.module"), and transpilation is required to make things continue to work in older (yet still supported) browsers (e.g. IE9) — without having to run or install anything extra.

Is the point which transpile.js is to be excised, the same point which support for a browser generation stops?

Curious if the transpiled compiler was at all "maintainable" in its Javascript variant. Did J2CL output a single binary, or did it output Javascript in separate modules in separate files which could then be edited alone and recompiled?

I can see trade-offs being made to annotate and also with various "stub" classes.. Lo, neat work.

Thank You!
Nate

[Nick Reid]

On Saturday, February 27, 2021 at 9:57:04 a.m. UTC-8 Nate wrote:

Hey Nick!

Thanks for the details. Good to know. If it's on the way out, we'll probably avoid looking at that code path further, and disable transpile.js via the goog.TRANSPILE "never" option.

I think I understand the function of this module, if I'm not mistaken: Parts of closure library adopted es6 syntax (made available via "goog.module"), and transpilation is required to make things continue to work in older (yet still supported) browsers (e.g. IE9) — without having to run or install anything extra.

Almost, but it's specific to these HTML tests. It's fairly easy to do transpilation on the server side, when the testing browser requests the code, if all the test code is in JS files. 

Is the point which transpile.js is to be excised, the same point which support for a browser generation stops?

I don't think those two events are necessarily connected. transpile.js will live as long as we support these HTML. 

Curious if the transpiled compiler was at all "maintainable" in its Javascript variant. Did J2CL output a single binary, or did it output Javascript in separate modules in separate files which could then be edited alone and recompiled?

J2CL emits more-or-less human readable Closure-style JS code, hence the name "Java to Closure". However, the JS is a reflection of the original Java code that the compiler is written in. You don't want to commit/edit that JS because it will get out of sync. It gets regenerated every time transpile.js is built to make sure it stays up to date.

The generated code also has a bunch of "goop" needed to replicate the subtle semantics of Java in a JVM. The best example is when static initializers execute: in Java static code executes at class-loading, when a class is first referenced by the running program, in JS classes may be loaded/executed well before anything references them, simply because they were included in the same HTTP response. It's basically impossible for a human to get this right consistently.

[Nate]

On Saturday, February 27, 2021 at 1:47:11 PM UTC-6 Nick Reid wrote:

On Saturday, February 27, 2021 at 9:57:04 a.m. UTC-8 Nate wrote:

Hey Nick!

Thanks for the details. Good to know. If it's on the way out, we'll probably avoid looking at that code path further, and disable transpile.js via the goog.TRANSPILE "never" option.

I think I understand the function of this module, if I'm not mistaken: Parts of closure library adopted es6 syntax (made available via "goog.module"), and transpilation is required to make things continue to work in older (yet still supported) browsers (e.g. IE9) — without having to run or install anything extra.

Almost, but it's specific to these HTML tests. It's fairly easy to do transpilation on the server side, when the testing browser requests the code, if all the test code is in JS files. 

I recall the older testing patterns (and made use of them); I think it was goog.require starting from an html file. Though, we did eventually move most to a generated template pattern. Noticed dom fragments [1] along with tests [2]; and some kind of generator [3]. It appears most of the old style tests are migrated in closure library after quickly scanning the commit log.

I presume this is the pattern you refer to? Maybe I'm mistaken. Do you have an example of those specific html tests in the library, or are they internal?

Many Thanks!
Nate

[1] https://github.com/google/closure-library/blob/8d73437c3390cd3aed9c69ab1fcdd9b958b998bd/closure/goog/ui/activitymonitor_test_dom.html

[2] https://github.com/google/closure-library/blob/8d73437c3390cd3aed9c69ab1fcdd9b958b998bd/closure/goog/ui/activitymonitor_test.js

[3] https://github.com/google/closure-library/blob/8d73437c3390cd3aed9c69ab1fcdd9b958b998bd/scripts/generate_closure_unit_tests.js

[John Lenz]

Nick was referring to Google internal tests and infrastructure, not the tests of the library itself.

--

---
You received this message because you are subscribed to the Google Groups "Closure Compiler Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to closure-compiler-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/closure-compiler-discuss/6a581586-9b8e-4ede-ba4a-29972ef7f426n%40googlegroups.com.

[John Lenz]

I also wanted to note that we did have a working J2CL build of the compiler not that long ago.   You can go back if you wanted but it was rough in a number of important ways (missing unicode support in the parser, unfortunate differences in the command line, etc).