Hello John.
> It's great to hear from you! It looks like there are considerable
> differences between our two projects. It's hard for me to figure out where
> to start in this email, so I apologize if this email is somewhat
> disorganized.
Thanks for such detailed answer! I'll do my best to comment and reply on the
subjects outlined by you.
> I took a brief look at bck2browser, so forgive me if I'm incorrect on a few
> points. Here are the points that set Doppio apart from bck2browser (some of
> which you mentioned):
>
> - We have a tech report that will probably shed some light on our design
> and goals:
https://web.cs.umass.edu/publication/details.php?id=2301
>
> - *Doppio is currently only a bytecode interpreter*. That's why we're
> slow right now.
Yeah, slow, but more correct. I really gave up on supporting classical
(wait/notify) multithreading.
> In contrast, bck2browser appears to be an offline
> compiler, correct?
I'd rather call bck2brwsr "online" - the compilation happens in the browser.
> (I wonder if Doppio can run bck2browser...)
I am pretty sure you can (when you can run javac). The bck2brwsr VM is in
http://source.apidesign.org/hg/bck2brwsr/file/release-0.7.2/rt/vm
and it using as small subset of OpenJDK classes, as possible
http://source.apidesign.org/hg/bck2brwsr/file/56abd247f421/rt/emul/mini
> We hope to
> add a JIT compiler over the next year.
Everything that is missing is potentially an object for co-operation. I don't
expect you to give up on what works (I don't want to do that either) -
understanding mutual wishes and seeking synergy is what I hope for: Could
bck2brwsr be your JIT compiler ;-?
> - *Doppio can simulate synchronous blocking functionality in terms of
> asynchronous browser functionality.* This is *the* most important piece
> of functionality that sets Doppio apart from any other language port to
> JavaScript that we are aware of, such as Emscripten.
Yeah, this requires simulation of stack on heap (which obviously means more
indirection and slowness). It is something I intentionally left from the
design of bck2brwsr. Rather than that we contacted a V8 expert to tell us how
to allocate registers (e.g. local variables and stack) effectively, so V8 can
properly guess their types.
> This is how we can do threads, synchronous file I/O, stdin/stdout
> emulation, and long-running loops -- we have the ability to pause
> execution and resume, which we plan to maintain at function call boundaries
> when we eventually add a JIT compiler. This ability comes at the sacrifice
> of speed -- we need an explicit representation of the stack at enough level
> of detail to pause and resume -- but I believe that we can still get pretty
> darn fast once we start compiling. There is no other way to emulate threads
> or synchronous functions in terms of asynchronous browser functionality.
Right. Ability to support multithreading is the most important advantage of
Doppio from my point of view.
For some applications single threaded execution is OK. Such applications could
start with using bck2brwsr. If/when they see a need for parallelism, could
they "just" replace bck2brwsr with Doppio? Without changing their code? Just
repackaging?
> This is also one of the core research portions of Doppio, since it's a
> fairly weird way to port a language.
>
> - *Doppio's primary goal is the ability to execute existing programs in
> the browser. *This includes applets, desktop programs, and programs
> written in other languages that target the JVM (e.g. JRuby/Jython). In
> contrast, it looks like Bck2Browser is aimed at allowing people to write
> *new* programs in Java for the browser with minimal overhead.
Excellent description of the intents. It matches my own understanding of the
goals.
> Note that we plan to support the Bck2Browser use case once we start
> working on a clean way to interop between JavaScript and Java programs.
Missing? That is an opportunity! Would you consider using
http://hudson.apidesign.org/hudson/job/bck2brwsr.javadoc/lastSuccessfulBuild/artifact/rt/core/target/site/apidocs/index.html
especially @JavaScriptBody?
> It hasn't been a development priority yet, but I have a feeling it will be
> relatively soon. We won't be as low-overhead, though, since we are using
> the normal JCL.
>
> - *Doppio uses the OpenJDK Java Class Library.*
Bck2Brwsr uses a forked version of OpenJDK class libraries based on build
JDK7-b147. Here is an example:
http://source.apidesign.org/hg/bck2brwsr/file/56abd247f421/rt/emul/mini/src/main/java/java/lang/String.java
it contains mostly original OpenJDK code enhanced with @JavaScriptBody and
other annotations.
Btw. you must have some mechanism for implementation of native methods as
well. Do you just consider it temporary and internal?
> One of the issues with
> projects like Emscripten and I imagine bck2browser is that a large amount
> of development effort goes into porting and maintaining JavaScript copies
> of core libraries. Since Doppio aims to be compatible with unmodified JVM
> programs, it is compatible with the non-native portion of the JCL.
Bck2Brwsr runs non-native bytecode unmodified as well.
> I want to note that I'm not ragging on Emscripten or bck2browser -- both
> are really cool projects that serve a slightly different purpose. :) I'm
> just trying to illustrate the differences, and justify them.
>
> I have some questions for you, though:
>
> - *How is bck2browser different from GWT?* We get this question a lot
> about Doppio, and I imagine you must get it somewhat often, too.
Yeah, I'd got the question so many times that I wrote a wiki page about that:
http://wiki.apidesign.org/wiki/GWT
> haven't used GWT directly, but I know that GWT has limited core-library
> support, only does whole-program compilation (so you need to have
> everything available at compile-time), and only supports Java code (as
> opposed to arbitrary JVM programs). But I also know it has an API for using
> external JavaScript libraries from Java code.
Right. The problem is that the GWT's JSNI binding is written in a comment -
e.g. for us who process the bytecode it is not available. However (and this is
a benefit of being part of NetBeans team), I convinced my Java editor colleague
to write a hint that converts GWT's JSNI comment into @JavaScriptBody
annotation:
http://source.apidesign.org/hg/bck2brwsr/file/56abd247f421/ide/editor/src/main/java/org/apidesign/bck2brwsr/ide/editor/JSNI2JavaScriptBody.java
It is a module for NetBeans that handles one by one (via hint) or batch (via
inspect&refactor) conversion of any GWT library into bck2brwsr ready library.
In theory - I have not used the conversion in a while, I found the whole GWT
too heavy weight and rather design the lightweight libraries from scratch.
> - *Do you plan to support some of the more complex Java features, such
> as Reflection and custom classloaders?*
Reflection is supported - I needed it to execute my TestNG tests in the brwsr.
However I intentionally tried to support as little as possible - when there is
a chance to throw SecurityException I usually throw it.
There is currently one ClassLoader in the system and loads everything. However
I will need to change that, especially because I want to provide support for
modularity. One module (~JAR file) will need one classloader. Not the highest
priority now however.
> I imagine not, since you're
> compiling offline, unless the classloaders load in precompiled classes.
I see what you mean by "offline" now. No, bck2brwsr does not compile offline, it
is a JIT (if I understand correctly what those three letters mean).
> Finally, on the subject of cooperation... if it's feasible, I'd love to
> cooperate with you, but I'm a bit worried about time constraints! I'm sure
> we're both really busy people. In any case, below are areas that I think we
> could potentially cooperate on:
>
> - *Compatibility tests:* Doppio aims for accuracy, so our tests
> sometimes test weird library corner cases (we also test all the weird
> floating point corner cases,
Do you emulate float or treat any float as double? I gave up on floats.
> and we have accurate long support
We had to write it as well.
> that we
> test, too).
Sharing these tests is indeed desirable. Our tests are based on TestNG and use
special @Compare annotation:
http://source.apidesign.org/hg/bck2brwsr/file/56abd247f421/rt/emul/compact/src/test/java/org/apidesign/bck2brwsr/tck/LongArithmeticTest.java
Our "launcher" runs these tests once in HotSpot VM and once in Rhino (possibly
also in a brwsr) and compares the results.
Can you run Doppio in JDK's Rhino? Probably not, I saw some references to
typed arrays and those are not supported in Rhino.
How could I launch Doppio from a HotSpot, tell it to execute some code and get
result? Bck2Brwsr has an HTTP server for that
http://source.apidesign.org/hg/bck2brwsr/file/56abd247f421/launcher/fx/src/main/java/org/apidesign/bck2brwsr/launcher/BaseHTTPLauncher.java
If there was a way to have our launcher for Doppio, we could easy co-operate
on sharing the tests.
> I'm worried that you won't have a need for these, which will
> require restructuring our compatibility tests (our tests intermingle
> corner cases with common cases). There's also a possibility of moving to
> JUnit tests when we get fast enough, since we support JUnit -- which could
> be an issue if bck2browser doesn't.
I don't like JUnit's archaic license
http://wiki.apidesign.org/wiki/CPL
and thus I decided to use TestNG.
> Right now, are tests are simple diffs between the stdout output of
> HotSpot Java and Doppio.
What is your harness? Or do you run all the computations, then print output
and compare visually?
> Granted, you absolutely have the legal right to use our compatibility
> tests in bck2browser. The above are the issues I expect if we want to
> move to a shared test repository.
>
> - *JSON4Brwsr API:* To be honest, I haven't looked at the API in-depth.
> We plan to have a story for JavaScript/Java interop sometime soon
> (hopefully over the next year).
Take a look at twitter demo to see the potential:
http://xelfi.cz/twttr/
I'd be glad to tell people who use JSON4Brwsr with Bck2Brwsr and complain
about threading to replace Bck2Brwsr with Doppio.
I see. As expected the problem is the mangling. This one is quite different to
bck2brwsr one:
http://wiki.apidesign.org/wiki/Bck2BrwsrMangling
> Once this work is done, porting things like the JSON4Brwsr API should be
> feasible (and hopefully easy).
I don't want to port a library to every Java VM in a browser. I'd rather write
it once and let it run on VMs.
> - *I'm currently working on BrowserFS -- a file system in the
> browser.*It emulates the Node JS filesystem API, and has a nice
> backend API for
> implementing arbitrary filesystem backends (e.g. browser localStorage,
> Dropbox, etc). I plan to open source it once I fix the bugs in the first
> backend (hopefully within the week!). It's completely standalone from
> Doppio, and we plan to move Doppio to it once it stabilizes.
>
> Let me know if you're interested in using it with bck2browser for File
> support. :)
Bck2Brwsr doesn't have emulation for java.io.File yet. I'll be more than
delighted to consume anything related to files.
> If you have any questions about Doppio/BrowserFS/etc, let me know -- I'd be
> happy to answer them! And if you have any other collaboration ideas, let me
> know.
I've tried to provide some links to actual sources. They may illustrate the
actual situation more than thousand words. If you want me to study some
resources more deeply don't hesitate to send me links as well.
I guess the initial round of "correcting mis-expectations" is finished. We'll
see what grows from that. Thanks for your reply.
-jt
> (PS: Thanks for NetBeans. :) It's nice to have a sane, stable, and open
> source development environment for Java programs.)
You are welcomed & btw. NetBeans 7.3 has improved JavaScript (but no
CoffeeScript) support.