First, to summarize Emscripten's test setup, it does the following: C/C
++ source code files are compiled into LLVM bitcode using Clang and
llvm-gcc, then run through Emscripten, then run in a JavaScript
engine, and the output compared to what the C/C++ code gives when
compiled and run using gcc. This is done separately for both normal
and optimizing mode, and running is checked in both V8 and
SpiderMonkey. So, this is high level (functional) testing,
essentially.
The test cases go from very high level stuff like a complete malloc
implementation, a script engine for another language, and a raytracer,
down to benchmarks like fasta and fannkuch, and finally down into
small 'unit tests' that target specific issues (polymorphism, taking
the address of a value on the stack, sizeof, etc., and various checks
to prevent regressions).
Regarding interacting with the document or window, I tend to think
it's better to focus first on getting self-contained code to work - on
building C/C++ code and seeing that it outputs the right stuff. Since
the Emscripten test setup already exists and does exactly that, we
could integrate it with llvm-js-backend and it would get a lot of test
coverage very quickly. Also I am very curious to compare the generated
code llvm-js-backend produces on those tests with what Emscripten
generates.
I do agree that tests for interacting with the document/window/etc.
are important. I think it's logical though to start with self-
contained code, then once that is solid to move into interaction with
external stuff. I was planning to add such tests to the Emscripten
test runner later on, but we can do that sooner of course.
- azakai
On Oct 11, 5:30 pm, David LaPalomento <dlapalome...@gmail.com> wrote:
> At this point, llvm-js-backend (ljb) has a reasonable number of
> unit-test style test cases that verify certain aspects of the code
> generation but require manual testing of semantics. This is a good
> starting point but far from where I'd like to be. The next testing
> priority I had identified was to start compiling a suite of bitcode
> test cases which interact directly with javascript objects like the
> document and window to actually produce some output. Ideally, this
> suite would be semantically verified using techniques similar to
> emscripten's test runner with the addition of something along the
> lines of env.js. If we can agree on a common interface to the native
> js objects from the IR, I imagine this test suite can be completely
> shared between the two projects. This would alleviate the semantic
> testing gap ljb has today while taking a step towards where I'd like
> to go: using high-level languages hosted on LLVM to build web UIs.
> Alon, have you started exposing javascript objects up into the IR yet?
> Does this sort of testing sound interesting to you?
> David
> On Thu, Sep 23, 2010 at 9:15 PM, Alon Zakai <aza...@mozilla.com> wrote:
> > So, it looks like there are two open source projects that have separately been working on translating LLVM bitcode to JavaScript, (in alphabetical order) emscripten and llvm-js-backend. We got to talking on email, and then thought to move the discussion to the mailing lists of both projects, since other people might find it interesting.
> > Our approaches are somewhat different, llvm-js-backend is, as the name implies, a JavaScript backend for LLVM. It's written in C++ and integrated in the LLVM codebase. Emscripten on the other hand parses human-readable LLVM bitcode, and is written in JavaScript. So sharing the core code isn't directly possible. However, we would really like to collaborate as much as possible otherwise, and it seems there is a lot of room for that:
> > 1. Test framework and cases. While the approaches and architectures are different, we can use the same tests for both, since they are trying to do the same thing. One project (emscripten) has automatic tests that compile C/C++ code into LLVM bitcode that is then tested (through a Python test runner), while the other (llvm-js-backend) has test cases written in LLVM bitcode directly (and run using llvm-lit). I think there is a need for both kinds of tests. We could collaborate on the scripts that run the tests, and on test case code (of which we will need a lot!).
> > 2. Standard library. Both projects need stdlib, math, stdio, etc. libraries. One approach is to write those in JavaScript (Emscripten is trying that, for now), another would be to compile a stdlib written in C/C++, after some porting. We can debate which approach is better, and work together on a single standard library that both projects can use.
> > 3. Discuss the various issues we need to tackle in order to run LLVM bitcode in JavaScript, and perhaps implement shared solutions. For example, how to implement pointers, loops, etc. etc. It might turn out that we can agree on a certain implementation of pointers, and share the code for it, etc.
> > We should probably fork into separate threads for each of these topics.
> > - azakai
