(opened May 10, 2013, over 9 months ago).
What you need to do
Blink developers (non-bindings):
- Nothing. The new compiler should just work, run faster, have fewer bugs, and be improved faster.
- If you want new IDL features, it's much easier now!
- Please have a quick look at the design doc (IDL compiler), particularly Back end (code generator), and the Jinja page for guidance on writing and editing the templates.
- Please add me (nbarth@) as a reviewer for CLs that edit the new compiler (bindings/scripts), until we're all used to hacking on it: it's rather differently structured from the old one.
The Tokyo bindings team rewrote the Perl IDL compiler in Python. This was a ground-up rewrite, and included significant cleanups of a hoary code base; it significantly reduces the technical debt in the bindings and should make future velocity much higher.
I (Nils) wrote the code, and haraken tirelessly reviewed it and suggested revisions.
Koji (ex) did initial work on the code generator (and chose the Jinja template processor), while Kouhei's rewrite of the SVG bindings (308818) significantly simplified the rewrite, meaning SVG handling was 1 day of work and a few dozen lines of code, not a month of work and thousands of line.
The rewrite was developed incrementally on-trunk via test-driven development, and switched when the two compilers had identical output (on tests and all actual code). There was virtually no impact on bindings developers, as I synced all Perl changes to Python, with a few developers hacking on the Python themselves even before the switch.
To improve hackability (ease of modification and understanding of code), primarily.
The Perl compiler was unmaintainable, due to organic growth over the years (since 2005, starting with KDE, then in WebKit). Also, we now use Python, not Perl, and this was the last significant piece of Perl in Blink (347866). Thus it needed to be rewritten in Python, and we took the opportunity to re-engineer it. This was easier once V8 was the only back end, hence why it was done when it was.
The new code is significantly more hackable, particularly in the code generator (the main component and complexity). This will allow faster and more correct development.
This is due to:
For example, the Perl code generator called the parser to read IDL files for any interfaces it encountered (e.g., to check the C++ class name)! This was replaced by computing the public information of all interfaces in a preliminary step, and (in rare cases) reading in any interfaces whose private information is needed in the front end.
- factoring the code generator into separate modules and templates (e.g., v8_attributes.py/attributes.cpp), rather than one monolithic module;
- factoring the compiler into a pipeline (lex/parse/construct/CG logic/CG template);
- the use of templates, rather than building the output by concatenating strings;
- moving complexity out of the code generator and into a global context step and the front end.
The new code has a design, and a design document: IDL compiler.
The new code is faster (roughly 25-30% faster in computation time, twice as fast in real time in ninja), and can be optimized further, thanks to its modularity. I expect to speed it up further in the next week (by optimized caching of the lexer-parser).
- IDL feature work (345506: Implement Web IDL spec): everyone can pitch in!
- Remove Perl from build (347863): one remaining script (make-css-file-arrays.pl): I'll do this unless anyone else wants to grab it
- Replace embedding resources by ad hoc Python with GRD resources (312586)
Continue cleanup: (341748, 345137, 345503, 345519): long-term; I'll primarily do this until it's clean enough
- Speed up build (part of 341748):
- caching optimized lexer and parser should improve speed
- precise dependency resolution in GYP to prevent unnecessary full rebuilds