line number mapping strategy

[Will Conant]

Bruno,

I've been thinking about mapping line numbers from generated source to
original source in streamline, and I've thought about attempting a few
changes in the code to see if I could get it working. I think I have a
solution figured out, but before I really get going, I thought I'd see
if you had given it much thought. Do you have a strategy in mind
already? How important do you think such a feature is?

Thanks!

--
Will Conant

[Bruno Jouhier]

This would be nice, especially if debuggers or even just exception
stack traces could understand it and display the right info.

I wanted to investigate this at some point but I had no time and I
don't really know what needs to be done to get the best results. If it
increases code size, it would be good if there was an option to turn
the mapping on/off.

So please go ahead. The best way is probably that you fork the repo
and send me a pull request.

Bruno

[Will Conant]

I have added an option to my fork of streamlinejs for preserving line
numbers in the translated code. You can check it out here:

https://github.com/willconant/streamlinejs

Right now, you'll only get this behavior if you call
streamline.translate(source, {preserveLineNumbers: true}), or if you
require a file that has the !!STREAMLINE!! tag in it. In all other
cases, I stick to the pretty-printed output.

The primary strength of this approach is that the top of your stack
traces will contain the correct source line number without any extra
processing. The biggest weakness that I've seen so far is that the
rest of the stack trace tends to have a lot of useless/confusing
entries where control is resumed by constructs that only exist in the
translated code. For example, every "branch neutralizer" adds an entry
to the stack trace with a line number that roughly corresponds to the
location in the source where the branch ends. That being said, I think
this is a really good start on making streamlined code easier to
debug.

Here's how I got it to work:

1) I mark the original tree of narcissus nodes with the property
_isSourceNode before the tree is modified.
2) After the tree is modified, if preserveLineNumbers is true, I use a
modified version of Narcissus.decompiler.pp that generates a map of
output line numbers to input nodes along with the pretty-printed
source.
3) I rearrange the lines of the pretty-printed source to correspond
with the line numbers found on the nodes in the map, being careful to
only use nodes that were marked with _isSourceNode.

I could reduce this to two steps, simply having
Narcissus.decompiler.pp only add newlines such that the output line
numbers would match the input line numbers, but I think there may be a
useful hybrid approach where the transformed code is pretty-printed,
but the line number map is made available through the API for
debugging purposes. Another possibility is simply adding comments to
the ends of lines in the output source which indicate the
corresponding lines in the input source.

I haven't tested this extensively, so it very well may be horribly
broken, but I thought it was worth sharing early so I could get some
feedback before spending any more time.

Cheers!

--
Will Conant

[Bruno Jouhier]

New cool feature: I integrated Will's line number magic and published
a new version on npm.

You can now control the way lines are formatted in the output. See
https://github.com/Sage/streamlinejs/wiki/Compilers for details.

Bruno