".zo"s and syntax-object srclocs

[Matthew Flatt]

As an experiment, I've changed the ".zo" writer and reader to preserve
source locations and 'paren-shape properties for syntax objects.

Saving source locations can be helpful in a couple of situations:

* For a macro implementor, when a macro is implemented by a module
that is compiled to ".zo" form and something goes wrong with a use
of the macro in another module, the expansion error can more likely
point to parts of the macro implementation.

* Unusual macros that depend on source-location information, such as
Scribble's `racketblock`, tend not to compose with other macros.
While a macro like `racketblock` can expand in a way that preserves
needed source locations, macros that expand to `racketblock` tend to
go wrong when compiled to ".zo" form. Similarly, a macro might treat
`[...]` or `{...}` specially, based on the 'paren-shape property,
and then other macros that expand to that one don't work right
unless the property is preserved.

There are also some drawbacks to keeping source locations:

* It costs space. Happily, this cost is lower than I thought it would
be: ".zo" files are only about 10% bigger on average (thanks to a
big pile of marshaling tricks that we've built up over the years).
The effect on run-time memory seems to be even smaller (thanks to a
big pile of on-demand-loading machinery).

* Source locations associated with a macro's implementation can be a
distraction for users other than the macro's implementor.
Furthermore, source locations can bleed through in cases other than
syntax errors.

For example, the change makes the `racket` REPL behave like this:

> (lambda (x) x)
#<procedure>
> (eval '(lambda (x) x))
#<procedure:...t/private/kw.rkt:394:14>

In the first case, the source location for the `lambda` expression
doesn't refer to a file, so it isn't included in the output. In the
second case, the S-expression for the `lambda` expression is
converted to a syntax object that has *no* source location, and so a
source location is eventualy inherited from the implementation of
the `lambda` macro.

On the whole, I think saving source locations is going to be better
than not, but we'll have to try it out.

I didn't yet attempt to change `quote-syntax/keep-srcloc` or other
things that expand in special ways to preserve source locations. If the
experiment works out, that's a next step.

[Robby Findler]

This sounds great to me! This has been a real source of confusion for
many over the years.

Robby

> --
> You received this message because you are subscribed to the Google Groups "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/20150903003024.514F86501CE%40mail-svr1.cs.utah.edu.
> For more options, visit https://groups.google.com/d/optout.

[Vincent St-Amour]

add1

Vincent

> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/CAL3TdONb7J7P50r1EQwOKnzXdKhkoWpLfC4r%3DqtMW15DBQ%3D46w%40mail.gmail.com.