[racket] meta-languages and going back to the "normal" reader

[Alexander D. Knauth]

If I have a meta-language like this:
#lang my-meta-lang <language>
And my-meta-lang is similar to at-exp in that it can accept any arbitrary language with any arbitrary reader
(as long as it looks at the readtable), then how do I escape back to the reader specified by <language>
from inside a reader macro from my-meta-lang?

What I’m trying to do is something like #lang afl <language> where afl adds rackjure-like anonymous function literals
to <language>.

So to parse this:
#lang afl racket
#λ(+ % 1)
It would use the racket reader but wrap it to use the afl-readtable, which includes dispatch-macros that would
read the (+ % 1) and parse the result into a lambda expression.

But if <language> was something else, with a different reader, then how could I use that to read the (+ %1 1).

For example if it was something like this:
#lang afl at-exp racket
#λ@+[% 1]

There’s also another problem. If it was this:
#lang afl <language>
#f
Or this:
#lang afl <language>
#false
Or some other thing starting with f that means something to <language>,
Then it would see the #f and hope that it would turn out to be #fn. If it doesn’t, then it uses the racket reader
(instead of the one provided by <language>) to read the #f or the #false.

So back to my original question: How do I escape back to the reader specified by <language>
from inside a reader macro?

By the way I can’t find anything in the docs about what the arguments to the read and read-syntax functions
provided by <language>/lang/reader.rkt are supposed to be or mean.

____________________
Racket Users list:
http://lists.racket-lang.org/users

[Matthew Flatt]

The readtable strategy works when <language> itself uses a
readtable-based reader. The idea is that you install a mapping for `#λ`
while leaving all the other mappings in place. If <language> uses a
readtable-based reader, then it picks up your extension, otherwise it
doesn't.

I think a `#lang afl at-exp racket` combination should work fine: `afl`
installs a handler for `#λ`, `at-exp` installs a handler for `@`, and
`racket` uses `read-syntax` to see both extensions.

Adding `#fn` support is a little trickier if you want to fall back to
`#f` or `#false` when the character after `#f` (as determined by a
peek) is not `n`. For that case, the readtable addition for `#f` should
remember the old readtable, and then when it needs to fall back, it
calls `read/recursive` with the saved readtable as the third argument.
That way, immediate parsing of `#f...` uses the saved readtable without
`afl` extensions, while parsing of sub-expressions will return to the
current readtable that includes the `afl` extensions.

Documentation for the functions from a "<language>/lang/reader.rkt" is
in section 1.3.18 of the Reference, which defines `#lang` (as being
"like `#reader`, which is described in the same section).

[Alexander D. Knauth]

On Jul 10, 2014, at 6:40 AM, Matthew Flatt <mfl...@cs.utah.edu> wrote:

The readtable strategy works when <language> itself uses a
readtable-based reader. The idea is that you install a mapping for `#λ`

while leaving all the other mappings in place. If <language> uses a
readtable-based reader, then it picks up your extension, otherwise it
doesn't.

I think a `#lang afl at-exp racket` combination should work fine: `afl`
installs a handler for `#λ`, `at-exp` installs a handler for `@`, and
`racket` uses `read-syntax` to see both extensions.

Well for some reason it doesn’t:

#lang afl at-exp racket/base

(map #λ(+ % 1) '(1 2 3)) ; read: bad syntax `#λ’

But also for some reason this does:

#lang at-exp afl racket/base

(map #λ(+ % 1) '(1 2 3)) ; '(2 3 4)

(map #λ@+[% 1] ‘(1 2 3)) ; ‘(2 3 4)

By the way I only just got this to work yesterday by doing basically this but for afl:

https://github.com/AlexKnauth/rackjure/commit/5fa266e672d529dde227ef216aaef157fa5c618c

Also is there any way to get something like this to work?:

#lang afl at-exp racket/base

@#λ(+ % 1)[1] ; read: bad syntax `#λ'

Adding `#fn` support is a little trickier if you want to fall back to
`#f` or `#false` when the character after `#f` (as determined by a
peek) is not `n`. For that case, the readtable addition for `#f` should
remember the old readtable, and then when it needs to fall back, it
calls `read/recursive` with the saved readtable as the third argument.
That way, immediate parsing of `#f...` uses the saved readtable without
`afl` extensions, while parsing of sub-expressions will return to the
current readtable that includes the `afl` extensions.

Do you mean like this?:

(define lambda-readtable (current-readtable))

(parameterize ([current-readtable orig-readtable])

  (read-syntax/recursive src in #f lambda-readtable))

Documentation for the functions from a "<language>/lang/reader.rkt" is
in section 1.3.18 of the Reference, which defines `#lang` (as being
"like `#reader`, which is described in the same section).

Ok I just found this in section 1.3.18:

The arity of the resulting procedure determines whether it accepts extra source-location information: a read procedure accepts either one argument (an input port) or five, and aread-syntax procedure accepts either two arguments (a name value and an input port) or six. In either case, the four optional arguments are the reader’s module path (as a syntax object in read-syntax mode) followed by the line (positive exact integer or #f), column (non-negative exact integer or #f), and position (positive exact integer or #f) of the start of the #reader form. 

But maybe there should be a link or something to section 1.3.18 from sections 17.2 and 17.3.1 of the Guide.  

That would make it a lot easier to find it.  

[Matthew Flatt]

I think the problem may be in `at-exp`.

If you change

pkgs/racket-pkgs/at-exp-lib/at-exp/lang/reader.rkt

and replace the use of `at-readtable` with `(make-at-readtable)`, does
that fix the problem?

[Alexander D. Knauth]

Ok now it does this:

#lang afl at-exp racket/base
(map #λ@+[% 1] '(1 2 3)) ; @+: unbound identifier in module in: @+

#lang at-exp afl racket/base ; different order

(map #λ@+[% 1] '(1 2 3)) ; ‘(2 3 4)

#lang afl at-exp racket/base ; original order
@#λ(+ % 1)[1] ; 2

#lang at-exp afl racket/base ; different order
@#λ(+ % 1)[1] ; read: bad syntax `#λ’

Also for the first error it has +[ highlighted instead of @+

Is there any way to get around this?

[Alexander D. Knauth]

On Jul 10, 2014, at 12:56 PM, Alexander D. Knauth <alex...@knauth.org> wrote:

> Ok now it does this:
>
> #lang afl at-exp racket/base
> (map #λ@+[% 1] '(1 2 3)) ; @+: unbound identifier in module in: @+

After doing the read-syntax/recursive thing now this first error goes away (I have no Idea why),
but the other ones are still the same.

> #lang at-exp afl racket/base ; different order
> (map #λ@+[% 1] '(1 2 3)) ; ‘(2 3 4)
>
> #lang afl at-exp racket/base ; original order
> @#λ(+ % 1)[1] ; 2
>
> #lang at-exp afl racket/base ; different order
> @#λ(+ % 1)[1] ; read: bad syntax `#λ’

This error is still there though.
Does at-exp use read-syntax/recursive? Would that fix this?
I don’t see why it would, but then I don’t see why it would fix the other error either.

[Matthew Flatt]

Moving the call to `make-at-readtable` made `#lang at-exp afl` work,
because it delays enough that `at-exp` picks up the additions to the
readtable from `afl`.

The problem with `#lang afl at-exp` is that you need even more of a
delay: the `at-exp` reader extension currently settles on the same
readtable for recursive reading as the readtable it extends, but you
want the recursive-reading readtable to be the current one at the time
that recursive reading happens.

I've changed `at-exp` so that it delays readtable decisions to make
things work the way you expect.