embedding a DSL into racket code

[Raoul Schorer]

Hi,

I have written a #lang, which works ok when it is in its own file. Now, I'd like to make it embedded through a macro such as '(my-lang ...)'

What I would like is essentially expand that:

#lang racket
... standard racket source ...
(my-lang ...)
... standard racket source ...

to that:

#lang racket
... standard racket source ...
... racket source expanded from my-lang ...
... standard racket source ...

But there's a problem, because although the 'my-lang' macro will expand my-lang code to racket code, it must expand to a valid expression, i.e:

(define-syntax (my-lang stx)
  #`(my-lang-expand (open-input-string (cadr (syntax->datum stx)))))

could yield '(begin (define a 3) a)' while I would want '(define a 3) a' so that the 'a' from 'my-lang' becomes a valid top-level identifier in the whole (racket) program.

I understand that this impllies that the '(my-lang ...)' macro would then modify its parent expression (the main program), so I guess that means breaking the normal expansion process.

Is that possible? Is there a better way?

Thanks!

Raoul

[Raoul Schorer]

So, I was partly mistaken about what the problem was. My embedding macro seems to work, but the bindings are incorrect.

#lang typed/racket
(require my-lang-compiler/expander
         math/array)
(define a (array #(3)))
(my-lang "a=:4")

should raise: 'module: identifier already defined in: a'

But it instead expands to:

...
(define-values:73 (a)
     (#%app:74
      unsafe-list->array:75
      (#%app:76 vector:77 (quote 1))
      (#%app:78 list:79 (quote 3))))
   (define-values:80 (a)
     (#%app:81
      unsafe-list->array:75
      (#%app:82 vector:83 (quote 1))
      (#%app:84 list:85 (quote:86 4))))
...

So, I guess that's because my expansion occurs after the binding stage. Is there some way to make this work so that 'a' from 'my-lang' is recognized as a top-level identifier?

Thanks again!

Raoul

Full expansion (just for reference):

(module anonymous-module typed/racket
  (#%module-begin
   (module configure-runtime '#%kernel
     (#%module-begin (#%require racket/runtime-config) (#%app configure (quote #f))))
   (begin-for-syntax
    (module*
     #%type-decl
     #f
     (#%plain-module-begin
      (#%declare #:empty-namespace)
      (#%require typed-racket/types/numeric-tower)
      (#%require typed-racket/env/type-name-env)
      (#%require typed-racket/env/global-env)
      (#%require typed-racket/env/type-alias-env)
      (#%require typed-racket/types/struct-table)
      (#%require typed-racket/types/abbrev)
      (#%require
       (just-meta 0 (rename racket/private/sort raw-sort sort))
       (just-meta 0 (rename racket/private/sort vector-sort vector-sort))
       (just-meta 0 (rename racket/private/sort vector-sort! vector-sort!))
       (only racket/private/sort))
      (#%app
       register-type
       (t-quote-syntax a)
       (#%app
        make-App
        (#%app make-Name (quote-syntax Array) (quote 1) (quote #t))
        (#%app:18 list -PosByte)))
      (#%app:19
       register-type
       (t-quote-syntax:20 a)
       (#%app:21
        make-App
        (#%app:22 make-Name (quote-syntax Array) (quote 1) (quote #t))
        (#%app:23 list -PosByte))))))
   (begin-for-syntax
    (#%app:24 add-mod! (#%app:25 variable-reference->module-path-index (#%variable-reference))))
   (define-values:26 (blame1:27)
     (#%app:28
      module-name-fixup:29
      (#%app:30 variable-reference->module-source/submod:31 (#%variable-reference:31))
      (#%app:32 list:29)))
   (begin-for-syntax
    (#%require:33 typed-racket/utils/redirect-contract)
    (module #%contract-defs-reference racket/base
      (#%module-begin:34
       (module:35 configure-runtime:35 '#%kernel:35
         (#%module-begin:35
          (#%require:35 racket/runtime-config:35)
          (#%app:35 configure:35 (quote #f))))
       (#%require:36 racket/runtime-path)
       (#%require:37 (for-meta:37 1 racket/base))
       (define-values:38 (contract-defs-submod)
         (let-values:38 (((contract-defs-submod)
                          (let-values:39 (((runtime?:40) (quote #t)))
                            (#%app:41
                             list:41
                             (quote:41 module:40)
                             '(submod ".." #%contract-defs)
                             (#%variable-reference:40)))))
           (let-values:42 (((get-dir:38) void:38))
             (#%app:43
              apply:44
              values:40
              (#%app:45
               resolve-paths:38
               (#%variable-reference:38)
               get-dir:38
               (#%app:46 list:40 contract-defs-submod))))))
       (begin-for-syntax:38
        (#%app:47
         register-ext-files:38
         (#%variable-reference:38)
         (let-values:38 (((contract-defs-submod)
                          (let-values:48 (((runtime?:40) (quote #f)))
                            (#%app:49
                             list:49
                             (quote:49 module:40)
                             '(submod ".." #%contract-defs)
                             (#%variable-reference:40)))))
           (#%app:50 list:40 contract-defs-submod))))
       (#%provide:51 contract-defs-submod)))
    (#%require:52 (submod "." #%contract-defs-reference))
    (define-values:53 (make-redirect2:54)
      (#%app:55 make-make-redirect-to-contract contract-defs-submod)))
   (module*
    #%contract-defs
    #f
    (#%plain-module-begin
     (#%declare #:empty-namespace)
     (#%require:56 (submod typed-racket/private/type-contract predicates))
     (#%require:57 typed-racket/utils/utils)
     (#%require:58 (for-meta:58 1 typed-racket/utils/utils))
     (#%require:59 typed-racket/utils/any-wrap)
     (#%require:60 typed-racket/utils/struct-type-c)
     (#%require:61 typed-racket/utils/prefab-c)
     (#%require:62 typed-racket/utils/opaque-object)
     (#%require:63 typed-racket/utils/evt-contract)
     (#%require:64 typed-racket/utils/hash-contract)
     (#%require:65 typed-racket/utils/vector-contract)
     (#%require:66 typed-racket/utils/sealing-contract)
     (#%require:67 typed-racket/utils/promise-not-name-contract)
     (#%require:68 typed-racket/utils/simple-result-arrow)
     (#%require:69 racket/sequence)
     (#%require:70 racket/contract/parametric)))
   (#%require:71 jacket-compiler/expander)
   (#%require:72 math/array)
   (define-values:73 (a)
     (#%app:74
      unsafe-list->array:75
      (#%app:76 vector:77 (quote 1))
      (#%app:78 list:79 (quote 3))))
   (define-values:80 (a)
     (#%app:81
      unsafe-list->array:75
      (#%app:82 vector:83 (quote 1))
      (#%app:84 list:85 (quote:86 4))))
   (#%provide)
   (#%app:87 void)))

[Michael Ballantyne]

Yeah, the `begin` shouldn't be a problem---`define`s splice out of a begin. And it doesn't look like you're missing the binding pass, because you are successfully expanding to module-level `define-values` forms.

I suspect that your problem relates to the set of scopes that end up on the binding identifier. How do you create the identifier `a` in `my-lang-expand`? I think it would make the most sense to copy the scopes from the string literal argument to `my-lang` onto the bindings that correspond to names parsed from the string, using `datum->syntax`.

As an aside, you might be interested in the udelim package for creating a reader with custom string literal syntax, in case the language you're embedding also has string literals: https://docs.racket-lang.org/udelim/index.html

[Raoul Schorer]

I suspect that your problem relates to the set of scopes that end up on the binding identifier.

I came to the same conclusion, however I do not understand how I would use `datum->syntax` to copy bindings even after reading on syntax objects, taints, etc. Do you mean using the `ctxt` parameter? Would you be kind enough to explain further, please?

The identifier `a` in `my-lang-expand` is trivially created by:

(define-syntax (=:/j stx)
  (syntax-parse stx
    [(_ n e)
     #`(begin (define n e))]))

In the docs, I however did read about `syntax-binding-set` and `syntax-binding-set-extend`, that apparently allow manually setting the bindings of a symbol. Would that be a potential solution?

[Michael Ballantyne]

Wherever you parse the string and construct a syntax object, you would use the original syntax object that contained the string as the ctxt argument to datum->syntax for the new syntax object.

For example, in your jacket-compiler project you’d change this definition:

(define-syntax (j s)
  #`(interpret-syntax-fragment/j () () #,(lex/j (open-input-string (cadr (syntax->datum s)))) #,(hasheq)))

to

(define-syntax (j s)
  #`(interpret-syntax-fragment/j () () #,(datum->syntax s (lex/j (open-input-string (cadr (syntax->datum s)))) #,(hasheq))))

--
You received this message because you are subscribed to a topic in the Google Groups "Racket Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/racket-users/IukMTySl4XQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/9f531b85-c0e9-4709-bad4-0d92a8ffd05b%40googlegroups.com.

[Raoul Schorer]

That actually did the trick!

Thank you very much for your time :-)

To unsubscribe from this group and all its topics, send an email to racket...@googlegroups.com.