I am a Racket beginner trying to create my own DSL.
As a long-time user of Xtext and other similar tools in the Eclipse ecosystem, I have come to Racket expecting that it would address similar concerns.
At the moment, I have mixed feelings: I find the metaprogramming facilities in Racket very effective, but at the same time I am struggling to achieve tasks that were supported natively by Xtext.
For those who don't know Xtext, here is a summary of how it works:
- A language project is based on a grammar with attribute annotations.
- The grammar is converted into a "metamodel", i.e. a set of classes where each grammar rule corresponds to a class.
- A parser is automatically generated. It can convert some source text into a "model", i.e. a set of instances of the classes from the metamodel.
- A model can be manipulated using Java APIs. Specialized languages are available to constrain a model, query it, transform it, or generate code using templates.
In Racket, I have started my language project by reproducing what I would have done in Xtext:
- I have created a grammar with bragg
- I have written a set of syntax classes that play the role of the metamodel
- Syntax objects play the role of the model, and I can get their attributes with syntax-parse
- I have written several macros that can generate Racket code in the simplest cases.
However, I miss some facilities that Xtext provides out-of-the-box:
- Racket syntax classes do not directly support inheritance.
- Syntax objects are not tied to syntax classes in a class-instance relationship, and I have to use syntax-parse every time I want to read an attribute.
- Xtext automatically creates child->parent references in the generated AST. In Racket, it seems that I cannot get the parent of a syntax object.
- Xtext provides a default mechanism for resolving named references, and a scoping API for languages that need specific scoping rules. The AST generated by Xtext is actually an object graph rather than a tree.
My main concern is about managing the scopes/lexical contexts in my language. I am still browsing the documentation but I have found no library or guide that addresses this issue.
The language examples that I have found are either too simple
(their scoping rules can be easily mapped to those of Racket through
macros), or use ad-hoc techniques, so that it is difficult to infer a general
methodology.
So far, I have made two attempts to work around these issues: (1) by creating a metamodel-like data structure using Racket structs, and transforming syntax objects into struct instances; or (2) using syntax objects only and attaching context data to each of them as a syntax property.
Both have strengths and weaknesses, and I am still feeling that I am not using Racket with the right mindset.
I hope I have made my concerns clear. Maybe I can create a small example to further illustrate what I want to do and where I am stuck.
Have you experienced similar concerns in one of your projects?
What design patterns would you recommend ?
Do you know any well-commented real-life example that I could use for inspiration?
Thanks in advance for your answers.
Guillaume Savaton
N.B: I have also published a similar question at stackoverflow two weeks ago, but it still has no answer: