BYU - Spring 2013 - CS 630 - 6/13 - Review
Matthew Ashcraft
Honu: Syntactic Extension for Algebraic Notation through Enforestation
Extensible programming languages are those in which any user can implement their own language extensions without having to get it approved by someone else first. As we discussed in the last paper, Racket is an ideal language for this because it allows language extensions to be made by simply including additional libraries. Extensible languages such as Racket allow for the acceleration of programming language design so that eventually any programming form we desire can be achieved. One of the features lacking from Racket up to this point has been the ability to parse and expand code that is not constrained by parentheses. That is where this work comes in. Honu is an extension of Racket that allows for code to be written in forms similar to that of C and Java, using curly braces. They say that Honu is an example of the middle ground that can be achieved between syntactic minimalism such as Lisp and Racket, and maximal grammatical freedom such as C and Java. Honu does this by adding a precedence-based parsing set to a parser which supports infix operators and syntax that is unconstrained by parentheses. The rest of this paper refers to infix operators, but it never defines what it is. Upon doing further research, I found that infix operators are those which perform operation on sets of values, such as + and -. Is this a valid explanation, or is there more to this that I didn't find online? The paper goes on to say that it turns a flat sequence of terms into a Lisp-like syntax tree. They call this enforestation. They have integrated this step into the macro-expansion machinery so that it can support local bindings to maintain hygienic conditions, macro-generating macros, and local macro bindings, all of which have proven essential for building expressive and composable languages.
To aid in the basic overview of Honu, they offer a simple and straightforward example to demonstrate its power. Using the example they show that is no return operator in Honu, but rather the result of last evaluated expression is the return value. Additionally it shows that semicolons are not needed in Honu, but can be used in ambiguous sequences that need to be forced to act one way or another. They also use that to explain the use of blocks. Blocks are created using curly braces, and can mix declarations and expressions, just like in C. After discussing the example they move onto the use of the macros they have introduced with Honu. They bind a name to a pattern-based macro, which consists of literals, patterns, and a body. They give an example a macro they have produced which removes boilerplate (What is a boilerplate?). This example essentially turns a block of code within a function into a macro within which is a block with a syntactic replacement of the original code. This syntactic replacement is a template wrapped in a syntax form. When this is evaluated it returns a syntax object. The expressions they use in Honu allow them to use macros together. Additionally, they allow the pattern section of their macro to use ellipsis to match repetitions of the preceding variable, literal, or group of terms grouped by $.
Another feature provided by the Honu language extension is that it still allows programmers to introduce new syntax classes. Their pattern form are like macros except they don't include an expansion body. The paper then goes on to talk about the operators available in Honu. They actually allow programmers to define new binary and unary operators. Their binary operators are always those that work on multiple variables, where as the unary operators only work on one at a time. Additionally they allow operators to have the properties of both, meaning they can work on one or more variables at a time. The unary operator happens to be very similar to a macro with a single expression subform, the only difference being that the former has a precedence level. This precedence level affects the way in which expressions using the operator are parsed. This is because the unary operator makes a recursive call to parse, which macros naturally don't do. The last point they make in explaining this Honu in this section is that Honu has no reserved words, which means that it doesn't distinguish between operator and identifier names.
The syntax of Honu is defined in a similar way to Scheme, except it has more structure than Scheme. The paper goes over eight of the different possible expressions available through the Honu syntax, of which can be read about there. They then reiterate the point that Honu does reserve any keywords, stating that parentheses, curly braces, square brackets, and semicolons may be used differently in new syntactic forms. This Scheme grammar is dependent upon parsing by a reader to form terms. This parser differentiates between numbers, identifiers, strings, commas, parentheses, comments, etc. The enforestation phase introduced by Honu is then explained. The enforestation and expansion of macros is interleaved. This allows the enforestation phase to be sensitive to bindings. The enforest function keeps track of infix operator parsing and precedence, and drives the enforestation process. The paper then explains what goes into the enforest function, along with a long example which we will discuss in class.
Programmers who write macros in the Honu language extension write pattern-based macros. These macros generate low-level macros that return new sequences of terms. The patterns within this function are then compiled, and broken up into terms that are consumed by the macro. The parsing of the Honu functions is then implemented using the enforest function. The parser applies the function to the top-level sequence of terms, and registers the bindings it finds along the way. The parser resolves the scope of each section of the code before it dives deeper into the macros. This means that until the scope surrounding a block is determined, that block will not be parsed. Also there are extra features that allow for macros to be defined after the block in which it is used. The paper then goes through some parsing examples, and ends off talking about related work. Unlike the last paper we read, this one actually has a conclusion. They make the final statement that “Macros in Honu ... offer the promise of a smooth path from building simple syntactic abstractions to whole new languages.”
With a few exceptions, this paper was actually quite clear and understandable. I felt like I understood the additional features offered by the Honu language extension and why it is a valuable addition to Racket. I had trouble understanding a few of the examples throughout the paper though. The example explaining how enforest worked was a little unclear. I was able to follow through half of the example, but after that I got lost. The questions I have were mentioned previously in this write-up.