I'm looking for a Lisp library to create text output a la e.g. ASP, JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in text files, which are then evaluated to create text output.) So far I've found:
I've also come across ALP [1] which is a research paper but no code. CL-EMB [2] supports the "server pages" style, but requires passing an "environment" to the evaluation function. This isn't ideal.
Have I missed any? Does anyone have experience with the above libraries?
On Fri, 2009-01-09 at 03:06 -0800, WalterGR wrote: > I'm looking for a Lisp library to create text output a la e.g. ASP, > JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in > text files, which are then evaluated to create text output.) So far > I've found:
> I've also come across ALP [1] which is a research paper but no code. > CL-EMB [2] supports the "server pages" style, but requires passing an > "environment" to the evaluation function. This isn't ideal.
> Have I missed any? Does anyone have experience with the above > libraries?
I don't know if you care for opinions a bit to the side of what you're asking about here, but I think templating; something like HTML-TEMPLATE, is a better idea in general: http://weitz.de/html-template/
> On Fri, 2009-01-09 at 03:06 -0800, WalterGR wrote: > > I'm looking for a Lisp library to create text output a la e.g. ASP, > > JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in > > text files, which are then evaluated to create text output.) So far > > I've found:
> > I've also come across ALP [1] which is a research paper but no code. > > CL-EMB [2] supports the "server pages" style, but requires passing an > > "environment" to the evaluation function. This isn't ideal.
> > Have I missed any? Does anyone have experience with the above > > libraries?
> I don't know if you care for opinions a bit to the side of what you're > asking about here...
I'm open to _any_ opinions, so long as they are offered as politely as yours. (Thanks, btw.)
hhmmmm... I was about to respond to your message in full but the sleeping pill is sinking in. If I continue writing, tomorrow I'll be able to undertand anything I wrote, but readers will likely be confused by the many non-sequiturs and references to that dratted! lemur...
I've been very happy with this combination with CL-SQL under SBCL/Linux. However, this is probably not the most popular combination as it seems like a lot of people adopt to Hunchentoot these days.
Petter -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
> On Fri, 2009-01-09 at 03:06 -0800, WalterGR wrote: > > I'm looking for a Lisp library to create text output a la e.g. ASP, > > JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in > > text files, which are then evaluated to create text output.) So far > > I've found:
> > I've also come across ALP [1] which is a research paper but no code. > > CL-EMB [2] supports the "server pages" style, but requires passing an > > "environment" to the evaluation function. This isn't ideal.
> > Have I missed any? Does anyone have experience with the above > > libraries?
> I don't know if you care for opinions a bit to the side of what you're > asking about here, but I think templating; something like HTML-TEMPLATE, > is a better idea in general: > http://weitz.de/html-template/
> I guess; "embedding data in Lisp is nicer than embedding Lisp in > data" .. IMHO - when possible.
Actually, please permit me to simply say - I understand the benefits of each approach. I find the high "turtle-factor" [1] of approaches like CL-WHO especially appealing. The Lisp-in-data would be only one of the tools in my toolbox - albeit a necessary one.
WalterGR wrote: > I'm looking for a Lisp library to create text output a la e.g. ASP, > JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in > text files, which are then evaluated to create text output.) So far > I've found:
...
Whatever you choose, Terence Parr makes a good argument for functional template design. In a nutshell, the output is a simple cross product of the template format and the specific data. Mixing fancier code in with the templates tends to be bad.
I recommend reading a couple of his papers before getting started.
+--------------- | Lars Rune Nøstdal <larsnost...@gmail.com> wrote: | > I don't know if you care for opinions a bit to the side of what you're | > asking about here, but I think templating; something like HTML-TEMPLATE, | > is a better idea in general: | > http://weitz.de/html-template/ | > | > ..and I think something like CL-WHO is an even better idea yet: | > http://weitz.de/cl-who/ | > | > I guess; "embedding data in Lisp is nicer than embedding Lisp in | > data" .. IMHO - when possible. | | Actually, please permit me to simply say - I understand the benefits | of each approach. I find the high "turtle-factor" [1] of approaches | like CL-WHO especially appealing. [1] "...it's Lisp all the way down!" +---------------
Yes, CL-WHO & HTOUT & HTMLGEN (and others shown on this page <http://www.cliki.net/Lisp%20Markup%20Languages>) are *extremely* nice to use, IMHO, mainly because you can nest HTML templates inside Lisp code inside HTML templates inside Lisp code... as far as you like.
What they lack (of course) is an infrastructure that makes them easy to use as more-or-less standalone web page source files. But it's actually very easy to wrap a trivial "defsystem"-like thing around a LOAD, so that if you haven't LOAD-ed a page before then do so and "auto-register" it, and if there is a compiled version then you check to see if the source is newer and if so recompile/reload that, etc. That's what I did back circa 2002 [as part of *my* first major production use of CL], and came up with my LHP ("Lisp-Handled Pages") format, of which you can see samples here:
[LHP currently uses HTOUT, though it could easily be converted to CL-WHO or others.]
+--------------- | The Lisp-in-data would be only one of the tools in my toolbox - | albeit a necessary one. +---------------
Well, "necessary" might be a bit strong, since even if using CL-WHO (or something like it) one can add an extended string readmacro (typically #"..."#) to allow large quantities of text or HTML to be included, including unescaped double-quotes.
But if you absolutely insist on having a "raw text" format that uses angle brackets the way HTML/XML do, like ASP or PHP, well, people have done that, too. Possibly the simplest version is the one I first heard of from Erik Naggum, who called it Enamel or NML, which was later refined by Tim Bradshaw into TML (Trivial Markup Language) and then DTML ("TML with macros", sayeth Tim). See the following for more details than I mention below:
Newsgroups: comp.lang.lisp From: Tim Bradshaw <t...@cley.com> Subject: Re: Difference between LISP and C++ Date: 24 Oct 2002 13:47:12 +0100 Message-ID: <ey3elagc8bz....@cley.com>
Basically, all of {N,T,DT}ML use angle brackets *as* sexp parentheses -- and thus there is no separate closing tag, only a matching right angle bracket -- but differs from {HT,X}ML in using a lexical marker (the "|" char, in NML/TML/DTML) separating the tag and attributes (if any) from the body of the element. In the [D]TML case, the attribute section is a plist of naked Lisp sexps which *are* evaluated (as in HTOUT & CL-WHO), and the body section is unevaluated plain text (thus double quotes need not be escaped) terminated by the matching right angle. E.g., whereas in CL-WHO or HTOUT (etc.) one might write this:
(:html (:head (:title () title)) ((:body :bgcolor "#ffffff") "Here is some body text, with a quoted \"foo\"," " and here is the value of FOO: " foo))
in TML it might be written this way:
<html | <head | <title | <lisp :val title |>>> <body :bgcolor "#ffffff" | Here is some body text, with a quoted "foo", and here is the value of FOO: <lisp :val foo |>>>
<aside|At various times, Erik used [...] or {...} to escape back into Lisp. I don't actually know what Tim currently uses for that these days in [D]TML, but at least the above is clearly feasible.>
One can unambiguously shortcut that into this (and Tim does, IIRC):
<html<head<title<lisp :var title>>> <body :bgcolor "#ffffff" Here is some body text, with a quoted "foo", and here is the value of FOO: <lisp :var foo>>>
Parsing {N,T,DT}ML into HTOUT/CL-WHO should be trivial, and is left as an exercise for the reader... ;-}
-Rob
----- Rob Warnock <r...@rpw3.org> 627 26th Avenue <URL:http://rpw3.org/> San Mateo, CA 94403 (650)572-2607
On Jan 9, 8:37 pm, r...@rpw3.org (Rob Warnock) wrote:
> WalterGR <walte...@gmail.com> wrote: > +--------------- > | The Lisp-in-data would be only one of the tools in my toolbox - > | albeit a necessary one. > +---------------
> Well, "necessary" might be a bit strong, since even if using CL-WHO > (or something like it) one can add an extended string readmacro > (typically #"..."#) to allow large quantities of text or HTML to > be included, including unescaped double-quotes.
a) "Necessary" as in "I have to convert a non-trivial amount of *SP code to use Lisp, and it is 'necessary' for me to do it quickly."
b) #"<a href="#section2">Go to section 2</a>"# though I see the read macro point in general.
> But if you absolutely insist on having a "raw text" format that uses > angle brackets the way HTML/XML do, like ASP or PHP, well, people have > done that, too.
The *SP approach is general enough that it's useful for generating text other than XML.
> Basically, all of {N,T,DT}ML use angle brackets *as* sexp parentheses -- > and thus there is no separate closing tag, only a matching right angle > bracket -- but differs from {HT,X}ML in using a lexical marker (the "|" > char, in NML/TML/DTML) separating the tag and attributes (if any) from > the body of the element. [snip]
> in TML it might be written this way:
> <html | > <head | <title | <lisp :val title |>>> > <body :bgcolor "#ffffff" | > Here is some body text, with a quoted "foo", > and here is the value of FOO: <lisp :val foo |>>>
> [snip]
> One can unambiguously shortcut that into this (and Tim does, IIRC):
> <html<head<title<lisp :var title>>> > <body :bgcolor "#ffffff" Here is some body text, with a > quoted "foo", and here is the value of FOO: <lisp :var foo>>>
> Parsing {N,T,DT}ML into HTOUT/CL-WHO should be trivial, and is left > as an exercise for the reader... ;-}
Right. But since HTML/XML/XHTML are "just" s-exprs, the approaches you outline are absolutely equivalent in expressiveness to the *SP approaches. What do the CL-WHO-esque approaches really buy someone?
So...
> But if you absolutely insist on having a "raw text" format that uses > angle brackets [snip]
s/angle brackets/parenthesis/g and back to you. :)
+--------------- | Right. But since HTML/XML/XHTML are "just" s-exprs, the approaches | you outline are absolutely equivalent in expressiveness to the *SP | approaches. +---------------
"Expressiveness", perhaps. But I've always found the *SP approaches to be *very* ugly & error-prone when it comes to embedding control structures such as loops, e.g.:
<html> <body> Here we have a sequence of numbers: <br> <?php $i=1; while($i<=5) { ?> This is number <?php echo $i; ?> <br> <?php $i++; } ?> </body> </html>
The fact that you can leave PHP-land (or ASP land, etc.) with a left bracket unclosed is simply horrifying to me! Now try doing this with a doubly-nested loop with additional IF-THEN-ELSEs in them, and things get *really* messy!
And if you want to avoid that, then you have to construct (almost) *all* of your output *inside* the *SP language [using "Response.Write" in ASP/VBScript or "echo" in PHP], which makes it equivalent in style [except worse, IMHO] to the HTOUT/CL-WHO-style methods -- and you're right back to quoting all your body text & and escaping quotes in text!
<?php echo "<html><body>Here we have a sequence of numbers:<br>"; $i=1; while($i<=5) { echo "This is number " . $i . "<br />"; $i++; } echo "</body></html>"; ?>
+--------------- | What do the CL-WHO-esque approaches really buy someone? +---------------
1. No such dangling brackets as above.
2. Can do everything in a natural Lisp environment, e.g.:
(with-html-output (s *standard-output*) (:html (:body "Here we have a sequence of numbers:" :br (dotimes (i 5) (htm "This is number" i :br)))))
3. Have *all* of Common Lisp available.
+--------------- | > But if you absolutely insist on having a "raw text" format that uses | > angle brackets [snip] | | s/angle brackets/parenthesis/g and back to you. :) +---------------
I don't "insist", myself, I just find HTOUT (or CL-WHO, etc.) more convenient when writing sizable pages with lots of control flow. E.g., a *small* example:
[I would show you much bigger examples with lots of tables with variously-formatted outputs from SQL database queries, but unfortunately they're part of clients' proprietary apps.]
-Rob
----- Rob Warnock <r...@rpw3.org> 627 26th Avenue <URL:http://rpw3.org/> San Mateo, CA 94403 (650)572-2607
On 2009-01-11, WalterGR <walte...@gmail.com> wrote:
> Right. But since HTML/XML/XHTML are "just" s-exprs, the approaches
Symbolic expressions denote objects that have semantics, whereas XML is just text salad.
Giving a hierarchical structure to text is a long way from having a rich data notation.
In XML, you can't for instance distinguish symbols, numbers, strings, lists, vectors. Not without embedding some convention for this which requires additional parsing that is outside of XML.
What is the XML notation for a double-precision floating point number?
Complex number?
What if I want to put the 255 into XML using hex? I'm writing a Lisp program, so I think I will pick #xFF. The XML parser will pull that out as a string, and we can use READ-FROM-STRING to finish the job.
> On 2009-01-11, WalterGR <walte...@gmail.com> wrote: >> Right. But since HTML/XML/XHTML are "just" s-exprs, the approaches
> Symbolic expressions denote objects that have semantics, whereas > XML is just text salad.
> Giving a hierarchical structure to text is a long way from having a > rich data notation.
> In XML, you can't for instance distinguish symbols, numbers, strings, > lists, vectors. Not without embedding some convention for this which > requires additional parsing that is outside of XML.
> What is the XML notation for a double-precision floating point number?
> Complex number?
So you write the declaration is XSL. On second thought, don't. Use Relax NG instead. XSL is verbose to the point of driving you insave.
> What if I want to put the 255 into XML using hex? I'm writing a Lisp > program, so I think I will pick #xFF. The XML parser will pull that out > as a string, and we can use READ-FROM-STRING to finish the job.
That would be a transform so you would use XSLT.
Other standards.. XML-RPC, SOAP, XQuery, XPath, XSS, SVG... The general idea is you define a XML language and then define the sematics which is persumably written in something else.
Another great response, thanks. I don't want to argue personal preferences at all, but I thought I'd respond with my thoughts, in case they're helpful (or at least non-harmful ;) for others reading this.
On Jan 11, 3:20 am, r...@rpw3.org (Rob Warnock) wrote:
> But I've always found the *SP approaches > to be *very* ugly & error-prone when it comes to embedding control > structures such as loops, e.g.: [snip]
> The fact that you can leave PHP-land (or ASP land, etc.) with a > left bracket unclosed is simply horrifying to me!
Well, that's caught instantly by the interpreter/compiler, just as with mismatched parenthesis in Lisp. (IIRC. Maybe not if you don't execute that codepath, but it should still be caught in testing.) The bigger issue...
> Now try doing > this with a doubly-nested loop with additional IF-THEN-ELSEs in > them, and things get *really* messy!
Right. Editor support would be key. I'm sure it's doable in Emacs (which has modes for e.g. PHP) though I don't know if it's been done for Lisp.
> +--------------- > | What do the CL-WHO-esque approaches really buy someone? > +---------------
> 1. No such dangling brackets as above. > 2. Can do everything in a natural Lisp environment, e.g.:
I can see the appeal.
> 3. Have *all* of Common Lisp available.
Right. That's why I am leery about "template engines." (insert bastardization of Greenspun's Tenth...)
On Jan 10, 4:37 am, r...@rpw3.org (Rob Warnock) wrote:
> Possibly the simplest version is the one I first heard > of from Erik Naggum, who called it Enamel or NML, which was later > refined by Tim Bradshaw into TML (Trivial Markup Language) and then > DTML ("TML with macros", sayeth Tim). See the following for more > details than I mention below:
I don't remember TML having any direct line of ancestry to NML, though I am sure I saw articles by Erik about NML and probably nicked ideas. DTML was Dynamic TML.
> <aside|At various times, Erik used [...] or {...} to escape back into > Lisp. I don't actually know what Tim currently uses for that these > days in [D]TML, but at least the above is clearly feasible.>
> One can unambiguously shortcut that into this (and Tim does, IIRC):
> <html<head<title<lisp :var title>>> > <body :bgcolor "#ffffff" Here is some body text, with a > quoted "foo", and here is the value of FOO: <lisp :var foo>>>
I'm pretty sure that DTML allowed that sort of thing. There were subtle distinctions between <foo> and <foo|>: the former is an empty element (in XML it would be <foo/> while the latter is an element whose body just happens to be empty.
There are some more hacks, notably a special case syntax which looks like <|x ... x|> where x is any character: typically this was used for literal sections which the TML parser ignored, using |: <||html-stuff- here-gets-passed-through||>, but in fact you could register completely general handlers for characters this allowing embedding of completely general things.
There was more, but I have forgotten it now - 5 years is a long time...
Tim Bradshaw <tfb+goo...@tfeb.org> wrote: +--------------- | r...@rpw3.org (Rob Warnock) wrote: | > Possibly the simplest version is the one I first heard | > of from Erik Naggum, who called it Enamel or NML, which was later | > refined by Tim Bradshaw into TML (Trivial Markup Language) ... | > details than I mention below: | | I don't remember TML having any direct line of ancestry to NML, though | I am sure I saw articles by Erik about NML and probably nicked ideas. +---------------
I didn't mean that the ancestry was direct, only that Erik had been talking about the defects in where SGML/HTML/XML ended up for what seemed like "forever" (though the earliest bits I have saved are from mid-1998), and then in August 2001 he presented Enamel/NML (in response to a comment by *you* about using HTOUT format to write documents instead of XML), and then you presented TML in December 2001 ["PROGN for *ML?", about the problems of "tocify"]. From my outside perspective, the similarity was that both of you had chosen to use "|" as the <tag|contents> separator, and that TML looked more refined (has progressed further) than what Erik had done, that's all. Sorry if I implied anything more.
+--------------- | > ...and then DTML ("TML with macros", sayeth Tim). | | DTML was Dynamic TML. +---------------
Yes, I knew that, but in this article, you said:
From: tfb+goo...@tfeb.org (Tim Bradshaw) Newsgroups: comp.lang.lisp Subject: Re: Be afraid of XML Date: 15 Mar 2004 12:04:40 -0800 Message-ID: <fbc0f5d1.0403151204.47798...@posting.google.com> ... r...@rpw3.org (Rob Warnock) wrote: > Google in the c.l.l. archives for "TML", a format Tim Bradshaw > uses that was inspired by some postings by Erik Naggum. In TML, > the above would be: > <operator|Ten pages of text>
For what it's worth, we've now used TML (in its incarnation as DTML, which is TML with a macro system) for pretty substantial documents ...
I think that's where I must have gotten the idea that the "dynamic" part was mainly the macros.
+--------------- | There are some more hacks, notably a special case syntax which looks | like <|x ... x|> where x is any character: typically this was used for | literal sections which the TML parser ignored, using |: <||html-stuff- | here-gets-passed-through||>, but in fact you could register completely | general handlers for characters this allowing embedding of completely | general things. +---------------
Neat! Thanks for the additional details.
-Rob
----- Rob Warnock <r...@rpw3.org> 627 26th Avenue <URL:http://rpw3.org/> San Mateo, CA 94403 (650)572-2607
WalterGR <walte...@gmail.com> writes: > It doesn't appear that CLPs permit "embedding arbitrary Lisp > expressions," unless I've missed something.
Not directly, but indirectly by defining clp functions.
Petter -- A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail?
On Jan 13, 9:18 am, r...@rpw3.org (Rob Warnock) wrote:
> I think that's where I must have gotten the idea that the > "dynamic" part was mainly the macros.
You're right, it was. I was being confusing - I wasn't disagreeing, just expanding the acronym really. While checking stuff I noticed that the manual talks about being written in "Trivial TML" which slightly gives the lie as to what TML stands for...
Sigh, one day I may have time to revive all this stuff. Probably when I've retired.
> I'm looking for a Lisp library to create text output a la e.g. ASP, > JSP, PHP, or erb. (That is, embedding arbitrary Lisp expressions in > text files, which are then evaluated to create text output.) So far
{head {title "foobar!"}} {body {h3 "foo!"} {p :align "justify" "some" "thing"} {p "Every string is escaped: <>&"} (format t "But this is not: <foo>") {p (emit-html:esc (identity "stuff can be <escaped> at runtime"))}}}
Which leads to predictable output. I made it so i wouldn't have to escape every single user-supplied string, forget some cases and get bitten by cross-site scripting a year later.
The reader hack even works with paredit and emacs' paren-matching. Yay!
</shameless-plug>
-- You only have power over people so long as you don’t take everything away from them. But when you’ve robbed a man of everything he’s no longer in your power — he’s free again. -- Aleksandr Isayevich Solzhenitsyn