proposal to use HSP in gitit.

7 views
Skip to first unread message

Thomas Hartman

unread,
Mar 27, 2009, 11:25:41 PM3/27/09
to gitit-discuss
I think the readability and maintainability of the formattedPage function (and a few similar others) would benefit hugely if gitit switched to an approach of using hsp and explicit arguments rather than HStringTemplate and the grab-bag Params value.

An example that suggests how nicely hsp could handle this is at

http://patch-tag.com/repo/haskell-learning/browse (hsp/HSP/helloWorld.hs demo, which uses included module HSP.Identity)

If you are amenable to this suggestion, I volunteer to recode this function, and perhaps some others, to use hsp.

This will make it a lot easier to use gitit in a modular way from external happs apps like patch-tag, and maybe others that, would like a branded drop-in cms.

The big win with hsp is that the correctness of the html is statically checked, and yet it looks like php/asp/name your common view, where haskell values can be directly plugged into a template in a type-safe way using <% %> tags.

Passing the arguments directly in rather than the Params grab bag reduces the global vars effect, which is beginning to feel like a real headache to me during  a refactor that should be a lot more trivial.

What do you think?

--
Thomas Hartman

Darcs hosting: patch-tag.com
The elegance of haskell, the simplicity of php:  happstack.com

Bruce D'Arcus

unread,
Mar 28, 2009, 12:20:05 PM3/28/09
to gitit-...@googlegroups.com
I can't really comment on Haskell templating (the example fragment is
a little too simply to evaluate, and I don't know how things look
now), but ..

On Fri, Mar 27, 2009 at 11:25 PM, Thomas Hartman
<thomash...@googlemail.com> wrote:

> This will make it a lot easier to use gitit in a modular way from external
> happs apps like patch-tag, and maybe others that, would like a branded
> drop-in cms.

Given how much power underlies gitit I wonder if there isn't room to
expand it beyond the traditional wiki? Do you have something specific
in mind here?

Bruce

Thomas Hartman

unread,
Mar 28, 2009, 3:22:25 PM3/28/09
to gitit-...@googlegroups.com
Gitit plus plugins could be the poster child for HAppS... maybe it is arlready.

But I think it would be easier to be a poster child if the templating system was closer to something more people understood, more maintainable too.

Patch-Tag plans to use gitit for at least repo browsing and wiki. I can certainly imagine adding additional "plug ins" to gitit such as a todo list, bug tracker, ledger billing system... birthday card reminder... wash the car... :) but I think for now I would settle for making gitit easier to hack on and use a plug-in (rather than standalone) and increasing the userbase.

Bruce D'Arcus

unread,
Mar 29, 2009, 12:30:25 PM3/29/09
to gitit-...@googlegroups.com
On Sat, Mar 28, 2009 at 3:22 PM, Thomas Hartman
<thomash...@googlemail.com> wrote:

> Gitit plus plugins could be the poster child for HAppS... maybe it is
> arlready.
>
> But I think it would be easier to be a poster child if the templating system
> was closer to something more people understood, more maintainable too.

Yes. My experience with web apps is limited to Python (web.py, django)
a bit of Ruby. Based on that, I was expecting to see a "templates"
directory with some nice, clean, templates. But, alas, they're not
there.

How does HSP compare to HStringTemplate?

> Patch-Tag plans to use gitit for at least repo browsing and wiki. I can
> certainly imagine adding additional "plug ins" to gitit such as a todo list,
> bug tracker, ledger billing system... birthday card reminder... wash the
> car... :) but I think for now I would settle for making gitit easier to hack
> on and use a plug-in (rather than standalone) and increasing the userbase.

For the record, I'm a scholar, which means I deal with documents, and
a bunch of related data.

These days my home site is built from content in a git repo that
consists of some RDF files for the later, and some markdown files for
the former. They're served up as static files generic by Python + some
templates.

But I could imagine something like gitit could be enhanced to handle
this sort of thing. It might have relevance for our previous
discussions of integrating citation support (where citation become
just one example built on flexible structured data).

Bruce

Gwern Branwen

unread,
Mar 29, 2009, 2:19:05 PM3/29/09
to gitit-...@googlegroups.com
One thing that concerns me is that hsp is rather complex. It was also apparently orphaned for a time. Worse, it touches on areas that very quickly bitrot. I worry that if gitit makes use of hsp, it will backfire in the future.

--
gwern
signature.asc

Don Stewart

unread,
Mar 29, 2009, 2:19:38 PM3/29/09
to gitit-...@googlegroups.com
gwern0:

Yes, this is my concern too:

* maintainers
* size
* dependencies

while we know Sterling is doing a good job with HStringTemplate.

-- Don

Anton van Straaten

unread,
Mar 29, 2009, 3:43:57 PM3/29/09
to gitit-...@googlegroups.com
Don Stewart wrote:
> gwern0:

>> One thing that concerns me is that hsp is rather complex. It was also apparently orphaned for a
> time. Worse, it touches on areas that very quickly bitrot. I worry that if gitit makes use of
> hsp, it will backfire in the future.
>>
>
> Yes, this is my concern too:
>
> * maintainers
> * size
> * dependencies
>
> while we know Sterling is doing a good job with HStringTemplate.

I like the idea of HSP, and it'd be nice to have it available as an
option, but I share the concerns about having it be the only supported
template system.

I'm not very familiar with HSP, though. How well does it support
changes to templates at runtime? With HStringTemplate, you can easily
modify a template or create new templates, and use them, while the
server is running (although gitit doesn't take advantage of this at the
moment). With HSP, to achieve this you'd need to have:

1. A full Haskell compilation environment on the deployment server.
2. Haskell recompilation of templates at runtime.
3. Dynamic loading of templates at runtime.

#1 could be a disadvantage in some situations. I'm not sure how viable
#3 is - the hs-plugin approach would presumably work, but that adds yet
more complexity.

I think Turbinado is using HSP. Does it do any of the above?

Anton

John MacFarlane

unread,
Apr 1, 2009, 11:28:52 AM4/1/09
to gitit-...@googlegroups.com
+++ Thomas Hartman [Mar 27 09 20:25 ]:

> I think the readability and maintainability of the formattedPage
> function (and a few similar others) would benefit hugely if gitit
> switched to an approach of using hsp and explicit arguments rather than
> HStringTemplate and the grab-bag Params value.
> An example that suggests how nicely hsp could handle this is at
> [1]http://patch-tag.com/repo/haskell-learning/browse

> (hsp/HSP/helloWorld.hs demo, which uses included module HSP.Identity)
> If you are amenable to this suggestion, I volunteer to recode this
> function, and perhaps some others, to use hsp.
> This will make it a lot easier to use gitit in a modular way from
> external happs apps like patch-tag, and maybe others that, would like a
> branded drop-in cms.
> The big win with hsp is that the correctness of the html is statically
> checked, and yet it looks like php/asp/name your common view, where
> haskell values can be directly plugged into a template in a type-safe
> way using <% %> tags.
> Passing the arguments directly in rather than the Params grab bag
> reduces the global vars effect, which is beginning to feel like a real
> headache to me during a refactor that should be a lot more trivial.
> What do you think?

Thomas,

After reading the discussion, I'm inclined to stay with HStringTemplate
for now, for the reasons others have given. But your concern with "the
Params grab bag" seems an independent issue. I'm sure there are ways
gitit could be improved by refactoring, so suggestions here are welcome.
The current design predated happstack 0.2, and some things could be
simplified now, though it would require some work.

John

Thomas Hartman

unread,
Apr 1, 2009, 3:48:19 PM4/1/09
to gitit-...@googlegroups.com
OK, that seems reasonable.

If I do any cleanup on gitit-based or filestore-derived functions for use in patch-tag, I'll try and pas them on.

Anton van Straaten

unread,
Apr 2, 2009, 2:13:21 AM4/2/09
to gitit-...@googlegroups.com
John MacFarlane wrote:
> +++ Thomas Hartman [Mar 27 09 20:25 ]:
>> Passing the arguments directly in rather than the Params grab bag
>> reduces the global vars effect, which is beginning to feel like a real
>> headache to me during a refactor that should be a lot more trivial.
>> What do you think?
...

> After reading the discussion, I'm inclined to stay with HStringTemplate
> for now, for the reasons others have given. But your concern with "the
> Params grab bag" seems an independent issue. I'm sure there are ways
> gitit could be improved by refactoring, so suggestions here are welcome.
> The current design predated happstack 0.2, and some things could be
> simplified now, though it would require some work.

I've recently done some work which may be relevant here.

I wanted to integrate Gitit into a more general Happstack server that I
use for various applications. I found that it made sense to integrate
some of the things I'm doing into the Gitit chain of content
transformations. However, there's currently no plugin mechanism for
transformations unrelated to Pandoc, so I ended up refactoring a few
Gitit.hs functions, like showPage and rawContents, to allow me to plug
in the steps I needed in a generic way.

Dealing with the Params a little more easily was one of the first issues
I ran into. For that and other reasons, I wrapped the Params in a
monad, conceptually like this:

type ContentTransformer a = StateT Params (WebT IO) a

The (WebT IO) is just a Happstack detail - when this monad is evaluated,
it returns a value of type (Web a), e.g. a Web Response which can be
returned to the client. (Also, I actually used a wrapper around Params,
not shown above, to avoid invasive modifications to Params.)

Functions that return a ContentTransformer thus have automatic access to
the Params. This doesn't in itself fix the "grab bag" aspect, although
the overall approach could be extended to make parameterization more
compositional. Besides, there are other benefits which could prove
useful in supporting optional features, such as an optional or
plugin-based HSP transformation.

I ended up refactoring seven core functions in Gitit.hs: showPage,
showRawPage, showFileAsText, showHighlightedSource, exportPage,
rawContents, and pageAsPandoc. I refactored them into a set of about 20
fairly general, single-purpose monadic combinators that can very easily
be combined and extended to produce custom content transformations.

For example, my implementation of Gitit's showPage function now looks
like this:

showPage :: String -> Params -> Web Response
showPage = runPageTransformer htmlViaPandoc

htmlViaPandoc :: ContentTransformer Response
htmlViaPandoc = cachedContents >>= stringToPandocCached
>>= pandocToWikiDivCached >>= addMathSupport >>= applyWikiTemplate

The use of the monad allows the combinators to be easily composed using
bind.

This means that to support an alternative template mechanism like HSP,
alongside HStringTemplate, it should only be necessary to define a
handler like the above, with the applyWikiTemplate combinator replaced
with one that handles HSP templates.

One way I'm using this is to support views of data that don't come from
a file. Since reading of files is also implemented as a combinator,
it's easy to replace that with something that reads content from some
other source, such as in-memory data or a relational database. For
example, displaying a Gitit user on a wiki page can be done with a
handler like this:

showUser :: String -> Params -> Web Response
showUser = runPageTransformer $ userResource >>= userView
>>= wikiDivify >>= wikiTemplate

This is supported by two small combinators - userResource for obtaining
user data, and userView for rendering it. The other two combinators are
general ones that already existed.

My current code is: at http://content.scheming.org/ContentTransformer

This isn't really supposed to be a "release" - I mainly wanted to get
some reactions and feedback before going too much further. However, the
code works, and should leave Gitit with all existing functionality plus
the additional capabilities I'm describing.

Using this approach, it should be quite simple to define an explicit
plugin mechanism for arbitrary transformers. All that's needed to
define a plugin is any necessary ContentTransformer combinators, along
with Gitit handlers that combine the appropriate combinators.

The main remaining requirement is to register those handlers with
Gitit's dispatcher. Currently, this can of course be done just by
editing Gitit.hs, but the combinator approach opens up the possibility
of a more data-driven mapping from URLs to combinator sequences.

Some of the possibilities here may go beyond the intended scope of
Gitit. Even if that's true, something like what Gitit already does
should be at the core of any good content management system, so it may
make sense for Gitit to expose its core functionality through a general
API along these lines - i.e., Gitit as a library - which would then
support various uses, ranging from a standard out-of-the-box wiki to a
much more customizable content management system.

Of course, any feedback is most welcome!

Anton

John MacFarlane

unread,
Apr 2, 2009, 6:16:58 AM4/2/09
to gitit-...@googlegroups.com
+++ Anton van Straaten [Apr 02 09 02:13 ]:

This looks great! I really like the idea. Let's see if others have
comments.

John

Simon Michael

unread,
Apr 2, 2009, 6:21:52 AM4/2/09
to gitit-...@googlegroups.com
Just, me too. It all sounds very good and I'll be trying it, thanks!

John MacFarlane

unread,
Apr 7, 2009, 12:43:21 PM4/7/09
to gitit-...@googlegroups.com
+++ John MacFarlane [Apr 02 09 03:16 ]:

I would like to integrate this into gitit, if it's okay with you.
I think it would make sense to put all of the functions now in
Gitit.Convert into Gitit.ContentTransformer.

Also, textToPandoc could be factored into two pieces -- one that does the
basic conversion, another that applies PageTransform plugins.

How does that sound?

John

Anton van Straaten

unread,
Apr 7, 2009, 2:03:57 PM4/7/09
to gitit-...@googlegroups.com
John MacFarlane wrote:
> I would like to integrate this into gitit, if it's okay with you.

Yes, that would be great, thanks!

> I think it would make sense to put all of the functions now in
> Gitit.Convert into Gitit.ContentTransformer.
>
> Also, textToPandoc could be factored into two pieces -- one that does the
> basic conversion, another that applies PageTransform plugins.
>
> How does that sound?

Sounds good to me. Unless you'd prefer to do it yourself, I'll do a new
version of ContentTransformer.hs with those changes.

Anton

John MacFarlane

unread,
Apr 7, 2009, 3:48:55 PM4/7/09
to gitit-...@googlegroups.com
+++ Anton van Straaten [Apr 07 09 14:03 ]:

Sure, go ahead. If you use git, you can get the latest development
version from the repository on github, make your changes on top of
that, and send them to me using 'git format-patch'.

John

Anton van Straaten

unread,
Apr 7, 2009, 4:52:26 PM4/7/09
to gitit-...@googlegroups.com
John MacFarlane wrote:
> If you use git, you can get the latest development
> version from the repository on github, make your changes on top of
> that, and send them to me using 'git format-patch'.

Will do. I should be able to send a patch tonight.

Anton

Anton van Straaten

unread,
Apr 9, 2009, 4:13:57 AM4/9/09
to gitit-...@googlegroups.com
I've sent a patch for the ContentTransformer change, off-list.

There's one minor design issue I want to mention while I'm thinking
about it. I originally used a Context data type to minimize the changes
to the Gitit source, i.e. to avoid having to change the Params
structure. It looks like this:

data Context = Context { ctxPage :: String
, ctxFile :: String
, ctxLayout :: PageLayout
, ctxParams :: Params
}

I'm not sure there's any particularly good reason that these fields
should be in a structure of their own any more. They could be merged
into the Params type now. Of course that makes Params bigger, but I
don't think the above factoring is particularly principled anyway.

The main effect that merging the fields into Params would have, other
than on code aesthetics, is to allow the debug logging to automatically
include the new fields in its output, since it prints the Params, not
the Context.

Anton

Reply all
Reply to author
Forward
0 new messages