Report on using Shake as a static site generator

[Ian Jeffries]

It ended up at about 200 lines, twice as long as my previous code that used a specialized static site generator.

The length increase was mainly due to having to write out some data structures (like Page for website articles) and web related functions that the static site generator used to provide. Writing out the data structures was a actually a big improvement because it made the code more explicit.

A lot of useful web related functions are currently embedded in other projects (for instance withUrls from Hakyll: https://jaspervdj.be/hakyll/reference/Hakyll-Web-Html.html). If more people start using Shake as a static site generator it might eventually become worth it to make a generator-agnostic package of utilities that are useful when making static sites.

The code is here: https://github.com/seagreen/housejeffries/blob/master/Main.hs I'd love to hear comments on it if I'm doing something weirdly, this was my first Shake project.

[Neil Mitchell]

Hi Ian,

Thanks for sharing your experiences. Having the code no shorter in
Shake does seem like a theme - it ends up being maintainable and
clear, rather than concise. I guess that isn't a problem provided the
code is still seen as an improvement. Some comments going through the
details:

* No need to put alwaysRerun on a phony, they are alwaysRerun as
standard, e.g. line 24.

* In Haskell, adding return () as anything other than the last
statement of a do block is unecessary, e.g. line 25.

* I'd personally use more restricted functions where possible - for
example map instead of fmap on line 44, mapM instead of traverse on
line 39, ++ instead of <> on line 56.

* For getPages, you reverse it. My best understanding is this means
you'll end up with pages in reverse sorted lexical order, while pages
have ids. Wouldn't it be better to sort the Pids after they are [Int],
so you can put them in a guaranteed order based on their Int value?

* You action on line 52 seems to always call clean, which seems a bit
weird. Or do you not care about persistence and minimal rebuilding at
all?

Overall, it looks simple, clear and well structured - a really nice
example of a moderate size Shake file.

Thanks, Neil

[Ian Jeffries]

> No need to put alwaysRerun on a phony, they are alwaysRerun as
standard, e.g. line 24.

Got it!

> In Haskell, adding return () as anything other than the last
statement of a do block is unecessary, e.g. line 25.

`return ()` was leftover editing junk. Still super embarrassing.

> I'd personally use more restricted functions where possible - for
example map instead of fmap on line 44, mapM instead of traverse on
line 39, ++ instead of <> on line 56.

Interesting. My ways probably won't change based on this thread, but I'm curious to hear your reasons. The fact that you prefer map to fmap makes me think you'd also prefer Data.HashMap.Strict.map (in the form of H.map or whatever) to fmap, is that right?

> For getPages, you reverse it. My best understanding is this means
you'll end up with pages in reverse sorted lexical order, while pages
have ids. Wouldn't it be better to sort the Pids after they are [Int],
so you can put them in a guaranteed order based on their Int value?

Oh dang, numerical order is definitely the way to go here. Nice catch.

> You action on line 52 seems to always call clean, which seems a bit
weird. Or do you not care about persistence and minimal rebuilding at
all?

This is the thing I was most interested in. Due mistakenly thinking `tree` showed hidden files (it doesn't, you have to use `tree -a`) I didn't think clean was making any difference. Now I see the `.shake.database` file that I had been destroying. Thanks for clearing that up!

Big picture comment: this was a fun experience, and Shake+Pandoc deserve to be mentioned alongside the other most popular static site generators. This doesn't seem like a big deal (static site generators are kind of glorified toys), but it's a nice demonstration of effective Haskell.

If anyone else writes a static site generators in Shake (I've already seen this one) let me know. Perhaps we'll get enough to make a list somewhere. Also, if anyone finds a great way of making Atom feeds in Haskell definitely let me know, in the meantime I will try to add erratic improvements to the `feed` library.

[Neil Mitchell]

>> I'd personally use more restricted functions where possible - for
> example map instead of fmap on line 44, mapM instead of traverse on
> line 39, ++ instead of <> on line 56.
>
> Interesting. My ways probably won't change based on this thread, but I'm
> curious to hear your reasons. The fact that you prefer map to fmap makes me
> think you'd also prefer Data.HashMap.Strict.map (in the form of H.map or
> whatever) to fmap, is that right?

Indeed, I tend to use H.map over fmap, and I don't expect you to
change your preferences. I just find it easier when reading the code
to use the slightly more monomorphic ones in the very common case, so
I can anchor my expectations, and then when I see "traverse" I know
it's doing something a little bit "more funky", and know I'm working
on an unusual type. It might also be habit - when I learnt Haskell of
those generalisations only fmap existed.

> Big picture comment: this was a fun experience, and Shake+Pandoc deserve to
> be mentioned alongside the other most popular static site generators. This
> doesn't seem like a big deal (static site generators are kind of glorified
> toys), but it's a nice demonstration of effective Haskell.

Glad you liked it. My experience of these tasks is that Shake gives
you incremental rebuilding (gently handy), but in many cases the big
wins are automatic parallelisation and robust exception handling .

Thanks, Neil

[Ian Jeffries]

Glad to share! Totally worth it for the primo code review, thank you.