Escaping in Markdown

[William Kong]

Hi there,

I'm fairly new to Haykll and I've noticed that when writing text in .markdown documents, certain characters such as apostrophes, quotation marks, ellispses and percent signs need to be escaped using \.

Does Hakyll or Pandoc have any utilities to parse/escape a certain subset of these characters automatically when it routes to an .html file? If so, how would one implement the method?

Thanks in advance,

William

[Kyle Marek-Spartz]

I don’t think that’s the intended behavior. Could you attach your site.hs?

–

Kyle Marek-Spartz

On May 1, 2014, 10:29:17 AM, William Kong <wwko...@gmail.com> wrote:

---

--
You received this message because you are subscribed to the Google Groups "hakyll" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hakyll+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[William Kong]

Here you go.

[William Kong]

Update: I've cleaned up my site.hs code so that it's much more bearable to read by adding a few more wrapper functions. It still should do what the old site.hs does.

On Thursday, 1 May 2014 12:00:12 UTC-4, Kyle Marek-Spartz wrote:

[Jasper Van der Jeugt]

You configuration looks reasonably standard, so I'm not sure what's
going on.

Could you provide some more information? Are these normal characters
inside markdown text blocks? What do they get rendered to when not
escaping them? What happens if you escape them?

Peace,
Jasper

On Thu, May 01, 2014 at 12:42:55PM -0700, William Kong wrote:
> Here you go.
>
> On Thursday, 1 May 2014 12:00:12 UTC-4, Kyle Marek-Spartz wrote:
> >
> > I don’t think that’s the intended behavior. Could you attach your site.hs?
> >
> > –
> > Kyle Marek-Spartz
> >
> >
> >

> > On May 1, 2014, 10:29:17 AM, William Kong <wwko...@gmail.com <javascript:>>
> > wrote:
> > ------------------------------

> > Hi there,
> >
> > I'm fairly new to Haykll and I've noticed that when writing text in
> > .markdown documents, certain characters such as apostrophes, quotation
> > marks, ellispses and percent signs need to be escaped using \.
> >
> > Does Hakyll or Pandoc have any utilities to parse/escape a certain subset
> > of these characters automatically when it routes to an .html file? If so,
> > how would one implement the method?
> >
> > Thanks in advance,
> > William
> > --
> > You received this message because you are subscribed to the Google Groups
> > "hakyll" group.
> > To unsubscribe from this group and stop receiving emails from it, send an

> > email to hakyll+un...@googlegroups.com <javascript:>.

> > For more options, visit https://groups.google.com/d/optout.
> >
> >
> >
>
> --
> You received this message because you are subscribed to the Google Groups "hakyll" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hakyll+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

> --------------------------------------------------------------------------------
> {-# LANGUAGE OverloadedStrings #-}
> import qualified Data.Map as M
> import Data.Monoid
> import qualified Data.Set as S
> import Hakyll
> import Text.Pandoc
> import Text.Pandoc.Options
>
>
>
> --------------------------------------------------------------------------------
>
> mathJaxScr :: String
> mathJaxScr = unlines ["<script type=\"text/javascript\" ",
> "src=\"http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML\">",
> "</script>"]
>
> sidebarList :: [Identifier]
> sidebarList = [ "contact.md",
> "notes.md",
> "pe.md",
> "programming.md" ]
>
> myFeedConfiguration :: FeedConfiguration
> myFeedConfiguration = FeedConfiguration
> { feedTitle = "wwkong.github.io"
> , feedDescription = "A blend of mathematics, finance, and computer science."
> , feedAuthorName = "William Kong"
> , feedAuthorEmail = "wwko...@gmail.com"
> , feedRoot = "http://wwkong.github.io/"
> }
>
> pandocMathCompiler =
> let writerOptions = defaultHakyllWriterOptions {
> writerExtensions = writerExtensions defaultHakyllWriterOptions,
> writerHTMLMathMethod = MathJax ""
> }
> in pandocCompilerWith defaultHakyllReaderOptions writerOptions
>
> --------------------------------------------------------------------------------
> postCtx :: Context String
> postCtx =
> dateField "date" "%B %e, %Y" `mappend`
> defaultContext
>
> postCtxTags :: Tags -> Context String
> postCtxTags tags = tagsField "tags" tags `mappend` postCtx
>
> mathCtx :: Context a
> mathCtx = field "mathjax" $ \item -> do
> metadata <- getMetadata $ itemIdentifier item
> return $ if "mathjax" `M.member` metadata
> then mathJaxScr
> else ""
>
>
> --------------------------------------------------------------------------------
> main :: IO ()
> main = hakyll $ do
>
> -- Basic folders
> match "images/*" $ do
> route idRoute
> compile copyFileCompiler
>
> match "css/*" $ do
> route idRoute
> compile compressCssCompiler
>
> match "files/*" $ do
> route idRoute
> compile copyFileCompiler
>
> -- Build tags
> tags <- buildTags "posts/*" (fromCapture "tags/*.html")
>
> -- Create posts
> match "posts/*" $ do
> route $ setExtension ".html"
> compile $ do
> pandocMathCompiler
> >>= saveSnapshot "content"
> >>= return . fmap demoteHeaders
> >>= loadAndApplyTemplate "templates/post.html" (mathCtx `mappend` postCtxTags tags)
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` postCtxTags tags)
> >>= relativizeUrls
>
> -- Home page
> match "index.html" $ do
> route idRoute
> compile $ do
> posts <- fmap (take 3) . recentFirst =<< loadAll "posts/*"
> let indexContext =
> listField "posts" (postCtxTags tags) (return posts) <>
> field "tags" (\_ -> renderTagList tags) <>
> defaultContext
>
> getResourceBody
> >>= applyAsTemplate indexContext
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` indexContext)
> >>= relativizeUrls
>
> -- Achives
> match "archive.html" $ do
> route idRoute
> compile $ do
> posts <- recentFirst =<< loadAll "posts/*"
> let indexContext =
> listField "posts" (postCtxTags tags) (return posts) <>
> field "tags" (\_ -> renderTagList tags) <>
> defaultContext
>
> getResourceBody
> >>= applyAsTemplate indexContext
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` indexContext)
> >>= relativizeUrls
>
> -- Default template and other basic pages
> match (fromList sidebarList) $ do
> route $ setExtension "html"
> compile $ pandocCompiler
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` defaultContext)
> >>= relativizeUrls
>
> -- Resume page
> match "resume.md" $ do
> route $ setExtension ".html"
> compile $ do
> cvTpl <- loadBody "templates/resume.html"
> defaultTpl <- loadBody "templates/default.html"
> pandocCompiler
> >>= applyTemplate cvTpl (mathCtx `mappend` defaultContext)
> >>= applyTemplate defaultTpl (mathCtx `mappend` defaultContext)
> >>= relativizeUrls
>
> -- Post list
> create ["posts.html"] $ do
> route idRoute
> compile $ do
> posts <- recentFirst =<< loadAll "posts/*"
> let ctx = constField "title" "Archive" <>
> listField "posts" (postCtxTags tags) (return posts) <>
> defaultContext
>
> makeItem ""
> >>= loadAndApplyTemplate "templates/tag.html" (mathCtx `mappend` ctx)
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` ctx)
> >>= relativizeUrls
>
> -- Post tags
> tagsRules tags $ \tag pattern -> do
> let title = "Posts tagged " ++ tag
> route idRoute
> compile $ do
> posts <- recentFirst =<< loadAll pattern
> let ctx = constField "title" title <>
> listField "posts" (postCtxTags tags) (return posts) <>
> defaultContext
> makeItem ""
> >>= loadAndApplyTemplate "templates/tag.html" (mathCtx `mappend` ctx)
> >>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` ctx)
> >>= relativizeUrls
>
> --- Atom Feed
> create ["atom.xml"] $ do
> route idRoute
> compile $ do
> let feedCtx = postCtx `mappend` bodyField "description"
> posts <- fmap (take 10) . recentFirst =<<
> loadAllSnapshots "posts/*" "content"
> renderAtom myFeedConfiguration feedCtx posts
>
> -- Render Templates
> match "templates/*" $ compile templateCompiler
>
> --------------------------------------------------------------------------------

[William Kong]

To be more specific, if I were to not escape any characters in my post under the configuration provided, I would get the error

commitBuffer: invalid argument (invalid character)

and the page would compile text up to the character that needed to be escaped, excluding that character. These are all normal characters in a markdown text block. Characters that need to be escaped include single and double quotes (' , ") and ellipses (...). In addition, an odd case is the abbreviation "e.g.". If the second period is not escaped, the text compiles but I get the result e.g.�.

Another odd thing is that if I removed the MathJax code (i.e. have the pandocMathCompiler take the default reader and writer options), I would need to escape even more characters such as round and square brackets ( (), [] )

Finally, I noticed that you mentioned a similar error in your FAQ and a solution for use on linux distros. I'm not sure if it's applicable here since the above don't seem to be special characters, but if it's related, is there a similar solution for Windows 8 users?

[Jasper Van der Jeugt]

By default, Hakyll will render some of these sequences to the
corresponding Unicode variants to produce a (subjectively) more
beautiful result (e.g. replace "..." with "…").

This fails if your setup does not support writing UTF-8.

You can turn this behaviour off by specifying

readerSmart = False

in the pandoc `ReaderOptions`.

Alternatively, see this thread [1] for a possible solution on Windows 7
(which I guess should also work for Windows 8).

[1]: https://groups.google.com/forum/#!topic/hakyll/jrGATyI1omI

Hope this helps,
Peace,
Jasper

> > > , feedAuthorEmail = "wwko...@gmail.com <javascript:>"

[William Kong]

Thanks, Jasper! The latter solution was what worked best for me.