[Haskell-cafe] GSoC Project Proposal: Markdown support for Haddock

[Johan Tibell]

Hi all,

Haddock's current markup language leaves something to be desired once
you want to write more serious documentation (e.g. several paragraphs
of introductory text at the top of the module doc). Several features
are lacking (bold text, links that render as text instead of URLs,
inline HTML).

I suggest that we implement an alternative haddock syntax that's a
superset of Markdown. It's a superset in the sense that we still want
to support linkifying Haskell identifiers, etc. Modules that want to
use the new syntax (which will probably be incompatible with the
current syntax) can set:

{-# HADDOCK Markdown #-}

on top of the source file.

Ticket: http://trac.haskell.org/haddock/ticket/244

-- Johan

_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Edsko de Vries]

Yes please!

-E

[Iustin Pop]

On Thu, Apr 04, 2013 at 06:41:19PM +0100, Edsko de Vries wrote:
> Yes please!

+1 as well. I find the current syntax too restrictive…

iustin

[kudah]

A sane markup for haddock would be greatly appreciated. I've grown
tired of noticing highlighted words arising from unescaped quotes all
over hackage.

On Thu, 4 Apr 2013 09:49:04 -0700 Johan Tibell <johan....@gmail.com>
wrote:

[Michael Snoyman]

+1

In case it can be useful in any way for this project, my markdown package[1] is certainly available for scavenging, though we'd likely want to refactor it to not use conduit (I can't imagine conduit being a good dependency for Haddock).

[1] http://hackage.haskell.org/package/markdown

[Simon Heath]

I humbly suggest reStructuredText rather than Markdown, which is what
is used by the Python community for documentation. Since it's specifically
made for documentation it may be nicer. But, I don't want to spark
a format argument.

There is also the Pandoc program, which is a universal-ish markup-
language-converter, conveniently written in Haskell. Might be a place
to start for this, regardless of the language chosen:
http://www.johnmacfarlane.net/pandoc/

Simon

Excerpts from Johan Tibell's message of 2013-04-04 09:49:04 -0700:

[Kim-Ee Yeoh]

On Fri, Apr 5, 2013 at 3:04 AM, Simon Heath <icef...@gmail.com> wrote:
> I humbly suggest reStructuredText rather than Markdown, which is what
> is used by the Python community for documentation. Since it's specifically
> made for documentation it may be nicer. But, I don't want to spark
> a format argument.

Could you say something about /why/ you make the suggestion? I, for
one, would be happy to google and read links, but what's missing from
that experience would be input from a fellow haskeller. In context. In
real-time. On topic.

-- Kim-Ee

[Konstantine Rybnikov]

I'd actually like to say that Python has great "documentation traditions" not just thanks to adoption of reStructuredText format, but a bit more. The greatest thing is an adoption of Sphinx [1] documentation engine. In Sphinx you write in extended (by special extensions) reStructuredText format, which is able to easily link to function definitions (with :func:`foo.bar.baz` or :class:`foo.bar.Baz`), other documents (with :doc:`other/doc`) and (most importantly) has extension called "autodoc" which goes and generates documentation for classes and functions automatically (by gathering it from docstrings and other things, similar to haddoc, if I'm not mistaken).

My main point is that thanks to adoption of "mixed documentation" techinique people usually tend to keep documentation with has these properties:

- it's up to date

- there's (most of the time) no need to keep separate API and non-API documentation

- API documentation can be easily be extended with long introduction without having to write it in source code (you can write it in document and then include autodoc)

I think Haskell's documentation would greatly benefit from (possible?) adoption of something like Sphinx, but I don't think it supports Haskell currently.

    [1]: http://sphinx.pocoo.org

[Aleksey Khudyakov]

On 4 April 2013 20:49, Johan Tibell <johan....@gmail.com> wrote:
> Hi all,
>
> Haddock's current markup language leaves something to be desired once
> you want to write more serious documentation (e.g. several paragraphs
> of introductory text at the top of the module doc). Several features
> are lacking (bold text, links that render as text instead of URLs,
> inline HTML).
>
> I suggest that we implement an alternative haddock syntax that's a
> superset of Markdown. It's a superset in the sense that we still want
> to support linkifying Haskell identifiers, etc. Modules that want to
> use the new syntax (which will probably be incompatible with the
> current syntax) can set:
>

If we are going to change haddock syntax we should add ability to add
math formulae to documentation. It's not currently possible and it makes
documenting numeric code properly difficult.

[Johan Tibell]

On Thu, Apr 4, 2013 at 9:49 AM, Johan Tibell <johan....@gmail.com> wrote:
> I suggest that we implement an alternative haddock syntax that's a
> superset of Markdown. It's a superset in the sense that we still want
> to support linkifying Haskell identifiers, etc. Modules that want to
> use the new syntax (which will probably be incompatible with the
> current syntax) can set:
>
> {-# HADDOCK Markdown #-}

Let me briefly argue for why I suggested Markdown instead of the many
other markup languages out there.

Markdown has won. Look at all the big programming sites out there,
from GitHub to StackOverflow, they all use a superset of Markdown. It
did so mostly (in my opinion) because it codified the formatting style
people were already using in emails and because it was pragmatic
enough to include HTML as an escape hatch.

[Ben Gamari]

Johan Tibell <johan....@gmail.com> writes:

> On Thu, Apr 4, 2013 at 9:49 AM, Johan Tibell <johan....@gmail.com> wrote:
>> I suggest that we implement an alternative haddock syntax that's a
>> superset of Markdown. It's a superset in the sense that we still want
>> to support linkifying Haskell identifiers, etc. Modules that want to
>> use the new syntax (which will probably be incompatible with the
>> current syntax) can set:
>>
>> {-# HADDOCK Markdown #-}
>
> Let me briefly argue for why I suggested Markdown instead of the many
> other markup languages out there.
>
> Markdown has won. Look at all the big programming sites out there,
> from GitHub to StackOverflow, they all use a superset of Markdown. It
> did so mostly (in my opinion) because it codified the formatting style
> people were already using in emails and because it was pragmatic
> enough to include HTML as an escape hatch.
>

For what it's worth, I think Markdown is a fine choice for very much the
same reason. RST has some nice properties (especially for documenting
Python), but Markdown is much more common. Moreover, I've always found
RST's linkification syntax a bit awkward.

Cheers,

- Ben

[Richard A. O'Keefe]

On 5/04/2013, at 12:34 PM, Johan Tibell wrote:
>
> Markdown has won. Look at all the big programming sites out there,
> from GitHub to StackOverflow, they all use a superset of Markdown.

Yes, but they tend to use _different_ supersets of Markdown.

Would it be too much to ask that a notation be used which has
a formal syntax and a formal semantics?

I mean, this *is* Haskell we're talking about, not some
slapped-together who-cares-if-a-primary-buffer-panel-falls-off
FireflyXXXXXXX dynamic programming language.

[Johan Tibell]

> Would it be too much to ask that a notation be used which has
> a formal syntax and a formal semantics?

We will document our superset, sure. That's what others did as well.
The point is using Markdown as the shared base.

[John MacFarlane]

+++ Simon Heath [Apr 04 13 13:04 ]:

> I humbly suggest reStructuredText rather than Markdown, which is what
> is used by the Python community for documentation. Since it's specifically
> made for documentation it may be nicer. But, I don't want to spark
> a format argument.
>
> There is also the Pandoc program, which is a universal-ish markup-
> language-converter, conveniently written in Haskell. Might be a place
> to start for this, regardless of the language chosen:
> http://www.johnmacfarlane.net/pandoc/
>
> Simon

(Pandoc author here.) It probably wouldn't make sense for a key
infrastructure component like Haddock to depend on a behemoth like
pandoc. But I could help out with a markdown-superset parser if needed.
I have an experimental thing here that could be used as a basis (it's 7x
faster than pandoc and uses 1/5 the memory, BSD licensed):
https://github.com/jgm/Markdown

Another idea: If someone contributed a Haddock markup writer to pandoc,
then documentation could be written in markdown (or RST or whatever) and
converted automatically to standard Haddock markup. David Lazar has
recently contributed a Haddock markup reader, but there is no writer.

Note: Creating a writer would be a bit tricky, because Haddock markup
isn't expressive enough for many of the constructions pandoc allows --
for example, if I'm not mistaken, you can't have multiple paragraphs
inside list items. Decisions would have to be made about how to deal
with such cases. There are also a few Haddock constructions that don't
correspond to anything in pandoc.

John

[Richard A. O'Keefe]

On 5/04/2013, at 2:00 PM, Johan Tibell wrote:

>> Would it be too much to ask that a notation be used which has
>> a formal syntax and a formal semantics?
>
> We will document our superset, sure. That's what others did as well.
> The point is using Markdown as the shared base.

Nononono.
Sure, the others "documented" their supersets.
But they did *NOT* provide what I am asking for:
- a FORMAL SYNTAX and
- a FORMAL SEMANTICS.
I tried to use one of these systems, and found myself
unable to determine which combinations of features were
legal and what legal combinations of features *meant*.
I corresponded with some people who had built markdown
parsers, and the answer was the same each time: they had
reversed engineered some other parser (typically a Perl
one) and bashed on it until they were bug-compatible.

If I want to get a particular effect in LaTeX or even in
HTML+CSS, I can usually figure it out *without* having to
run any program. If I want to get a particular effect in
Markdown, I flounder around and end up doing without.

I am sick of "documentation" that vaguely hints at things,
and I am especially sick of Markdown so-called documentation.

To say it one more time: I was unable to use the official
Markdown documentation,
http://daringfireball.net/projects/markdown/syntax,
to guide the construction of a parser.

For example, <br> is a valid URL enclosed in <. . .>, so
is it a link, as the "Automatic Links" section would suggest,
or is it embedded HTML, as the "Inline HTML" section would
suggest? Can you tell *from the documentation*?

For another example, is *foo**bar**ugh* supposed to map to
<em>foo<strong>bar</strong>ugh</em> or to
<em>foo</em><em>bar</em><em>ugh</em>?
Again, I'm not asking "what does this or that *program* do",
I'm asking "can you tell from the documentation what they
*ought* to do?"

If there is an unambiguous specification of Markdown somewhere
(specification; not program), I would be glad to see it.

[Ivan Lazar Miljenovic]

I don't think so; this was one of the big issues recently when people
were trying to get Gruber to actually _do_ something with Markdown as
there were all these corner cases.

>
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskel...@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe

--
Ivan Lazar Miljenovic
Ivan.Mi...@gmail.com
http://IvanMiljenovic.wordpress.com

[Kim-Ee Yeoh]

On Fri, Apr 5, 2013 at 10:44 AM, Ivan Lazar Miljenovic
<ivan.mi...@gmail.com> wrote:
> I don't think so; this was one of the big issues recently when people
> were trying to get Gruber to actually _do_ something with Markdown as
> there were all these corner cases.

In that case, surely this is an opportunity to convene a committee (a
la H98) to craft a formal spec?

-- Kim-Ee

[Ivan Lazar Miljenovic]

On 5 April 2013 15:49, Kim-Ee Yeoh <k...@atamo.com> wrote:
> On Fri, Apr 5, 2013 at 10:44 AM, Ivan Lazar Miljenovic
> <ivan.mi...@gmail.com> wrote:
>> I don't think so; this was one of the big issues recently when people
>> were trying to get Gruber to actually _do_ something with Markdown as
>> there were all these corner cases.
>
> In that case, surely this is an opportunity to convene a committee (a
> la H98) to craft a formal spec?

There have been calls for such a committee:
http://www.codinghorror.com/blog/2012/10/the-future-of-markdown.html

>
> -- Kim-Ee

--
Ivan Lazar Miljenovic
Ivan.Mi...@gmail.com
http://IvanMiljenovic.wordpress.com

[Krzysztof Skrzętnicki]

I think this thread has went to became a quest for One True Markup Language...

I would like to propose a different approach: while we are at extending Haddock why not make it possible to choose from a set of different documentation plugins? Something that works akin to the type: Text -> Environment -> Documentation, where Documentation is Haddock AST (possibly extended)?

That would make most people happy and put the source code writer in power: they could choose the syntax they would like to write their docs. Whether it is Haddock markup, Github Markdown, RST or OneTrueMarkup (c) (r) (patented).

The task of defining such an API could be actually harder than writing support for any particular syntax but I think it is more beneficiary in the long run.

Bonus points: provide the integration with .cabal files so that each package can inform which plugins it expects.

Extra bonus points: let the plugin be simply the name of a function (or datatype that implements a typeclass) which in turn would be taken from a well-defined scope. (think: library that can provide its own plugin for documenting itself).

Best regards,
Krzysztof Skrzętnicki

[Andrew Butterfield]

On 4 Apr 2013, at 22:53, Aleksey Khudyakov wrote:

> If we are going to change haddock syntax we should add ability to add
> math formulae to documentation. It's not currently possible and it makes
> documenting numeric code properly difficult.

How about support for .lhs files?
- both those with bird-tracks (which I don't use anymore)
and \begin{code}...\end{code} (which I do use).

My .lhs files are also LaTeX sources - I guess some way to nicely integrate
haddock markup/down/whatever with LaTeX stuff would be needed

+1 in any case

>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskel...@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe

--------------------------------------------------------------------
Andrew Butterfield Tel: +353-1-896-2517 Fax: +353-1-677-2204
Lero@TCD, Head of Foundations & Methods Research Group
Director of Teaching and Learning - Undergraduate,
School of Computer Science and Statistics,
Room G.39, O'Reilly Institute, Trinity College, University of Dublin
http://www.scss.tcd.ie/Andrew.Butterfield/
--------------------------------------------------------------------

[Aleksey Khudyakov]

On 5 April 2013 12:20, Andrew Butterfield

<Andrew.Bu...@scss.tcd.ie> wrote:
>
> On 4 Apr 2013, at 22:53, Aleksey Khudyakov wrote:
>
>> If we are going to change haddock syntax we should add ability to add
>> math formulae to documentation. It's not currently possible and it makes
>> documenting numeric code properly difficult.
>
> How about support for .lhs files?
> - both those with bird-tracks (which I don't use anymore)
> and \begin{code}...\end{code} (which I do use).
>
> My .lhs files are also LaTeX sources - I guess some way to nicely integrate
> haddock markup/down/whatever with LaTeX stuff would be needed
>

I'm not sure that it would help. If we to use haddock markup it need to
support math typesetting. And LaTeX IMHO isn't right tool for creating
hyperlinked API reference

[Andrew Butterfield]

I'm not proposing the LaTeX is used for hyperlinking the
reference - hence my comment about nicely integrating

Perhaps a \begin{haddock} ... \end{haddock} environment* ?

* This would only affect those using LaTeX/lhs - everyone else could haddock** as usual

** haddock = whatever markdow/up/sideways scheme you guys come up with...

--------------------------------------------------------------------
Andrew Butterfield Tel: +353-1-896-2517 Fax: +353-1-677-2204
Lero@TCD, Head of Foundations & Methods Research Group
Director of Teaching and Learning - Undergraduate,
School of Computer Science and Statistics,
Room G.39, O'Reilly Institute, Trinity College, University of Dublin
http://www.scss.tcd.ie/Andrew.Butterfield/
--------------------------------------------------------------------

[Petr Pudlák]

Hi,

I also support the idea of having Markdown for Haddock. Using some well established markup language would make Haddock much easier to adopt and use.

While I like the idea of allowing any markup language (let's say supported by Pandoc) and freedom it gives to developers, it also has also drawbacks: It makes contributing more difficult, if a project uses some wierd, non-standard markup language.

Concerning math expressions, what about using Markdown with MathJAX, like math.stackexchange.com does?

  Best regards,

  Petr Pudlak

2013/4/5 Andrew Butterfield <Andrew.Bu...@scss.tcd.ie>

[Rogan Creswick]

On Fri, Apr 5, 2013 at 10:12 AM, Petr Pudlák <petr...@gmail.com> wrote:

While I like the idea of allowing any markup language (let's say supported by Pandoc) and freedom it gives to developers, it also has also drawbacks: It makes contributing more difficult, if a project uses some wierd, non-standard markup language.

I feel pretty strongly about this; I don't think we gain much by allowing arbitrary formats.  I think the same argument for coding conventions applies here -- consistency is much more important for readability than flexibility.

--Rogan

[John Wiegley]

>>>>> Johan Tibell <johan....@gmail.com> writes:

> I suggest that we implement an alternative haddock syntax that's a superset
> of Markdown.

Definite +1 from me too.

--
John Wiegley
FP Complete Haskell tools, training and consulting
http://fpcomplete.com johnw on #haskell/irc.freenode.net

[dag.od...@gmail.com]

Personally I think Markdown sucks, although perhaps less than Haddock markup.

Still:

* No document meta data

* No code block meta data like language for syntax highlighting

* No tables

* No footnotes

* HTML fallback is insecure

* Confusing syntax (is it []() or ()[] for links?)

* Syntax that gets in the way (maybe I don't want *stars* emphasized)

* Above point leads to non-standard dialects like "GitHub Markdown" (no, GitHub doesn't use markdown)

* Not extensible, leading to even more non-standard hacks and work-arounds (GitHub Markdown, Pandoc Markdown, other Markdown libraries have their own incompatible extensions)

* Not well suited for web input (e.g. four-space indentation for code blocks), although not that important for Haddock

An important thing to note here is that no, Markdown has *not* won because no one is actually using *Markdown*.  They're using their own, custom and incompatible dialects.

Only two of the above points apply to reStructuredText (web input and syntax getting in the way), and those particular points don't apply to Creole.  Therefore I tend to advocate Creole for web applications and reStructuredText for documents.

[dag.od...@gmail.com]

I forgot the mention the craziness with the *significant trailing whitespace*.

[Simon Heath]

On Thu, Apr 4, 2013 at 11:22 PM, Kim-Ee Yeoh <k...@atamo.com> wrote:
> Could you say something about /why/ you make the suggestion? I, for
> one, would be happy to google and read links, but what's missing from
> that experience would be input from a fellow haskeller. In context. In
> real-time. On topic.

Pretty much the same reasons Richard O'Keefe has already said more
vigorously: Markdown is ambiguous, and thus difficult to write good parsers
for, and that makes it hard to write good text that does exactly what you
want without struggling and clawing at it and eventually just using some
dirty hack workaround you found on Google. Or changing your document's
contents to handle a formatting bug/misdesign.

People may say "well Markdown has won", in which case Markdown _has_ won.
But there are still plenty of good alternatives out there that are widely
used (RST is just my personal favorite). And there since there are good
alternatives out there which don't have these problems, why not use them
instead?

I didn't really want to start a markup-language-flame-war... except that I'm
glad that people are discussing it and thinking about these issues...
So maybe I did. :-(

Simon

[John MacFarlane]

I like markdown and use it all the time. While I acknowledge the
problems that have been pointed out, markdown has the advantage of being
easily readable "as it is" in the source document, and not looking like
markup.

But I do want to point out one problem with markdown as a format for
documentation in Haskell files. Consider:

----------------------------------------------------
module MyModule
{-
# Introduction

This is my module
-}
where
import System.Environment

main = getArgs >>= print
----------------------------------------------------

Now try to compile with -cpp, and you'll get an error because of the '#'
in column 1. '#' in column 1 is common in markdown (and even
indispensible for level 3+ headers).

One could work around this by disallowing level 3+ headers, by allowing
the headers to be indented, or by introducing new setext-like syntax for
level 3+ headers, but it is a problem for the idea of using a markdown
SUPERset.

John

+++ dag.od...@gmail.com [Apr 05 13 21:59 ]:

> I forgot the mention the craziness with the *significant trailing
> whitespace*.
>

> Ticket: [4]http://trac.haskell.org/haddock/ticket/244

> -- Johan
> _______________________________________________
> Haskell-Cafe mailing list

> [5]Haskel...@haskell.org
> [6]http://www.haskell.org/mailman/listinfo/haskell-cafe
>
> References
>
> 1. mailto:dag.od...@gmail.com
> 2. mailto:dag.od...@gmail.com
> 3. mailto:johan....@gmail.com
> 4. http://trac.haskell.org/haddock/ticket/244
> 5. mailto:Haskel...@haskell.org
> 6. http://www.haskell.org/mailman/listinfo/haskell-cafe

[Alexander Kjeldaas]

+1 for concistency.

Also, consider interop with non-haskell environments.  For example showing the documentation of a function in emacs, eclipse, on github, and from a javascript library.

All of these can be engineered around, and tooling can be provided.

But let me give an example: the other week I was looking for a command-line tool to extract javadoc to display as contextual information in emacs.  There is no such tool.  Javadoc is "java only".  For me, if I could not hack it up in an hour, it was too much work.  The solution was rather to craft a specific google search, use "I'm feeling lucky", and extract the subsection containing the documentation for the function.

Often the most useful format for documentation is contextual help in an IDE/editor, so don't forget that use-case.

Alexander

[Roman Cheplyaka]

Looks like a bug in cpphs to me (CC'ing Malcolm). It should respect
comments. E.g. GNU cpp strips C comments.

Roman

* John MacFarlane <j...@berkeley.edu> [2013-04-05 16:04:32-0700]

[Ivan Lazar Miljenovic]

On 8 April 2013 07:12, Roman Cheplyaka <ro...@ro-che.info> wrote:
> Looks like a bug in cpphs to me (CC'ing Malcolm). It should respect
> comments. E.g. GNU cpp strips C comments.

Not quite: http://hackage.haskell.org/trac/ghc/ticket/4836

--
Ivan Lazar Miljenovic
Ivan.Mi...@gmail.com
http://IvanMiljenovic.wordpress.com

[Conrad Parker]

On 6 April 2013 01:57, John Wiegley <jo...@fpcomplete.com> wrote:

>>>>> Johan Tibell <johan....@gmail.com> writes:

> I suggest that we implement an alternative haddock syntax that's a superset
> of Markdown.

Definite +1 from me too.

+1

Conrad. 

[Malcolm Wallace]

And cpphs strips C comments too. :-)

But seriously, John's use-case is the exact opposite of what you suggest. John wants to keep the # inside the comment block. You suggest to remove the comment-block altogether?

When I checked the example with cpphs, it turns out that the # line is retained, generating a warning but not an error. I expect Gnu cpp is the software that throws an error.

In my opinion, it is perfectly valid to have intentional preprocessor directives inside Haskell comments.

Regards,
Malcolm

[Niklas Hambüchen]

Could you elaborate a bit on which markdown features you support (or
even better: write it into your module haddocks)?

Thinks like
- autolink detection
- ```language blocks?

Also, you build on performance-oriented libraries - it would be cool if
you could make a small benchmark comparing with the standard
C/Python/Ruby parser implementations; AFAIK there is a standard Markdown
test suite that this could run against.

Concerning the project proposal:

I especially find the last feature useful for programming documentation,
and would love to have them in a potential haddock succesor. I was also
pleasantly surprised that pandoc seems to handle all of this (even with
code syntax highlighting).

On 05/04/13 02:10, Michael Snoyman wrote:
> In case it can be useful in any way for this project, my markdown
> package[1] is certainly available for scavenging, though we'd likely
> want to refactor it to not use conduit (I can't imagine conduit being a
> good dependency for Haddock).
>
> [1] http://hackage.haskell.org/package/markdown

[Roman Cheplyaka]

* Malcolm Wallace <malcolm...@me.com> [2013-04-08 10:00:11+0100]

> And cpphs strips C comments too. :-)
>
> But seriously, John's use-case is the exact opposite of what you
> suggest. John wants to keep the # inside the comment block. You
> suggest to remove the comment-block altogether?

No, my point was that the preprocessor has to be aware of the language's
comments, and treat them accordingly. In this case, "accordingly"
probably means leave them intact.

> When I checked the example with cpphs, it turns out that the # line is
> retained, generating a warning but not an error. I expect Gnu cpp is
> the software that throws an error.
>
> In my opinion, it is perfectly valid to have intentional preprocessor
> directives inside Haskell comments.

Could you give an example where this is useful?

#ifs and #ifdefs can be moved outside of the comment most of the time.
And macro expansions inside the comments are rather exotic.

On the other hand, the liberty to write whatever one wants inside a
comment feels important to me.

[Roman Cheplyaka]

* Ivan Lazar Miljenovic <ivan.mi...@gmail.com> [2013-04-08 10:18:32+1000]

> On 8 April 2013 07:12, Roman Cheplyaka <ro...@ro-che.info> wrote:
> > Looks like a bug in cpphs to me (CC'ing Malcolm). It should respect
> > comments. E.g. GNU cpp strips C comments.
>
> Not quite: http://hackage.haskell.org/trac/ghc/ticket/4836

This seems to be a different issue — about unlit, not cpp.

[John MacFarlane]

+++ John MacFarlane [Apr 05 13 16:04 ]:

> I like markdown and use it all the time. While I acknowledge the
> problems that have been pointed out, markdown has the advantage of being
> easily readable "as it is" in the source document, and not looking like
> markup.
>
> But I do want to point out one problem with markdown as a format for
> documentation in Haskell files. Consider:
>
> ----------------------------------------------------
> module MyModule
> {-
> # Introduction
>
> This is my module
> -}
> where
> import System.Environment
>
> main = getArgs >>= print
> ----------------------------------------------------
>
> Now try to compile with -cpp, and you'll get an error because of the '#'
> in column 1. '#' in column 1 is common in markdown (and even
> indispensible for level 3+ headers).
>
> One could work around this by disallowing level 3+ headers, by allowing
> the headers to be indented, or by introducing new setext-like syntax for
> level 3+ headers, but it is a problem for the idea of using a markdown
> SUPERset.
>
> John

Let me amplify my original comment with one more observation about
problems using markdown to comment Haskell code.

Markdown blockquotes start with '>' (usually in the leftmost column).
But this causes problems when your source document is bird-style
literate Haskell:

--------------------------------------
This is my literate Haskell module. As
someone said:

> literate Haskell is great!

Oops, that will be interpreted by GHC
as code, not comment.

> main = print $ reverse [1,2]
--------------------------------------

You can work around this by indenting the first '>' one space, which is
still valid Markdown, but it's a bit awkward. Obviously, we'd want any
Haddock markup successor to work in literate Haskell too.

reStructuredText has fewer potential conflicts and might be a more
sensible choice. But one would need to write a correct parser for
it. The pandoc parser doesn't cover 100% of rST, and differs in other
ways from the docutils parser (e.g. it allows markup inside links).
For full compatibility you'd probably want to copy the python module's
parsing algorithm exactly.

John

[Evan Laforge]

Can't we just add some features to haddock? There are a lot of ways
to improve haddock a lot, and no one is doing them, so my impression
is that haddock doesn't really have active maintainers. Adding a
whole new backend seems risky, unless it results in new maintainers
joining.

For my personal bikeshed contribution, I would like to see haddock
move in the way of fewer markup characters and rules, not more. Since
haddock is not "statically checked", the only way to find out if I put
in an error is to run haddock and then visually inspect the output,
unless of course it was a syntax error, in which case the error
message is often not very good. I can easily haddock individual files
since I have a custom build system, but I imagine cabal users would
have to haddock the entire project every time. I regularly see
haddock errors in released packages so I'm not the only one.

There are lots of ways to improve haddock a lot. For example, better
parse error messages. Make ""s smarter so they don't try to link
things that are obviously not modules. Or complain if it's not a
module. Or better, get rid of them entirely and use single quotes for
that. And make single quotes work for non-imported symbols.
Incremental support for cabal. Perhaps even deprecate @ and use ' for
that too.

One thing I think HTML got right is that there are only two characters
that need to be quoted. Of course that's at the cost of all the
markup being wordy, but the more you move in the markup-style DWIM the
more little rules you have to remember.

On Thu, Apr 4, 2013 at 6:00 PM, Johan Tibell <johan....@gmail.com> wrote:
>> Would it be too much to ask that a notation be used which has
>> a formal syntax and a formal semantics?
>
> We will document our superset, sure. That's what others did as well.
> The point is using Markdown as the shared base.
>

[Malcolm Wallace]

On 8 Apr 2013, at 14:52, Roman Cheplyaka wrote:

>> In my opinion, it is perfectly valid to have intentional preprocessor
>> directives inside Haskell comments.
>
> Could you give an example where this is useful?

> ... macro expansions inside the comments are rather exotic.

{- | Some module documentation.

#define WEBSITE http://some.really.rather.long/and/tedious/URL/that_I_dont_want_to_type_too_often
You can find more information about Foo at WEBSITE/Foo and Bar at WEBSITE/Bar
-}

As you say, the #define could equally live outside the comment, but I don't see why we should have an arbitrary restriction that it _must_ live outside the comment. As you also say, "the liberty to write whatever one wants inside a comment feels important", and if that includes the intentional use of CPP, why not?

Regards,
Malcolm

[MigMit]

Отправлено с iPad

08.04.2013, в 21:44, Evan Laforge <qdu...@gmail.com> написал(а):

> Can't we just add some features to haddock?

No, we can't. At the very least we should FIX haddock before adding features.

[Ivan Lazar Miljenovic]

On 9 April 2013 05:08, MigMit <migue...@yandex.ru> wrote:
>
>
> Отправлено с iPad
>
> 08.04.2013, в 21:44, Evan Laforge <qdu...@gmail.com> написал(а):
>
>> Can't we just add some features to haddock?
>
> No, we can't. At the very least we should FIX haddock before adding features.

How specifically does haddock need to be fixed?

--

Ivan Lazar Miljenovic
Ivan.Mi...@gmail.com
http://IvanMiljenovic.wordpress.com

_______________________________________________

[Evan Laforge]

On Mon, Apr 8, 2013 at 3:49 PM, Ivan Lazar Miljenovic
<ivan.mi...@gmail.com> wrote:
> On 9 April 2013 05:08, MigMit <migue...@yandex.ru> wrote:
>>
>>
>> Отправлено с iPad
>>
>> 08.04.2013, в 21:44, Evan Laforge <qdu...@gmail.com> написал(а):
>>
>>> Can't we just add some features to haddock?
>>
>> No, we can't. At the very least we should FIX haddock before adding features.
>
> How specifically does haddock need to be fixed?

Well, many of my specific suggestions could be considered fixes,
rather than features. E.g. better parse errors, smarter quotes.

[Michael Snoyman]

It supports ```language blocks, but not autolink detection. I have not fully documented which features are supported. I also haven't done any performance analysis versus other tools, simply because my goal is in no way high efficiency. It is fast enough for my use cases, and I don't intend to spend significant time optimizing unless a problematic level of inefficiency is discovered. If anyone else wants to put together benchmarks, I'll be happy to lend some guidance.

[David Waern]

2013/4/8 Evan Laforge <qdu...@gmail.com>

Can't we just add some features to haddock?  There are a lot of ways
to improve haddock a lot, and no one is doing them, so my impression
is that haddock doesn't really have active maintainers.  Adding a
whole new backend seems risky, unless it results in new maintainers
joining.

As one of the official maintainers (the other one is Simon Hengel), I agree that Haddock can barely be called actively maintained and it has been like that for a long time now. It's sad, but we lack of time, which is the usual excuse of course. A more resourceful maintainer would be great, so if someone reading this is motivated and wants to help out, we'd love to be contacted by you.

I don't think a SoC project for Markdown support would be risky at all (with a sufficiently good student), however. And it wouldn't require a new backend, the backend would still be the existing HTML backend in Haddock.

David

[Joe Nash]

I would be interested in discussing this project with a potential mentor if one happens to be reading. I'm a second year Computer Science student at the University of Nottingham, very interested in doing a haskell.org SoC project.

[Johan Tibell]

On Tue, Apr 9, 2013 at 12:40 PM, Joe Nash <joe...@blackvine.co.uk> wrote:
> I would be interested in discussing this project with a potential mentor if
> one happens to be reading. I'm a second year Computer Science student at the
> University of Nottingham, very interested in doing a haskell.org SoC
> project.

Normally I would volunteer to mentor any project I propose, but my
schedule doesn't allow for it this summer. Perhaps Mark Lentczner is
interested. He did some Haddock work before. I'm happy to provide
input on what I think the feature should be about (in fact, I will
probably write a blog post about it, like I do every year before
GSoC).

-- Johan

[Yitzchak Gale]

Johan Tibell wrote:
> I suggest that we implement an alternative haddock syntax that's a
> superset of Markdown.

Here is a previous thread on this exact topic, from five years ago:

http://www.haskell.org/pipermail/haskell-cafe/2008-February/039846.html

It mentions a few additional shades of bikeshed color
that have not yet been proposed in the current thread.

Enjoy,
Yitz

[Mateusz Kowalczyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Have you written the post? I'm interested in applying with this as a
project and it would be nice to be able to have a read beforehand.

- --
Mateusz K.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJRc/bJAAoJEM1mucMq2pqXzO4P/jET1Khe/H+rOWe7HVdGrLbq
x+fPETfhhFZGatRtsKm2HaO8CgLOAyZcV21x5gCRgdoNhAQb9cXeYKRRKEaQYIit
joiOGLtn5t45RYt+7d/C3/45CpjaukI2BJ819lLnFHNTebiGEi3UhANzdp6W2uJJ
K34tucRxChUL+Th7RzgyvkkgdiVbA8/arDhTCHLX1mQ1DQVm3T0y7M4GxzAc6OqV
bb15QA0tH/3/rKGnVc3y+po7EHs+oO5H4XKGvibbdNH0fz8M0XzxnK8oZ0SU2eYE
PyELmfI3z7jlKVhnOy+LPw+7+mMP2Ju/lFCWB4S+aotEjPFb4VszUCPwrvqxQsrB
nZ2Ekt76IBV58och/lPFG138C1pFwVG+HVE3JMRnWPC271meTiyMtMz9ZMEkDRVL
Hq9Djb7IUxgp0G8Z676U4jFH+PzmVVxXi39JIO05aTSxfrJAlYbpRUX5sizCiMv0
xi6osV8PQHurMXIm04CpRFcrqpdiMo9jpIpZyntBTEmJAiIZR7sTIYoqdaI0UmNl
bQGdr1fqm5EZm8Iz8x99BQjJrbGrL6UT0qTFmaWJ+o8HPu5zW+ekbWa7G8yMCiq+
tTnN4kH0uVZwwzG5tcM70RUg2bsg3bJSMgfE8Qf340d9lfALeMyPiGoQ+o5CDJ1l
hJXAjpXFaP8tWPyW5Kld
=syxC
-----END PGP SIGNATURE-----