Query: What is Markdown? Which implementation do you use?

[TiddlyTweeter]

Recently there has been renewed discussion of "Markdown".

My problem with it is both ...

1 -- A useful approach to markup by Gruber;

2 -- And a huge complexity of variant implementations of Gruber's vision.

When people say TW "Markdown" is not full "Markdown" ...

A -- Which variant Markdown flavour are they using?

B -- Which features of (their used) Markdown flavour do they need in TW?

Properly clarifying this would help a lot finally solve this issue better.

It is likely only possible to solve the majority cases.

I have NO idea what they are.

TT

[Saq Imtiaz]

Since this is related, I'll mention here that the markdown plugin from the official library will try to parse wikitext, after markdown. So you can get the best of both worlds, markdown and macros/transcludes, or disable this feature.

This is what wikitext features are enabled by default:

https://github.com/Jermolene/TiddlyWiki5/blob/master/plugins/tiddlywiki/markdown/config_renderWikiTextPragma.tid

[HC Haase]

Since this is related, I'll mention here that the markdown plugin from the official library will try to parse wikitext, after markdown. So you can get the best of both worlds, markdown and macros/transcludes, or disable this feature.

That is very very nice. I would love to use markdown instead, but markdown sadly have another problem.; linking

markdown links will not get registered as backlinks in other tiddlers or in the count widget:(

[TiddlyTweeter]

Saq Imtiaz wrote:

 

Since this is related, I'll mention here that the markdown plugin from the official library will try to parse wikitext, after markdown. So you can get the best of both worlds, markdown and macros/transcludes, or disable this feature.

HC Hasse replied

That is very very nice. I would love to use markdown instead, but markdown sadly have another problem.; linking

markdown links will not get registered as backlinks in other tiddlers or in the count widget:(

Let us pretend that a Markdown Link could be converted to TW link early in the parse process. Could it become usable thereafter for subsequent ops like "backlinks"?

A thought

TT

[Saq Imtiaz]

Note that you can enable [[...]] style prettylinks by adding on to the end of that pragma.

I haven't checked, but would think they would then show up in backlinks. It's not markdown syntax, but seems that a lot of markdown users are used to this syntax too.

[TiddlyTweeter]

Saq Imtiaz wrote:

Note that you can enable [[...]] style prettylinks by adding on to the end of that pragma.

I haven't checked, but would think they would then show up in backlinks. It's not markdown syntax, but seems that a lot of markdown users are used to this syntax too.

 Regarding the OP this is a very useful comment.

The point with "Markdown" in TW is to try get a better handle on what "fits well enough" major Variant flavours.

I'm sure we can push it on. The problem is not so much TW, I think, its that "Markdown" is actually quite variant---especially when extended beyond The Gruber startup patterning.

TT

[PMario]

Hi,

There are several specifications, that define the text/markdown MIME type and the variants thereof. File extension is .md

text/markdown is specified at [RFC7763] The text/markdown Media Type since March 2016. It mentions markdown-variants in [chapter 10]

Some important variants are registered / described in [RFC7764] Guidance on Markdown: Design Philosophies, Stability Strategies, and Select Registrations

Common variants are: 

• MultiMarkdown
• GitHub-Flavored Markdown
• Pandoc
  
• Fountain (Fountain.io)
  
• CommonMark
  
• kramdown-rfc2629 (Markdown for RFCs)
  
• rfc7328 (Pandoc2rfc)
  
• PHP Markdown Extra

CommonMark is the newest spec and probably the most complete, mentioned on [page 14]

Depending on the library, that is used, it may be possible to select different variants. IMO it very much depends on the personal usecase, which setting makes sense.

TW default MD library (remarkable-js) supports: CommonMark + popular syntax extensions.

As Saq pointed out, the library creates it own AST, which makes it possible to pass unknown elements to the TW parser, _after_ parsing MD.

BUT there are some limitations. _Not_ every combination of markdown syntax and TW syntax may produce the expected output. It is meant to be used with "prose text" and a little bit of [[link]] magic. .... THAT's it!

If we want to have 100% compatibility we would need to implement our own MD variant into TW.

------------------------

There is a What if? thread, that I started 2017, that discusses the possibility to implement CommonMark as a subset of the TW syntax AND create a Markdown variant that defines the missing TW features.

So it would be possible that "TW syntax X" would be a valid Markdown Variant.

have fun!

mario

PS @Eric Shulman - Someone (not me) marked my thread as completed. Could you please remove it?

[TiddlyTweeter]

PMario, very useful post. Mainly because it gives much of the needed perspective on the Markdown Issues. Perspective is needed.

My OP partly came from frustration at folk posting "I need Markdown" with mainly never saying WHICH Markdown they are talking about.

Systems that are "Markdown" in practice are pretty diverse. It IS difficult to address the issue optimally without knowing more on specific usages or scope.

That was Point 1.

TT

[TiddlyTweeter]

PMario wrote:

Some important variants are registered / described in [RFC7764] Guidance on Markdown: Design Philosophies, Stability Strategies, and Select Registrations

Common variants are: 

• MultiMarkdown
• GitHub-Flavored Markdown
• Pandoc 
  
• Fountain (Fountain.io) 
  
• CommonMark 
  
• kramdown-rfc2629 (Markdown for RFCs) 
  
• rfc7328 (Pandoc2rfc) 
  
• PHP Markdown Extra

This is my Point 2 ...

Take the case of FOUNTAIN.IO. It is a really good illustration of the problem of getting specific. I'm familiar with Fountain style Markdown and a user of it in specialist software and via a crude hack parser I made for personal use in TW.

Why is it relevant to TW? It isn't. And that is the point. Let me show why.

Fountain is a very well designed markup system solely for screenplays. Its robustness is evident in that major commercial screenplay software now uses it as a standard import/export format. 

It integrates the basic Gruber Markdown, mainly for text emphasis. The rest of it is an interpretive parser that can understand the specific layout of screenplays very accurately.

Its workings are uber-Gruber in that most of the time it detects screenplay components without any explicit markup at all. The screenplay layout in plain text spacing IS its markup.

The point here is ... Fountain is an example of highly specialised extension incorporating Markdown in a seamless way but radically adding other markup components for IT's purpose.

The issue is this: When evaluating "A better Markdown for TW" what are we specifically talking about? It sure isn't Fountain. But WHAT is it?

Best wishes

TT

 

[Mat]

TiddlyTweeter wrote:

The issue is this: When evaluating "A better Markdown for TW" what are we specifically talking about? It sure isn't Fountain. But WHAT is it?

Answer:

For native TW: The markdown that most people are used to / expect (Hidden agenda: to ensure they are attracted to, and feel they can use, TW.)

For modified TW or "editions": Whatever fits the context. For a screenplay edition, maybe it's Fountain.

<:-) 

[PMario]

On Thursday, May 21, 2020 at 9:58:23 AM UTC+2, TiddlyTweeter wrote:

...

The issue is this: When evaluating "A better Markdown for TW" what are we specifically talking about? It sure isn't Fountain. But WHAT is it?

I think those TW rules should understand markdown

 Paragraphs in WikiText

 Lists in WikiText

 Linking in WikiText

 Images in WikiText

 Headings in WikiText

 Formatting in WikiText

 Code Blocks in WikiText

 HTML in WikiText

 Hard Linebreaks in WikiText

According to the CommonMark spec ... with some exceptions. So the MIME type may be text/markdown; variant=tiddlywiki

CM spec contains a lot of redundant rules and start / end markers. We need them for the important stuff like transclusions, macros ... and so on. see: https://tiddlywiki.com/#WikiText

-mario

[Mark Kerrigan]

Hello

My quick thought:

As a general rule, I find mixing Markdown and TiddlyWiki wikitext not the best idea. My main use case for Markdown is as an interoperable format. If I were to mix Markdown and wikitext, then I could no longer be certain the document I create will be interoperable if I put the text through a Markdown processor. Also curious how others are using Markdown in TW. Of course I'm fine using wikitext since I've been writing in TW for so long, but I can see how some people don't want to have to remember yet another syntax.

- Mark

[TiddlyTweeter]

Ciao Mat

Always love your enthuse!

Right. In sense of two audiences.

1 - Special needs (e.g. Fountain.io)

2 - Some kind of shared GENERIC, as is, minimal Markdown DOMINANT SUPER-SET.

It is 2 that is the issue of the OP. Does 2 exist?

PMario is very good on the issue.

But the final solution remains slippy.

TT

[TiddlyTweeter]

Mark Kerrigan wrote:

 

As a general rule, I find mixing Markdown and TiddlyWiki wikitext not the best idea.

 

My main use case for Markdown is as an interoperable format.

 

If I were to mix Markdown and wikitext, then I could no longer be certain the document I create will be interoperable

Ciao Mark

Spot-on. IF a new user is Markdown Happy then it makes MOST sense to directly support it natively, NO compromise. If you mix up TW WikiText with some variant of Markdown you likely end up with a Nasty SpongeBob that FUs decent portability.

The OP was really trying to tease out a non-TW issue. That demands for better parsing of "Markdown" are way too GENERIC.

 WHICH end-versions if Markdown are we needing to support? 

TT

[PMario]

On Friday, May 22, 2020 at 1:50:26 PM UTC+2, TiddlyTweeter wrote:

 WHICH end-versions if Markdown are we needing to support? 

Only 2

CommonMark and variant=tiddlywiki ... Yes the goal is to "rule the world".

If we have enough users and IF we register a variant, which probably is a tough undertaking in its own, others may support it, if their users demand it.

-mario