Markdown, Djot and SILE - markdown.sile v1.4.0 release

[Didier Willis]

Greetings,

I am glad to announce that markdown.sile has just reached its 1.4.0 milestone (https://github.com/Omikhleia/markdown.sile)

This "anniversary release" occurs a year after the work was initiated and introduces cool new features, besides the usual bug fixes.

Using the Djot or Markdown lightweight markup to a generate great-looking PDF with SILE has never been easier.

After trying this collection, if you enjoy it as much as I do, please add a "Star" to its GitHub repository and spread the word. It will be appreciated!

Regards,

Didier

[Andreas Reuleaux]

Hi, and thanks for sharing (and sorry for this late feedback of mine - I
have taken a closer look only now):

And while I am personally not using markdown/sile or djot/sile at this
time (I am sticking to sile's .sil files for now):
I learned about djot (cool!) through a similar earlier release
message of yours some time ago.

The one thing that I am envious about though:
captioned-figures, and -tables, as defined in:
markdown.sile/packages/markdown/commands.lua

Maybe it would be possible to factor those out, and make them
available to normal/mortall sile-users (ones like me: sticking
to .sil / .lua / .fnl files for now - i.e. not using .md or .dj)?

This seems a pretty basic requirement for me (and thus overdue in sile): have not only
images/tables, but some captions with them.

Thanks, just my 2c.

.A

Didier Willis <didier...@gmail.com> writes:

> Greetings,
>

> I am glad to announce that *markdown.sile* has just reached its 1.4.0

> milestone (https://github.com/Omikhleia/markdown.sile)
> This "anniversary release" occurs a year after the work was initiated and
> introduces cool new features, besides the usual bug fixes.

> *Using the Djot or Markdown lightweight markup to a generate great-looking
> PDF with SILE has never been easier.*

[Didier Willis]

Thanks for the feedback, A !

Regarding your question:

The one thing that I am envious about though:
captioned-figures, and -tables, as defined in:
markdown.sile/packages/markdown/commands.lua

Maybe it would be possible to factor those out, and make them
available to normal/mortall sile-users (ones like me: sticking
to .sil / .lua / .fnl files for now - i.e. not using .md or .dj)?

Actually they are kind of already factored out, in more powerful versions, in the resilient.book class from my other collection, resilient.sile (with support for automated numbering, advanced styling, and other goodies).

The commands in  "packages/markdown/commands.lua" are, if you look at them, just wrappers using those provided by the document class if available, or implementing a much simpler and naive version as fallback if not available. In other terms, installing resilient.sile and using its resilient.book class, you'll get the fuller/better versions.

The degraded/limited fallbacks are just here so as to get something decent rather than nothing at all when using other classes not providing them (typically, the small default book class from the core SILE distribution) and I have no plan personally to extract them elsewhere and to maintain them outside of that context. I did not want to force people interested in using Djot or Markdown to also use my resilient classes, but honestly I don't consider the default SILE book class to be a sane choice for anything serious.

And then it works of course in SIL language (= .sil files) too, e.g.

\begin[class=resilient.book]{document}
Lorem ipsum...

\begin{captioned-figure}
\img[src=./temp/dreambook.png, width=50%lw]
\caption{Some caption}
\end{captioned-figure}

\end{document}

IMHO, Markdown and Djot are easier for non-specialists that the SIL TeX-like language, but all workflows are legit.

(And they are more portable, too.)

Regards,

Didier

[Didier Willis]

Oh, and an additional comment:

This seems a pretty basic requirement for me (and thus overdue in sile): have not only images/tables, but some captions with them.

I am not sure it is "long overdue", or that it is really the role of SILE's core distribution to cover such things.

My personal viewpoint is that SILE's core ought to be comparable to (plain) TeX from Knuth, i.e. responsible for providing the necessary low level blocks and algorithms,

Ceteris paribus - all other things being equal. The TeX ecosystem then has LaTeX from Lamport, ConTexT, etc. -- coming on top of the engine, that is, advanced frameworks from other minds, covering these kinds of things, along with lots of extensions distributed on the CTAN.

That possibly could be where 3rd-party modules for the SILE ecosystem actually stand - with distribution, in our case, on LuaRocks https://luarocks.org/m/sile. None of these things, despite looking "fairly basic" on the surface, are that easy and definitive (suffice to look, in the TeX world, say, at the many different table packages: LaTeX's default, but also booktabs, ctable, tabu...)

In other terms, this is also a call to potential package and class developers to come forward and enrich the ecosystem on LuaRocks ;)

Rgds,

Didier.

[Andreas Reuleaux]

OK, thanks a lot, and great! - I will have a closer look at
resilient.sile then -- (and thanks for your other additional comment as well).

.A


Didier Willis <didier...@gmail.com> writes:

> Thanks for the feedback, A !
>
> Regarding your question:
>
> The one thing that I am envious about though:
> captioned-figures, and -tables, as defined in:
> markdown.sile/packages/markdown/commands.lua
>
> Maybe it would be possible to factor those out, and make them
> available to normal/mortall sile-users (ones like me: sticking
> to .sil / .lua / .fnl files for now - i.e. not using .md or .dj)?
>
>
> Actually they are kind of already factored out, in more powerful versions,

> in the *resilient.book* class from my other collection, *resilient.sile*

> (with support for automated numbering, advanced styling, and other goodies).
>
> The commands in "packages/markdown/commands.lua" are, if you look at them,
> just wrappers using those provided by the document class if available, or
> implementing a much simpler and naive version as fallback if not available.

> In other terms, installing *resilient.sile* and using its resilient.book

[Freely-Given.org]

On 19/08/23 04:53, Didier Willis wrote:

My personal viewpoint is that SILE's core ought to be comparable to (plain) TeX from Knuth, i.e. responsible for providing the necessary low level blocks and algorithms,

Ceteris paribus - all other things being equal. The TeX ecosystem then has LaTeX from Lamport, ConTexT, etc. -- coming on top of the engine, that is, advanced frameworks from other minds, covering these kinds of things, along with lots of extensions distributed on the CTAN.

If that does turn out to be the official SILE philosophy, it seems like something that should be near the introduction in the manual to help us noobs understand how to best make use of the system? (I've just started working thru the manual again trying to get my head around it all and to try to understand your Bible typesetting example. Found no PPA for Ubuntu Lunar, but was able to get the Docker version running by adding sudo and making a few other adjustments to the alias.)

Rob.

[Didier Willis]

On August 19, 2023, Freely Given.org a écrit :

On 19/08/23 04:53, Didier Willis wrote:

My personal viewpoint is that SILE's core ought to be comparable to (plain) TeX from Knuth, i.e. responsible for providing the necessary low level blocks and algorithms,

Ceteris paribus - all other things being equal. The TeX ecosystem then has LaTeX from Lamport, ConTexT, etc. -- coming on top of the engine, that is, advanced frameworks from other minds, covering these kinds of things, along with lots of extensions distributed on the CTAN.

If that does turn out to be the official SILE philosophy, it seems like something that should be near the introduction in the manual to help us noobs understand how to best make use of the system?

As said, it's a personal viewpoint. I wouldn't make assumptions too soon :)

• The 3rd-party package management with LuaRocks is fairly new, it was introduced in 0.14.0 about one year ago. It's a game changer IMHO...
  
• We only re-vamped the Manual in 0.14.9 not that long ago, so a better introduction to 3rd-party package ecosystem now occurs early in the "Packages" chapter (§5.2), and there are also mentions of it earlier (§2.7)

It doesn't mean the Manual cannot be further improved, of course.

But on the other hand, one size cannot fit all. For instance, my personal opinion (again) is there's no point authoring books directly in the SIL language. Keeping the comparison with the TeX world, there are still people using plain TeX in the wild, and you wouldn't ask Knuth to rewrite his TeXbook with noobs in mind, with a gentle introduction to LaTeX and friends, would you? ;) 

There are whole books on the topic...

Again, all things equal, my resilient.sile collection of modules has it's own 100+ page manual...

(I've just started working thru the manual again trying to get my head around it all and to try to understand your Bible typesetting example. [...])

On the topic of my experimental Bible typesetting example, I still have to provide some instruction and example how to use it.

It's pretty straightforward once you see it in action, but heh, there are only 24 hours a day and wrapping up resilient.sile v2.1 was kind of an effort.

I'll push some working example with instructions as soon as I can.

Cheers,

Didier.

[Vadim Plessky]

суббота, 19 августа 2023 г. в 06:23:03 UTC+3, didier...@gmail.com:

On August 19, 2023, Freely Given.org a écrit :

On 19/08/23 04:53, Didier Willis wrote:

My personal viewpoint is that SILE's core ought to be comparable to (plain) TeX from Knuth, i.e. responsible for providing the necessary low level blocks and algorithms,

Ceteris paribus - all other things being equal. The TeX ecosystem then has LaTeX from Lamport, ConTexT, etc. -- coming on top of the engine, that is, advanced frameworks from other minds, covering these kinds of things, along with lots of extensions distributed on the CTAN.

But on the other hand, one size cannot fit all. For instance, my personal opinion (again) is there's no point authoring books directly in the SIL language. Keeping the comparison with the TeX world, there are still people using plain TeX in the wild, and you wouldn't ask Knuth to rewrite his TeXbook with noobs in mind, with a gentle introduction to LaTeX and friends, would you? ;) 

There are whole books on the topic...

Again, all things equal, my resilient.sile collection of modules has it's own 100+ page manual...

Hello Didier,

You have highlighted very good point, re. : "there's no point authoring books directly in the SIL language."

What would be your recommendation for authoring books/manual?

I am managing technical documentation for a complex IT subsystem, it's already 1000+ pages and growing.

Majority of docs was in MS Word format when I came to this company. And different documenta had different fonts, syles, and "unknown authors" (no way to find who wrote them and when).

I managed to align formats across different docs to almost similar styles, changes fonts and migrated docs to LibreOffice ODT.

But while LibreOffice has nice UI interface, its formatting ideology has in many aspects same issues as MS Word (for very known reason).
Reformatting 1000+ pages which may have "hidden formats" in text/paragraph is very time consuming and challenging.
I am going forward to do complete refactoring somewhere in the beginnging of 2025 and looking for a right set of tools to do all the tasks.

Best regards,
Vadim Plessky

[Didier Willis]

Hello Vadim

You have highlighted very good point, re. : "there's no point authoring books directly in the SIL language."
What would be your recommendation for authoring books/manual?

I am managing technical documentation for a complex IT subsystem, it's already 1000+ pages and growing.

Majority of docs was in MS Word format when I came to this company. And different documenta had different fonts, syles, and "unknown authors" (no way to find who wrote them and when).

I managed to align formats across different docs to almost similar styles, changes fonts and migrated docs to LibreOffice ODT.

But while LibreOffice has nice UI interface, its formatting ideology has in many aspects same issues as MS Word (for very known reason).
Reformatting 1000+ pages which may have "hidden formats" in text/paragraph is very time consuming and challenging.
I am going forward to do complete refactoring somewhere in the beginnging of 2025 and looking for a right set of tools to do all the tasks.

 

That's a tough question, especially with technical documentation in mind...

My point (if I remember correctly that old discussion) was that authoring content directly in SIL is not a good option (in my opinionated view at least.

One should rather pick up some standard markup (e.g. Markdown or Djot, or even DocBook if one wants it that way, etc.).

These are nearly always preferable than a custom low-level specific command language, as there are other tools supporting them for authoring, editing, converting to something else, etc.) and they offer higher-level abstractions with cleaner semantics. This is what a end-user should author content with, picking the right language (be it later converted to something tools such as SILE can cook to PDF, but that's a different issue).

I've dedicated a section or two to that topic of the "input language" in my documentation for my markdown.sile package, you might want to check it https://www.calameo.com/books/007349338f6a46a6b6fb5 - Chapter 2 and esp. §2.1 and 2.2. I'm mostly focused on novel-like books, literary essays, etc. there, however.

Now, technical documentation is another beast, generally speaking, unless the said documentation is simple enough that Markdown could be used...

So it really depends on expectations. Manufacturing companies, for instance, often have specific needs for assembling product documentation from several bits (depending on model variants, different legislation rules, available options, etc.).

There are specific markup languages for such things - notably DITA (which comes as XML originally, but also has Markdown variants), a very structured model used in the industry. It does have its own tooling. On the side of open source solutions, for instance, there are the  "official" DITA-OT toolkit and xmlmind's DITA toolkit (I once tried both, and largely preferred the latter in my own use case). Both use XSL-FO in their the print workflow, and for PDF production they then use either commercial XSL-FO processors, or the open source Apache FOP. (I once considered making a DITA renderer in SILE, but didn't pursue the idea -- Not only it's a niche domain (albeit a lively one), but FOP is not a bad tool in that case, where all the heavy-lifting of the input language is well done by the said toolkits. FOP is very legit in this case, whatever people think regarding XSL-FO in general -- See I am not necessarily considering SILE, TeX or whatever other typesetting system are suitable for every use case in the world! ;) )

That is to say, technical documentation is not an easy topic, depending on what one calls "technical" here and the general product workflow of a given company.

And if the documentation has lots of float images, eek.

Moreover with sources in DOC or ODT, one faces other challenges (hidden markup, as you noticed, redundant or non-semantic style changes, layers of font-switching cancelling each other, etc.). Sometimes Pandoc does a good job at converting them to a lighter format, sometimes it doesn't really deliver up to the expectations (e.g. mixes up with the markup, keeps it complex and idiosyncratic, or worse... loses information).

TL;DR. There is no straightforward answer to your question, I am afraid :rofl:

Regards,

Didier.