Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Discussing Extending the POD Spec

0 views
Skip to first unread message

Paul "LeoNerd" Evans

unread,
Sep 21, 2023, 1:45:04 PM9/21/23
to pod-p...@perl.org
Hi all,

I'm interested in opening some discussions on how we can further extend
the POD specification, adding a few new features.

Over the years there's been a couple of suggestions of features that
people want adding to POD been made on the perl5-porters@ list or other
places, that have gone back and forth and fizzled out mostly on the
observation that it isn't really clear who would be responsible for
updating/adding code to the various POD formatters or other consumers.
I'd like to get to the bottom of this sometime, as I feel there's a
steadily-growing list of "things that would be nice" that we don't have
in POD.

I don't want to turn this thread into a discussion on the features
themselves, but rather to work out *who* are the people to really get
into that chat, and what level of interest people have in actually
moving forward. I would like to be in a position where we can
confidently go about adding some new features and abilities to the
format where currently it is lacking.

There's ultimately a choice to be made - is POD still alive, as a
specification, a technology? Can we continue to add new features to it?
If not, then we'd have to look into how to replace it with something
else that can continue to support what we want, and I think that would
be a shame to be in such a position. I'd much prefer to extend what we
already have, than have to throw it all out and replace it with
something else entirely.

Let me know your thoughts,

--
Paul "LeoNerd" Evans

leo...@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

Russ Allbery

unread,
Sep 21, 2023, 2:15:05 PM9/21/23
to Paul "LeoNerd" Evans, pod-p...@perl.org
"Paul \"LeoNerd\" Evans" <leo...@leonerd.org.uk> writes:

> I don't want to turn this thread into a discussion on the features
> themselves, but rather to work out *who* are the people to really get
> into that chat, and what level of interest people have in actually
> moving forward.

I am interested. I probably support the most primitive output formats,
but I am still intending to actively support those formatters, and follow
discussions here. There are a few things that I've wanted to have in POD
for a while, although nothing that rose to the level of wanting to do the
work required to get it widely supported.

--
#!/usr/bin/perl -- Russ Allbery, Just Another Perl Hacker
$^=q;@!>~|{>krw>yn{u<$$<[~||<Juukn{=,<S~|}<Jwx}qn{<Yn{u<Qjltn{ > 0gFzD gD,
00Fz, 0,,( 0hF 0g)F/=, 0> "L$/GEIFewe{,$/ 0C$~> "@=,m,|,(e 0.), 01,pnn,y{
rw} >;,$0=q,$,,($_=$^)=~y,$/ C-~><@=\n\r,-~$:-u/ #y,d,s,(\$.),$1,gee,print

Paul "LeoNerd" Evans

unread,
Sep 21, 2023, 7:45:04 PM9/21/23
to Russ Allbery, pod-p...@perl.org
On Thu, 21 Sep 2023 10:57:17 -0700
Russ Allbery <r...@cpan.org> wrote:

> There are a few things that I've wanted to have in POD
> for a while, although nothing that rose to the level of wanting to do
> the work required to get it widely supported.

Ah curious - I'd be happy to hear ideas.

My overall feeling is that individual little updates may not seem
worthwhile on their own, but combining all the efforts together may
result in something that feels like it had enough weight to justify a
round of updates to code here and there.

In quick summary, my thoughts are:

* underline
* code blocks with language names
* tables
* and we shouldn't forget about images



U<underline> - feels pretty simple and self-explanatory. Totally
stolen from Pod6 - L<https://docs.raku.org/language/pod#Underlined>


=code perl

# This is a code fence like Markdown's ``` marker. It supports
# specifying the language, so maybe highlighters can know about it,
# but otherwise behaves like a regular verbatim paragraph
my $code = "We know this is definitely Perl now";

=end

=code

This is a simple verbatim paragraph that is definitely *not* Perl, so
highlighters should not attempt to highlight any of these words as if
they were "perl code", because they are not.

=end

=code python

"""Other languages could be specified too and highlighters would then
handle them appropriately."""
import os
import sys

=end


=begin table

Inspiration | Pod6's L<https://docs.raku.org/language/tables>
--------------------------------------------------------------
Additionally | Markdown has something very similar to this.

=end


And probably we should think something about images but that one is
more complex because it's simply an external filename reference and
raises *many* questions about where those files are hosted and how
they're served over http or whatever other mechanism is being used to
view the file.

Dmitry Karasik

unread,
Sep 22, 2023, 3:45:04 AM9/22/23
to pod-p...@perl.org
> I don't want to turn this thread into a discussion on the features
> themselves, but rather to work out *who* are the people to really get
> into that chat, and what level of interest people have in actually
> moving forward. I would like to be in a position where we can
> confidently go about adding some new features and abilities to the
> format where currently it is lacking.

I once tried to arrange a consensus on images in pod, and after what I thought
was indeed a consensus, I invested lots of time into drafting and the
implementation. However on the later stage some people behind the github
aliases I can't recognize, and who never participated in the discussion, vetoed
the whole thing. That was not nice and failrly frustrating. So I agree the
process is as clear as mud. If the community can resolve this social problem,
there are old threads about the images in pod and I believe many have still
have interest in this.

--
Sincerely,
Dmitry Karasik

Paul "LeoNerd" Evans

unread,
Sep 22, 2023, 7:45:05 AM9/22/23
to Dmitry Karasik, pod-p...@perl.org
On Fri, 22 Sep 2023 09:32:35 +0200
Dmitry Karasik <dmi...@karasik.eu.org> wrote:

> So I agree the process is as clear as
> mud. If the community can resolve this social problem, there are old
> threads about the images in pod and I believe many have still have
> interest in this.

It sounds, therefore, like there's a lack of process at all. It isn't
clear to me whether new additions should happen first in the actual
spec, and we expect implementors to support them, or should actual
implementations just start adding behaviours and we hope to pull them
together into a coherent standard at the end?

Since there is one specification document - being pod/perlpod.pod in
perl core itself - that would feel to be a good place to begin the
process.

Maybe we should begin by raising a PR to change that file, and hope to
get some review comment and general acceptance from people on the
implementation side before proceeding.

Would that work as a process?

Paul "LeoNerd" Evans

unread,
Sep 22, 2023, 8:15:04 AM9/22/23
to Mark Overmeer, pod-p...@perl.org
On Fri, 22 Sep 2023 11:02:44 +0200
Mark Overmeer <ma...@overmeer.net> wrote:

> Just as Dmitry Karasik experienced, I encountered blockades by core
> maintainers to even think about improvements...

<with my PSC hat on> Hi. I'm here.

> My biggest complaints:
> - Pod uses visual markup, like =item, where it should use
> semantic markup, like =sub
> - Pod does not support anything related to Object Oriented features,
> like documenting inheritance relationships

The formatting-verses-semantic issue is a large one, yes. I could
imagine a way to solve it somewhat similar to how HTML-vs-CSS does
things, or the manpage macros in nroff, or if you stare at it weirdly,
the Scheme r7rs-small vs r7rs-large difference.

That would be, that Pod itself would have two layers to it. There needs
to be a few extra bits and pieces of purely visual style (e.g. my
underline idea), but those seem relatively simple and uncontentious to
be added.

But next up, you'd want to define a big set of "semantic directives",
which could be expanded out by some sort of translation layer by
application of a template or somesuch, into being the visual layer
instead. Quite what form that expansion layer takes - whether it is a
piece of code somewhere, or a template file in some format - would need
further looking at. But the overall idea is that it solves the same
kind of problem as r7rs small-vs-large split: Document authors can
write using the full set of semantic directives if they wish, while
implementors only have to care about building the visual formatting
stuff, knowing that the templating translation bit in the middle will
do its thing.

Such a mechanism would need to be easily extensible of course. Whenever
I look at someone's attempt to define a list of what semantic things
they'd want to document I always find that list comes up short. Did you
think about events? Signals? Mutable accessors as separate from mere
method calls? Did you list the standard config file keys your class
looks at? Does your UI widget define its stylesheet parameter names and
default keybindings? The list goes on. The moment you start to let
people define semantic things as real entities aside from simply being
a formatted document of text, people will want to come up with their
own new categories of thing to write about. You must allow them to do
so.

> - Pod limits the quality of the manpage: support for more
> fine-grained links, for instance, could produce better HTML
> accessibility.

Yes; the existing L<> is a little disappointing in that respect.

> So, back in 2003, I developed OODoc (OpenOffice did not exist yet).
...
> Eh, but it's my 2003 view of the World. It needs some (mainly visual)
> cleanup.

Mmm; it seems somewhat similar to Pod6 there.

> > [...], than have to throw it all out and replace it with
> > something else entirely.
>
> The distance between what POD is, and what a convenient documentation
> syntax should be, is really big.

Perhaps, perhaps not. It depends how you look at it.

I'm coming at this from having written App::sdview ("Structured
Document Viewer"), which is intended as a nicer viewer for manpages and
similar. It reads nroff, POD, Markdown, renders to a terminal, and can
output in those formats plus also HTML. What I've found is that trying
to cross-convert between nroff/POD/Markdown is largely
straight-forward on the common overlap. The reason I want to add things
like underline and code fences with language names to POD is that those
are relatively small easy things it's missing from some of the others.
But largely it fills a similar goal doing similar things.

Certainly if you compare POD to something like Javadoc, the differences
become much greater. Javadoc is intended verymuch for attaching
documentation to language-specific things. Nobody would use Javadoc for
writing a generic document that was unattached to a specific piece of
code; whereas look at all the .pod files sitting around in core perl
source that are unattached to code - be it Perl or C code.

> Perl-Core developers have shown to refuse even small useful changes
> in the past.
>
> I wrote a first version of pod6 for Perl6, which Damian transformed
> into the existing pod for Raku. We may finally get an acceptable
> quality pod syntax in Perl5, if we integrate that Raku spec into
> Perl5. That might finally convince some Core developers to accept
> dire needed improvements.

Yes I've been reading the Pod6 docs, and there's a few bits I want to
steal - U<underline> and tables being two. But overall I still feel
it's trying to edge its way much more in the direction of e.g. Javadoc
and become something very tightly integrated to the surrounding Raku
code, and in doing so ends up forcing a lot more rigid structure on
authors. I ultimately don't know whether that is a good thing or not.

Dmitry Karasik

unread,
Sep 22, 2023, 9:15:04 AM9/22/23
to Paul "LeoNerd" Evans, pod-p...@perl.org
> Since there is one specification document - being pod/perlpod.pod in
> perl core itself - that would feel to be a good place to begin the
> process.
>
> Maybe we should begin by raising a PR to change that file, and hope to
> get some review comment and general acceptance from people on the
> implementation side before proceeding.
>
> Would that work as a process?

If that is a question to me personally I'm flexible and everything is good as I
believe that the process is for people, not the vice versa. However, I believe
that I'm the wrong person to ask about this. I would stick to your original
proposition about the social aspect, how to make sure that the PR will get
attention of the people with the commit flag, will not get ignored, and the
discussion of a new feature won't result an a yet another stonewalling. And
also how to make sure that if there is an agreement that the feature, the PR
merging _will_ follow and will not be "forgotten".

As for the technicalities, part of my PR was a separate Image.pod dedicated
just for this aspect, so that perlpod.pod could just refer to that file, if
anything.

--
Sincerely,
Dmitry Karasik

Paul "LeoNerd" Evans

unread,
Sep 22, 2023, 11:15:06 AM9/22/23
to Dmitry Karasik, pod-p...@perl.org
On Fri, 22 Sep 2023 12:36:11 +0100
"Paul \"LeoNerd\" Evans" <leo...@leonerd.org.uk> wrote:

> It sounds, therefore, like there's a lack of process at all. It isn't
> clear to me whether new additions should happen first in the actual
> spec, and we expect implementors to support them, or should actual
> implementations just start adding behaviours and we hope to pull them
> together into a coherent standard at the end?
>
> Since there is one specification document - being pod/perlpod.pod in
> perl core itself - that would feel to be a good place to begin the
> process.
>
> Maybe we should begin by raising a PR to change that file, and hope to
> get some review comment and general acceptance from people on the
> implementation side before proceeding.

Well, here goes... A PR to add U<text> for underline.

https://github.com/Perl/perl5/pull/21509

Russ Allbery

unread,
Sep 22, 2023, 11:45:04 AM9/22/23
to Paul "LeoNerd" Evans, pod-p...@perl.org
"Paul \"LeoNerd\" Evans" <leo...@leonerd.org.uk> writes:
> Russ Allbery <r...@cpan.org> wrote:

>> There are a few things that I've wanted to have in POD
>> for a while, although nothing that rose to the level of wanting to do
>> the work required to get it widely supported.

> Ah curious - I'd be happy to hear ideas.

One of the things I've wanted from nearly the start was some way of
marking up metasyntactic variables. POD is full of things like:

perl <file>

or:

push ARRAY,LIST

or the like, using various conventions mostly done via plain text, but
with proper markup they could get different formatting, different colors
in Pod::Text::Color, etc. Right now, some places use italics, some use
all caps, some use angle brackets, some don't mark them at all, etc.

The hardest thing about maintaining Pod::Man is that there are a bunch of
things that it is supposed to do to help not require excessive markup of
Perl code, including complicated rules about which C<> blocks get quotes
around them, but those rules all assume Perl. Right now, people have to
pass in flags to pod2man to disable the guesswork that doesn't apply to
their documentation. I'm not sure how this could be fixed with better
semantic markup without clusttering POD with a ton of markup, but I feel
like it should be possible.

Also, I know most folks here know this, but a reminder that people use POD
for a ton of things that are entirely unrelated to Perl, in part because
it's one of the easiest ways to write man pages without having to learn an
obscure formatting language and without having to write a ton of
impossibly verbose semantic markup.

Tables are an obvious thing to add and presumably can be implemented
natively in *roff although I have never used tbl and it might be a fair
bit of work. Implementing them for Pod::Text sounds really hard, but
maybe it won't be as bad as I'm fearing or someone else will do it for me.

I would like there to be a POD command for indicating the license of the
POD file itself so that the license text could be included in output
formats that support embedded comments, such as *roff and HTML. Not sure
what to do with the license in Pod::Text, but I think this would be better
than putting the license of the POD file in the user-visible text like
everyone does now. Really, that should be the license of the thing the
POD file is documenting, since that's what the user is reading about, and
that's often not the same as the license of the POD.

There was a fairly fleshed out proposal a while back for footnote support
that never went anywhere.

Paul "LeoNerd" Evans

unread,
Sep 22, 2023, 12:30:05 PM9/22/23
to Russ Allbery, pod-p...@perl.org
On Fri, 22 Sep 2023 08:29:41 -0700
Russ Allbery <r...@cpan.org> wrote:

> "Paul \"LeoNerd\" Evans" <leo...@leonerd.org.uk> writes:
> > Ah curious - I'd be happy to hear ideas.
>
> One of the things I've wanted from nearly the start was some way of
> marking up metasyntactic variables. POD is full of things like:
>
> perl <file>
>
> or:
>
> push ARRAY,LIST
>
> or the like, using various conventions mostly done via plain text, but
> with proper markup they could get different formatting, different
> colors in Pod::Text::Color, etc. Right now, some places use italics,
> some use all caps, some use angle brackets, some don't mark them at
> all, etc.

I'm not quite sure I get what you mean here.

> The hardest thing about maintaining Pod::Man is that there are a
> bunch of things that it is supposed to do to help not require
> excessive markup of Perl code, including complicated rules about
> which C<> blocks get quotes around them, but those rules all assume
> Perl. Right now, people have to pass in flags to pod2man to disable
> the guesswork that doesn't apply to their documentation. I'm not
> sure how this could be fixed with better semantic markup without
> clusttering POD with a ton of markup, but I feel like it should be
> possible.

Hmmm - I wonder if there'd be merit in adding another code - perhaps
T<text> inspired by HTML's `<tt>`, to suggest "here is fixedwidth but
don't presume it to be Perl". Much like my related thoughts about
`=code perl` vs `=code` or whatever...

> Also, I know most folks here know this, but a reminder that people
> use POD for a ton of things that are entirely unrelated to Perl, in
> part because it's one of the easiest ways to write man pages without
> having to learn an obscure formatting language and without having to
> write a ton of impossibly verbose semantic markup.

Yes indeed - I use it all the time for just such a purpose.

> Tables are an obvious thing to add and presumably can be implemented
> natively in *roff although I have never used tbl and it might be a
> fair bit of work. Implementing them for Pod::Text sounds really
> hard, but maybe it won't be as bad as I'm fearing or someone else
> will do it for me.

Youknow, we recently added Term::Table to perl core as a dependency of
Test2::Suite; but it would make easy work of rendering tables out of
POD as well.

> I would like there to be a POD command for indicating the license of
> the POD file itself so that the license text could be included in
> output formats that support embedded comments, such as *roff and
> HTML. Not sure what to do with the license in Pod::Text, but I think
> this would be better than putting the license of the POD file in the
> user-visible text like everyone does now. Really, that should be the
> license of the thing the POD file is documenting, since that's what
> the user is reading about, and that's often not the same as the
> license of the POD.

I could imagine a few things on that note. Maybe

=meta licence GFDL

=meta copyright Author Name, 2023

Gives an easy place to hang on other things that might be useful for
things like POD->HTML or POD->nroff conversion too, e.g.

=meta title Here Is My Lovely Document

> There was a fairly fleshed out proposal a while back for footnote
> support that never went anywhere.

Ahyes - footnotes. I had forgotten about them. They'll be fun for me to
implement in App::sdview::Output::Tickit, and then think about what POD
formatting for them might look like. A task for future-me I feel...

Russ Allbery

unread,
Sep 22, 2023, 12:45:04 PM9/22/23
to Paul "LeoNerd" Evans, pod-p...@perl.org
"Paul \"LeoNerd\" Evans" <leo...@leonerd.org.uk> writes:
> Russ Allbery <r...@cpan.org> wrote:

>> One of the things I've wanted from nearly the start was some way of
>> marking up metasyntactic variables. POD is full of things like:
>>
>> perl <file>
>>
>> or:
>>
>> push ARRAY,LIST
>>
>> or the like, using various conventions mostly done via plain text, but
>> with proper markup they could get different formatting, different
>> colors in Pod::Text::Color, etc. Right now, some places use italics,
>> some use all caps, some use angle brackets, some don't mark them at
>> all, etc.

> I'm not quite sure I get what you mean here.

=item new(ARGS)

Create a new Pod::Man object. ARGS should be a list of key/value
pairs, where the keys are chosen from the following.

ARGS has no markup because no markup really fits, but it should be marked
up because it's not regular text. I want to be able to write:

=item new(M<args>)

Create a new Pod::Man object. M<args> should be a list of key/value
pairs, where the keys are chosen from the following.

and then the formatter can decide whether to use all-caps, a different
font, a different style, or whatever. Similarly for common man page
synopsis cases:

pod2man [B<--center>=I<string>] [B<--date>=I<string>]

should be:

pod2man [B<--center>=M<string>] [B<--date>=M<string>]

The harder problem is how to handle the sort of sample commands that are
currently set off in a verbatim block, where we can't use inline markup,
such as:

This adds:

.mso <language>.tmac
.hla <language>

to the start of the file, which configure correct line breaking for
the specified language.

That should ideally be M<language>, but obviously that doesn't work there.
Sphinx has special syntax for this in the :file: and :samp: roles, but
that wouldn't work with POD verbatim blocks.

Ron Savage

unread,
Sep 23, 2023, 11:45:04 PM9/23/23
to pod-p...@perl.org
Hi All

While you make many valid points, there is a bit of a lack of a broader
vision, crippled perhaps by an over-commitment to text mode.

This problem has been solved with the advent of the TiddlyWiki, which
use WikiText.

They use CamelCase a lot. A CamelCase word is automatically linked
within a single TiddlyWiki by the inbuilt editor to another point within
the text. Hence the MainMenu I use in every TiddlyWiki uses CamelCase.

And yes, they are best viewed with a web client (browser).

See tiddlywiki.com. Downloads are free.

Search there for WikiText for more. The homepage is of course a
TiddlyWiki.

Note also the broad ecosystem surrounding TiddlyWikis.

Here is one of mine: http://savage.net.au/misc/symbolic.language.html

It's a download accompanying a TV script I'm preparing to submit to ABC
TV here in Melbourne Victoria.

Click on the MainMenu item TiddlyWikis for more about the TiddlyWiki
concept, especially about the fact that the inbuilt edit has a little
command language and can generate HTML for you.

I keep a directory full of these, one per project.

---
Cheers
Ron
savage.net.au
0 new messages