[svn:perl6-synopsis] r14421 - doc/trunk/design/syn
la...@cvs.perl.org
Date: Wed Jun 13 13:55:24 2007
New Revision: 14421
Modified:
doc/trunk/design/syn/S02.pod
Log:
Block comments should not hide POD, pointed out by TheDamian++
Modified: doc/trunk/design/syn/S02.pod
==============================================================================
--- doc/trunk/design/syn/S02.pod (original)
+++ doc/trunk/design/syn/S02.pod Wed Jun 13 13:55:24 2007
@@ -14,7 +14,7 @@
Date: 10 Aug 2004
Last Modified: 13 Jun 2007
Number: 2
- Version: 111
+ Version: 112
This document summarizes Apocalypse 2, which covers small-scale
lexical items and typological issues. (These Synopses also contain
@@ -169,10 +169,11 @@
The entire final line counts as part of the comment. It does
not matter whether the intervening lines start with C<#> or not.
-Block comments may be nested within other block comments (with
-the same or differing brackets) but will ignore any other comment
-mechanism including POD, so this mechanism may be used to hide POD
-even from the pod parser, along with any associated code. The parser
+Block comments may be nested within other block comments (with the
+same or differing brackets). POD comments may also be nested within
+block comments. (These are still visible to the POD parser; if you
+wish to comment out a block of mixed POD and Perl 6 code, either use a
+POD comment around it all, or prefix every line with C<#>.) The parser
must report mismatched openers or closers to prevent, for example,
unwitting use of bare C<}> as a closer for an opening C<#{>.
@@ -192,11 +193,6 @@
in the face of strings containing bracket characters, and the probability
of such bracket skew increases with the length of the commented code.
-The POD parser must also recognize these comments in order to ignore
-them. For instance, it could treat C<#[[> and C<#]]> as a shorthand for
-C<=begin comment_2square> and C<=end comment_2square>. (The C<2> indicates
-degree of repetition, as described below.)
-
=item *
For all quoting constructs that use user-selected brackets, you can open
Jonathan Lang
> +Block comments may be nested within other block comments (with the
> +same or differing brackets). POD comments may also be nested within
> +block comments. (These are still visible to the POD parser; if you
> +wish to comment out a block of mixed POD and Perl 6 code, either use a
> +POD comment around it all, or prefix every line with C<#>.) The parser
> must report mismatched openers or closers to prevent, for example,
> unwitting use of bare C<}> as a closer for an opening C<#{>.
...or put it all in a block comment, and prefix POD's block tags with C<#>.
What's the rationale for keeping POD comments that are nested in block
comments visible to the POD parser? It seems to me that the least
surprising behavior would be for the POD comment to be swallowed up by
the block comment, on the basis that a block comment ought to be
equivalent to prefixing every line from its start to end with a C<# >.
Conversely, I'd expect a block comment that's nested inside a POD
comment to be passed to the POD parser as ordinary POD text. In
short, whichever kind of comment comes first should dominate.
--
Jonathan "Dataweaver" Lang
Damian Conway
> What's the rationale for keeping POD comments that are nested in block
> comments visible to the POD parser?
The rationale is that Perl 6 and Pod 6 have been designed to be completely
independent and uncoupled. That way, you can look at a piece of Pod without
worrying about any syntactic or semantic effects from the surrounding Perl
code...and look at a piece of Perl code without worrying about any syntactic
or semantic effects from the surrounding Pod. We think that's highly valuable.
The separation of Perl and Pod also makes parsing each language considerably
easier. As the two languages are now specified, the Perl parser doesn't have
to worry about embedded Pod, since it can be removed in a preprocessor stage,
or easily handled as part of the parser (since the Pod syntax is simple and
distinctive and morally equivalent to whitespace). Likewise the Pod parser
doesn't have to worry about Perl, since no Perl construct can affect the
meaning of embedded Pod. This second point is by far the more important, since
it means that the (comparatively simple) Pod parser doesn't have to have a
full implementation of a (far more complex) Perl parser within it.
The separation also means that you can--potentially, at least--use Pod to
document source in *any* language (i.e. not just Perl) provided that language
allows for a preprocessor to strip the Pod from any source before compilation.
And because Pod ignores "ambient" source code, the standard Pod parsing and
transformation tools will be able to be used, regardless of the interleaved
language.
> It seems to me that the least surprising behavior would be for the POD
> comment to be swallowed up by the block comment, on the basis that a
> block comment ought to be equivalent to prefixing every line from its
> start to end with a C<# >. Conversely, I'd expect a block comment
> that's nested inside a POD comment to be passed to the POD parser as
> ordinary POD text. In short, whichever kind of comment comes first
> should dominate.
"Least surprise" always depends on your underlying model of a phenomenon.
My underlying model is that documentation and the source it's documenting
should be entirely orthogonal. So, to me, it would be very surprising if a
programming construct (block comments) interacted with documentation. Or vice
versa.
To put it another way:
* At the Perl 6 level, all Pod is syntactically and semantically
invisible and therefore already equivalent to a comment.
So there's no need for Perl 6 comments to interact with Pod
(since commenting out a comment is a null operation, as far as
Perl is concerned).
* At the Pod 6 level, all Perl code is merely meaningless ambient
background noise and therefore already equivalent to whitespace.
So there's no need for Pod comments to interact with Perl 6 (since
commenting out whitespace is a null operation, as far as Pod is
concerned).
For all of the above reasons, I'm strongly convinced that the total separation
of Church (i.e. documentation of belief) and State (i.e. specification of
action) is as a good thing for programming languages as it is for nations. :-)
Damian
Mark Overmeer
> My underlying model is that documentation and the source it's documenting
> should be entirely orthogonal. So, to me, it would be very surprising if a
> programming construct (block comments) interacted with documentation. Or
> vice versa.
>
> invisible and therefore already equivalent to a comment.
>
> background noise and therefore already equivalent to whitespace.
We had a private discussion about this already three years ago: I
fully disagree! If the code and the documentation are not entangled,
why do you want to put them in the same file? Why do you put them in
the same distribution even?
No, the documentation is all about the code. The docs present everything
what the user should know about the code. The docs are the user's view
of the implementation, and the code is the computer's view on the same.
Realizing that, there are a few problems:
- is everything in the code documented?
- is everything documented correctly?
- during development, the interfaces change. Do we update the docs?
Getting people to document (and maintain the documentation) is not easy.
For some people, it's just not in their system. So, if you can make it
easier to do, if you can avoid the need for duplication (write it in
code, write the same again in the docs), if you can save some typing,
you probably end-up with better code.
Last week, I restructured the code of MailTools... published in 1995,
one of the oldest modules around. Also one of the most used modules
and described in many books. Changing the manual-page organization from
"at-end-of-file" to "interleaved" put the docs close to the respective
code. I found many mistakes in the docs, during this process: methods
which were not documented, methods which were not implemented, parameters
which were not or wrongly described...
As some people know, I wrote OODoc which extends the syntax of POD(5)
with logical markup and some simple avoidance of replication. For the
MailBox distribution (which is quite large), that saved me typing of
700.000 characters (35%)! Do you know how long it takes to type those?
So how much time did I gain to spend on code quality?
I had suggested syntax like this, in Perl6:
class Mail::Message {
`base class for message types
.has $msgid;
`The (internet wide) unique string which identifies this
`message object. May be undef as long as the message is
`begin composed.
.has $content_type = 'text/plain';
}
You see that there is little room for errors in the documentation.
Before people start flame-wars: the syntax is just to start a
discussion, to show how simple it could be. I am not stuck on that.
This can be automatically translated into (traditional POD like this)
(leaving out some blank lines for shortness sake)
=head1 NAME
Mail::Mesage - base class for message types
=head1 INHERITANCE
Mail::Message
isa Object
Mail::Message
is extended by Mail::Box::Message
=head1 METHODS
=head2 Attributes
=over 4
=item C<$msgid>
The (internet wide) unique string which identifies this
message object. May be undef as long as the message is
begin composed.
=item C<$content_type> (default 'text/plain')
=back
Damian, can you show how you would document the same code in POD6
syntax to get a comparible short man-page?
IMO: code and docs are two representations on one thinking processes,
named "programming". They are the opposit not orthogonal: parallel
developments with a little shifted focus.
The separation between State and Church is only about power: that the
Pope can't tell the President how to rule a country. But the people
need to merge their religious believes with their social duties.
Are you designing for "The Power" or "The People"?
--
Regards,
MarkOv
------------------------------------------------------------------------
Mark Overmeer MSc MARKOV Solutions
Ma...@Overmeer.net solu...@overmeer.net
http://Mark.Overmeer.net http://solutions.overmeer.net
Moritz Lenz
Mark Overmeer wrote:
> We had a private discussion about this already three years ago: I
> fully disagree! If the code and the documentation are not entangled,
> why do you want to put them in the same file? Why do you put them in
> the same distribution even?
>
> No, the documentation is all about the code. The docs present everything
> what the user should know about the code. The docs are the user's view
> of the implementation, and the code is the computer's view on the same.
I agree.
And while writing a class in Perl 6 the other day I noticed that copied
& pasted the signature of method to the pod:
=begin pod
=head3 C<method from_string(Str $s);>
initialize the Sudoku from a string C<$s>, with a 0 denoting an empty cell
and a number between 1 and 9 a clue.
Note that there is currently no way to use this function for sizes bigger
than 9x9 overall length.
=end pod
method from_string(Str $s){
# implementation of that method here
}
Since method signatures are very expressive in Perl 6, there should be a
way of accessing them in the POD without copy & paste. If you don't
think that's necessary: try it out for yourself. Write a class and
document it properly. I'm sure you'll end up doing the same as I did.
ATM I don't know that should be implemented, but perhaps somebody else
can think of a good way.
--
Moritz Lenz
http://moritz.faui2k3.org/ | http://perl-6.de/
Jonathan Lang
time being. Instead, a couple of thoughts:
1. It has always been my understanding that comments intimately relate
to documentation, despite being part of Perl's syntax. As such, they
are a technical violation of this "separation of Church and State",
being State-sponsored religion as it were. (Who'd have thought that
we'd be talking politics and religion on a programming language
mailing list? Egad!) Not that I'm advocating the removal of
comments; their sheer practical value makes up for their heathen ways.
2. Getting block comments to hide POD blocks wouldn't require the POD
parser to have a full implementation of a Perl parser. It would
require the POD parser to have a _limited_ implementation of a Perl
parser, one that's capable of identifying block comments. And IIRC,
you already have to do something like this with respect to block
quotes:
say :to(END);
=begin POD
blah blah blah
=end POD
END
If I understand matters correctly, the "POD code" in the above example
isn't POD code at all, but rather the content of a multi-line quote.
So extending the POD parser's awareness of Perl syntax to handle block
comments as well isn't much of a stretch.
--
Jonathan "Dataweaver" Lang
Thomas Wittek
> =begin pod
>
> =head3 C<method from_string(Str $s);>
> =end pod
>
> method from_string(Str $s){
> # implementation of that method here
> }
>
> Since method signatures are very expressive in Perl 6, there should be a
> way of accessing them in the POD without copy & paste.
As I read "=head 3 method..." I also had the idea that semantically more
meaningful directives might be a good idea.
I mean POD uses constructs like headlines, lists, blocks, italic etc.
which all describe _how it looks like_ and not _what it is_.
A head3 might be the headline of a method documentation as well as one
introducing the contact information for the author of a module.
The directive doesn't have much semantics.
Other people might use head2 for documenting methods, what leads to a
pretty inconsistent look of the documentation.
So maybe directives like method, sub, attribute, class etc. might be a
better choice regarding semantics.
Of course those semantics are directly given in the code, so why not use
them as MarkOv proposed?
It's a bit like HTML<->XML, where the former lacks most of the semantics
and makes the information processing - not to speak about a consistent
look over several documents - a lot harder.
I could imagine a semantic documentation in Perl6, that could be
translated to XML/HTML+CSS or to POD(6) for formatting it.
A semantic documentation could also be very useful in IDEs, where the
IDE could clearly (without guessing) determine the documentation for a
certain element.
Also you could automatically test if every method/class/.. has been
documented etc.
Semantics are very useful in documentation, why throw them away?
--
Thomas Wittek
http://gedankenkonstrukt.de/
Jabber: strea...@jabber.i-pobox.net
Mark Overmeer
> So maybe directives like method, sub, attribute, class etc. might be a
> better choice regarding semantics.
See OODoc::Parser::Markov
> It's a bit like HTML<->XML, where the former lacks most of the semantics
> and makes the information processing - not to speak about a consistent
> look over several documents - a lot harder.
In HTML, you have logical markup as well: EM, STRONG, KEY, CODE, etc
With id and class, you can make any annotation you like:
<div class="method">
my_method OPTIONS
This is the description of a method.
</div>
I love the power of CSS.
> I could imagine a semantic documentation in Perl6, that could be
> translated to XML/HTML+CSS or to POD(6) for formatting it.
The nicest thing would be that the semantic docs become part of the parse
tree, which then (using standard introspection) can be used to generate
manual pages, natively into POD, roff, HTML, whatever.
I see no reason why entangled docs are so hard to parse for Perl6,
as Damian arguments. Even it being difficult is not a good reason to
make the life of all programmers harder.
--
Juerd Waalboer
> So maybe directives like method, sub, attribute, class etc. might be a
> better choice regarding semantics.
Yes, a better choice indeed. But I would still not be happy with it,
because there would still be a lot of code duplication.
method foo (:$bar = 5) { ... }
I don't want to have to mention *again* that the thing is a "method",
and that it is called "foo", that it has a "named argument" identified
as "$bar", which defaults to 5.
This is why I (long time ago) suggested "is documented". Like Mark, I do
not really care about the actual syntax much:
method foo is documented("Foos its argument interactively")
(
:$bar = 5 is documented("Object to be fooed"),
# I'm not sure about the precedence of "is".
) {
...
}
The backtick is rather cute and saves a lot of typing. It's like a
comment (#), but ends up as *external* documentation. That's nice.
> Semantics are very useful in documentation, why throw them away?
Why not have both? With normal POD as suggested by Damian, you could
still generate it from something else. A few macros could help ignore
the inline documentation.
--
korajn salutojn,
juerd waalboer: perl hacker <ju...@juerd.nl> <http://juerd.nl/sig>
convolution: ict solutions and consultancy <sa...@convolution.nl>
Aankhen
> I had suggested syntax like this, in Perl6:
>
> class Mail::Message {
> `base class for message types
>
> .has $msgid;
> `The (internet wide) unique string which identifies this
> `message object. May be undef as long as the message is
> `begin composed.
>
> .has $content_type = 'text/plain';
> }
I always thought this could be achieved using traits:
class Mail::Message is doc("base class for message types") {
.has $msgid is doc("The (internet wide) unique string which
identifies this message object. May be undef as long as the message
is begin composed.");
.has $content_type = 'text/plain';
}
--
Aankhen
(We have no branches.)
Aankhen
> It's a bit like HTML<->XML, where the former lacks most of the semantics
> and makes the information processing - not to speak about a consistent
> look over several documents - a lot harder.
Actually, that's incorrect. HTML is a markup language with a
particular set of elements, some of which have semantic meaning
attached and some of which don't. XML, on the other hand, is a means
of writing your own markup languages; this has two consequences in
this context:
1. It is just as easy—if not easier—to have an XML dialect containing
elements with absolutely no meaning.
2. Even if the dialect contains only elements with well-defined
semantics, it's still completely meaningless to a generic XML parser.
A parser must be intimately familiar with the dialect to understand
that any element has semantic meaning.
If you were referring to XHTML vs. HTML, I would like to point out
that XHTML 1.0 is merely a reformulation of HTML 4.01 in XML. The
elements and their semantics are unchanged. XHTML 1.1 modularizes the
DTD and adds a few Ruby (annotation, obviously, not language)
elements, in addition to a few other minor changes. As for XHTML 2.0,
that's still a long way off. :-)
Thom Boyer
> 2. Getting block comments to hide POD blocks wouldn't require the POD
> parser to have a full implementation of a Perl parser. It would
> require the POD parser to have a _limited_ implementation of a Perl
> parser, one that's capable of identifying block comments. And IIRC,
> you already have to do something like this with respect to block
> quotes:
Actually, the POD parser would have to be a fairly complete Perl parser.
As your example shows:
>
> say :to(END);
> =begin POD
> blah blah blah
> =end POD
> END
>
> If I understand matters correctly, the "POD code" in the above example
> isn't POD code at all, but rather the content of a multi-line quote.
> So extending the POD parser's awareness of Perl syntax to handle block
> comments as well isn't much of a stretch.
If *I* understand matters correctly, Perl 6 will see that as identical to
say :to(END);
END
*because* we want the POD parser be simple, rather than an
almost-complete parser of Perl. After all, there are many ways to "hide"
POD-like text in Perl.
You might think, "Well, how hard is it to look for ":to(XYZ)" followed
by a line containing XYZ?" Going beyond the fact that there are actually
quite a few ways to spell ":to(XYZ)", Perl code that generates Perl code
will trip you up:
say "say :to(END);";
say "Here's my text";
say "=begin comment";
say "My auto-generated code even contains comments on it's strings!"
say "=end";
say "More of my text";
say "END";
You have to parse all string syntaxes to avoid that pitfall. And once
you take care of that issue, you have to handle the problem of parsing
Perl code that's trying to parse Perl code, looking for say statements
that use here-doc syntax containing POD comments:
my $checker = regex {
say \s* \: to \( \w+ \) ; $ # match a say with here-doc
(
<! \s* $1 > .* $
)* # match any lines that don't start with $1
=for comment # match the beginning of an embedded POD comment
}
(No, I don't approve of the indentation of that example. But should your
program fail just because somebody doesn't follow good style?)
And once you've extended the parser to handle *that* case, Finagle's Law
dictates that someone will put the above code sample into a Perl string
literal!
In summary:
As your example pointed out, the POD parser would have to handle all the
various string syntaxes of Perl 6. Then, there is also regular
expression syntax that has to be covered. And who knows what else I'm
overlooking?
Compare that with the simplicity that $larry & $damian are promoting.
Yes, it'll be a pain to get the effect you *wanted* from
> say :to(END);
> =begin POD
> blah blah blah
> =end POD
> END
but the cost to the POD parser is just not worth it.
=thom
-----
Finagle's Law: the perversity of the Universe tends towards a maximum.
Ruud H.G. van Tol
> The nicest thing would be that the semantic docs become part of the
> parse tree, which then (using standard introspection) can be used to
> generate manual pages, natively into POD, roff, HTML, whatever.
I like to call them: lexical comments.
--
Groet, Ruud
Thom Boyer
> I mean POD uses constructs like headlines, lists, blocks, italic etc.
> which all describe _how it looks like_ and not _what it is_.
I think Damian would take exception to that statement. He worked quite
hard to make sure that POD describes _meaning_ rather than _appearance_.
He even renamed B<> from "bold" to "basis", I<> from "italic" to
"important", and U<> from "underline" to "unusual". All of those are
fairly odd choices, with the possible exception of "important", but they
were clearly made with an eye to backwards compatibility of POD while
changing people's focus from how it looks to what it is.
> A head3 might be the headline of a method documentation as well as one
> introducing the contact information for the author of a module.
> The directive doesn't have much semantics.
> Other people might use head2 for documenting methods, what leads to a
> pretty inconsistent look of the documentation.
Well, I'd argue that head3 has plenty of semantics. It's a level-3
header. How that's rendered is decided elsewhere.
I think the issue is not that head3 is insufficiently semantic, it's
just that you want something that's even more specific. And POD 6 was
designed with exactly that kind of thing in mind. You can, according to
the existing S26, say things like:
=Method the method synopsis goes here....
=begin Parameters
=item foo is the fooiest parameter
=item bar is the barstest parameter
=end Parameters
Furthermore, the Perl parser actually *keeps* the POD chunks, so you can
access all that information in the context of the Perl parse. Damian and
Larry were clearly laying the groundwork for exactly the sort of
javadoc-ish features you are asking for.
=thom
-----
The supreme irony of life is that hardly anyone gets out of it alive.
--Robert A. Heinlein [Job: A Comedy of Justice]
Moritz Lenz
> Moritz Lenz:
>> =begin pod
>>
>> =head3 C<method from_string(Str $s);>
>> [..]
>> =end pod
>>
>> method from_string(Str $s){
>> # implementation of that method here
>> }
>>
>> Since method signatures are very expressive in Perl 6, there should be a
>> way of accessing them in the POD without copy & paste.
>
> As I read "=head 3 method..." I also had the idea that semantically more
> meaningful directives might be a good idea.
That may be my fault, I didn't care to look if there was a more semantic
way to describe it.
> I mean POD uses constructs like headlines, lists, blocks, italic etc.
> which all describe _how it looks like_ and not _what it is_.
Headlines, lists and blocks are IMHO semantic markup.
If I'd say "Huge font in bold" that'd be descriptive rather than
semantic markup.
> A head3 might be the headline of a method documentation as well as one
> introducing the contact information for the author of a module.
> The directive doesn't have much semantics.
That doesn't make a headline worthless. It just shouldn't be abused the
way I did it ;)
Larry Wall
: Compare that with the simplicity that $larry & $damian are promoting.
Yes, and the simplicity we're promoting here is mostly *syntactic*
simplicity. It's obvious that at a semantic level, there has to be a
certain amount of incest between the two spheres (if indeed there are
only two). From the Perl 6 perspective, information flows easily from
the documentation to the program via the %=FOO pod twigil, because
the Perl 6 parser is required to embed a pod parser specifically to
guarantee that such values will be introspectable.
Getting information to flow the other direction is more problematic
because documentation is essentially passive data and must be
interpreted by something external (often several different somethings).
However, for a given language that uses pod for documentation,
there can certainly be some conventions for making references to
nearby declarations without having to repeat the entire declaration.
Such references could be either by name or by position. Positional
notation forces you to intersperse things you might not want to.
What programmer really wants to see their function signature splattered
out over many lines, or wants the beginning of the declaration off
the screen before you get to the function body? Also, positional
notation doesn't deal well with factoring out identical names from
related entities that just want to share a chunk of documenation.
So I think a named reference is a good compromise, where the name in
the documentation (in some easily recognized syntactic form) refers
to the next declaration (or set of declarations) of that same name.
Hopefully the pod either knows implicitly or has been told explicitly
how to parse out such declarations for insertion into the document.
(And a positional notation could just be a degenerate case of not
specifying the name (or using a name of * maybe), so it finds the
next declaration of any name. Maybe even some kind of wildcarding
works for hybrid situations.)
The syntax for such named forward references is open for bikeshedding.
(I've intentionally not given any examples in this message to avoid
prejudicing the matter.)
Larry
Mark Overmeer
> the existing S26, say things like:
>
> =Method the method synopsis goes here....
> =begin Parameters
> =item foo is the fooiest parameter
> =item bar is the barstest parameter
> =end Parameters
Where is the link with the code? That's the point: there is no
automatic checking/avoidance of repetition. Besides, your example
is not defined by S26: one can decide to use the tags, someone
else chooses other names, and they then cannot be combined into
one nice homogeneous set of pages. That's a horror!
And if you really like above syntax, why not define
=method the method synopsis goes here....
=option foo is the fooiest parameter
=default foo 10
=requires bar is the barstest parameter
Which is close to how OODoc is extending POD for Perl5.
IMO We can (should) do better for Perl6.
--
Jonathan Lang
> So I think a named reference is a good compromise, where the name in
> the documentation (in some easily recognized syntactic form) refers
> to the next declaration (or set of declarations) of that same name.
> Hopefully the pod either knows implicitly or has been told explicitly
> how to parse out such declarations for insertion into the document.
> (And a positional notation could just be a degenerate case of not
> specifying the name (or using a name of * maybe), so it finds the
> next declaration of any name. Maybe even some kind of wildcarding
> works for hybrid situations.)
>
> The syntax for such named forward references is open for bikeshedding.
> (I've intentionally not given any examples in this message to avoid
> prejudicing the matter.)
My first instinct for such a thing would be to surround the perl code
to be referenced with POD-like tags:
=code name
class foo
...
=/code
...the idea being that a Perl parser treats the =code and =/code lines
as line comments, while a Pod parser would take the block of lines
that exists between the two and attaches it to 'name', to be rendered
as is whenever 'name' is referenced.
In short, =code and =/code would define a target which could then be
referenced as few or as many times as you want within your regular
perl code using something like
=ref name
...where POD would substitute
class foo
...
every time that it sees
=ref name
--
Expanding on this idea:
* if you don't specify a name on a '=ref' line, you automatically
reference the next '=code' section to be found. If you don't bother
naming a '=code' section, this is the only way that it can be
referenced. If you give '^' as the name, then you automatically
reference the last code section that was found. So:
=begin pod
=ref
=end pod
=code
foo
=/code
=begin pod
=ref ^
=end pod
would render the same as
=begin pod
foo
=end pod
=begin pod
foo
=end pod
* Put a number instead of a name, and it counts up (if negative) or
down (if non-negative) through the code blocks, with -1 being the
previous code block and 0 being the next one. A blank name is
equivalent to 0, and a '^' is equivalent to -1. This should be used
sparingly.
* Add or subtract a number to/from a name, and you count up or down
from that name, with '+1' giving the first code block following the
named codeblock and '-1' giving the last code block prior to it.
* In the same way that '=code'...'=/code' is designed to mimic the
'=begin'...'=end' syntax of POD blocks, you could introduce a
variation of '=code' that mimics the '=for' syntax by grabbing the
next line or block of code (say, '=codeline' and '=codeblock'), where
a block of code is terminated by a blank line.
* the final violation of the separation of Church and State would be
to have the Church ask the State to do something for it. Let a
'=code' line specify a parser (such as perl): Pod passes the block of
code to the specified parser; that parser then (presumably) goes
through its paces with the goal of extracting language-specific
documentation (such as the previously suggested 'is doc' traits),
formatting the result as a text block, and handing that block back to
the POD parser. If Pod can't talk to the language parser or vice
versa, Pod simply uses the code block as is.
--
Jonathan "Dataweaver" Lang
Damian Conway
that we made for Pod 6. Here I will merely summarize those reasons, and answer
a specific question.
* Pod and Perl (or any other ambient source code) are to be syntactically
separated, even when there are semantic interconnections
* Perl 6 will be able to access ambient Pod via $=POD (and other
$=WHATEVER variables)
* It doesn't make sense for Pod to access ambient Perl, since Pod is a
passive mark-up notation, not an executable language
* It is self-evidently the case that documentation can be usefully
derived from source code, as well as from explicit mark-up
* That does not, however, mean that source code and explicit mark-up are
(or should be) either equivalent or syntactically commingled
* Rather, it means that the *tools* that extract such documentation need to
be able to extract it from both explicit mark-up and source code
* This will be possible in Perl 6, since both the Pod and Perl 6 parsers
will be accessible from Perl 6.
* Pod 6 is not a presentational mark-up scheme; in fact, it has no
presentational elements at all
* Pod 6 is both a structural and a semantic scheme; you can specify both the
structure of a document, and the meaning of its various components
* Structural mark-up features are entirely lower-case (=head, =item, =table)
at the block level, and single-letter upper-case (B<>, N<>, S<>) at the
embedded level
* Semantic mark-up features are entirely upper-case (=METHOD, =PURPOSE,
=DEFAULT)
* The semantic features are intended to be used to facilitate the creation and
use of tools that can autoextract documentation elements directly from code
* Such tools would (usually) convert the information extracted from
[source + semantic-Pod] into purely structural Pod, which could then
be fed to any suitable pod-to-whatever converter
* In other words, the tool-chain envisaged is something like:
perl6doc pod2whatever
source
+ purely
structural Pod -------> structural -----------> plaintext
+ Pod \----------> HTML
semantic Pod \---------> XML
\--------> roff
\-------> MPEG
\etc.
* Which means that Pod 6 needs to be a suitable pure-structural-mark-up
target language with an extensible semantic overlayer
* Which is precisely what it has already been designed to be
* To summarize the summary:
- Use Perl for what Perl is good for (specification)
- Use Pod for what Pod is good for (commentary)
- Then generate unified documentation by extracting information from
wherever its available (source or mark-up), according to what the
reader asks for
- The issue is not having sufficiently powerful Pod syntax;
the issue is having sufficiently powerful documentation-generating *tools*
To answer Mark's specific question:
> class Mail::Message {
> `base class for message types
>
> .has $msgid;
> `The (internet wide) unique string which identifies this
> `message object. May be undef as long as the message is
> `begin composed.
>
> .has $content_type = 'text/plain';
> }
>
> Damian, can you show how you would document the same code in POD6
> syntax to get a comparible short man-page?
Like so:
class Mail::Message {
=PURPOSE Base class for message types
has $msgid;
=for PURPOSE
The (internet wide) unique string which identifies this
message object. May be undef as long as the message is
begin composed.
has $content_type = 'text/plain';
}
Damian
PS: I agree that there needs to be a mechanism for abstracting names
within Pod and for extracting those names from ambient code. I will
propose such a mechanism and the syntax for it in a subsequent email
(probably early next week).
Damian Conway
> =begin POD
> blah blah blah
> =end POD
> END
>
> If I understand matters correctly, the "POD code" in the above example
> isn't POD code at all, but rather the content of a multi-line quote.
No. It's Pod. *Any* line that begins with '=begin' always starts a Pod
block. Always.
To get the multi-line quote, you'd need:
say :to(END);
=begin POD
blah blah blah
=end POD
END
Damian
Chas Owens
snip
> To get the multi-line quote, you'd need:
>
> say :to(END);
> =begin POD
> blah blah blah
> =end POD
> END
>
> Damian
>
Would this work as well?
say :to(END);
\x{3D}begin POD
blah blah blah
\x{3D}end POD
END
Damian Conway
> Would this work as well?
>
> say :to(END);
> \x{3D}begin POD
> blah blah blah
> \x{3D}end POD
> END
Yes, except for the :to actually needing to be qq:to (which was wrong
all the way through these examples, BTW).
You could also just use:
say qq:to(END);
Larry Wall
: Chas Owens asked:
:
: >Would this work as well?
: >
: >say :to(END);
: >\x{3D}begin POD
: >blah blah blah
: >\x{3D}end POD
: >END
:
:
: Yes, except for the :to actually needing to be qq:to (which was wrong
: all the way through these examples, BTW).
Well, and the fact that it's \x[3D] these days, not \x{3D}.
Larry
Mark Overmeer
> Larry and Thom have each ably explained the rhyme and reason of the choices
> that we made for Pod 6. Here I will merely summarize those reasons, and
> answer a specific question.
>
> * Pod and Perl (or any other ambient source code) are to be syntactically
> separated, even when there are semantic interconnections
Why? Who says that?
> * Perl 6 will be able to access ambient Pod via $=POD (and other
> $=WHATEVER variables)
Cannot find anything about that in S26
> *
> *
> * That does not, however, mean that source code and explicit mark-up are
> (or should be) either equivalent or syntactically commingled
Of course: both serve a different purpose. That's why I would like to
avoid replicating code information in the docs which I have to write: I do
not want to write the attribute names again, and so on. [my impression is
that I use this argument exactly in the opposite direction as you intent]
The code tells how to do it, the docs describe how to use it, the comments
explain how it works. And on some places these tasks overlap. Then, you
have the choice either to write the same thing twice, or (define a
syntax to) share.
> *
> * This will be possible in Perl 6, since both the Pod and Perl 6 parsers
> will be accessible from Perl 6.
And we can access internet as well. But that doesn't say much, does it?
> *
> * Pod 6 is both a structural and a semantic scheme; you can specify both the
> structure of a document, and the meaning of its various components
Yes, and that is one of the things which worries me most *You can*.
It's full freedom, like XML, and without a convention everyone may
think-up there own way of documenting the same kinds of code elements.
Ever tried to merge two aged databases with personnel information?
It's the same horror.
Of course, programmers get all the freedom of the world (the already had)
but we need POD6 only to document Perl6. That is a limited task. Some
people use POD to write books, but is that a reason to give everyone these
features? What is the purpose to people this power (and the challenge
to use it) when there is such a limited set of things to document?
> * Such tools would (usually) convert the information extracted from
> [source + semantic-Pod] into purely structural Pod, which could then
> be fed to any suitable pod-to-whatever converter
The whole point of the debate, is that IMO the focus (and only real
challenge) is this task: combine information about the Perl program
and the documentation about those elements in an as good as feasible
way. All the rest is just syntax. On the moment, I am not convinced
that this task sufficiently facilitated. Many features in POD6 will
complicate this task too much.
> * In other words, the tool-chain envisaged is something like:
>
> perl6doc pod2whatever
>
> source
> + purely
> structural Pod -------> structural -----------> plaintext
> + Pod \----------> HTML
> semantic Pod \---------> XML
> \--------> roff
> \-------> MPEG
> \etc.
^^^^^^^^^^^^ can/will be different *per module*
A structure which is very well known, for instance from LaTex,
looks like this:
perl6doc pod2whatever
source doc
+ -------> tree --------------> plaintext
structural Pod ^ \----------> HTML
| \---------> XML
semantic \--------> roff
definition \-------> MPEG
|
style sheets ---'
In this structure, the responsibility of how things get interpreted
is not for the programmer, so consistent over all modules. We can
make search.cpan.org and manual-pages with a consistent structure.
The actual formatting is only at the back-end, which knows what
the back-end is capable of. So search.cpan.org can design a
nice CSS. Websites have really improved since CSS: finally all
the pages within a website look the same.
And what I would like to see is that doc-tree and Perl6 AST are one.
Producing docs then can be done via standard introspection.
> * Which means that Pod 6 needs to be a suitable pure-structural-mark-up
> target language with an extensible semantic overlayer
> * Which is precisely what it has already been designed to be
With your structure, yes.
> * To summarize the summary:
> - Use Perl for what Perl is good for (specification)
> - Use Pod for what Pod is good for (commentary)
- Use expressions where expressions are good for (calculation)
- Use regexes where regexes are good for (matching)
...
i.e. it is not a valid argument: expressions and regexes
are integrated.
> - Then generate unified documentation by extracting information from
> wherever its available (source or mark-up), according to what the
> reader asks for
The semantics differ per module, so the reader cannot (for instance)
use her own style sheet: the superset of definitions is not known.
> - The issue is not having sufficiently powerful Pod syntax;
> the issue is having sufficiently powerful documentation-generating
> *tools*
IMO it is: the issue is to have sufficiently integrety in code and markup
to be able to create documentation-generating tools which produce enough
quality. And at the same time give the programmer the mimimal burden on
writing documentation, to increase the chance that it is present and good.
[this last sentence is my only design criterium]
> To answer Mark's specific question:
>
> > class Mail::Message {
> > `base class for message types
> >
> > .has $msgid;
> > `The (internet wide) unique string which identifies this
> > `message object. May be undef as long as the message is
> > `begin composed.
> >
> > .has $content_type = 'text/plain';
> > }
> >
> > Damian, can you show how you would document the same code in POD6
> > syntax to get a comparible short man-page?
>
> Like so:
>
> class Mail::Message {
> =PURPOSE Base class for message types
>
> has $msgid;
> =for PURPOSE
> The (internet wide) unique string which identifies this
> message object. May be undef as long as the message is
> begin composed.
>
> has $content_type = 'text/plain';
> }
This is just a syntax transformation, where I can live with. No
problem. But it is not the whole story. "PURPOSE" is not in S26.
So: you need at least a few lines more in above counter-example
to make this work.
- the semantic pod
- in the purely structural pod
- in pod2whatever
more?
In my "vision", the example is complete. Everything else is determined
by the processing tools and style sheets.
> PS: I agree that there needs to be a mechanism for abstracting names
> within Pod and for extracting those names from ambient code. I will
> propose such a mechanism and the syntax for it in a subsequent email
> (probably early next week).
I am looking forward to it.
Do not understand me wrong: for the choosen approach to produce
documentation, you did an excellent job with the specification. It
is well written, well documented, and even implemented.
But I do not see how this approach contributes to the homogeneous set
of manual-pages for Perl modules that the end-user prefers. And I feel
that its freedom makes my life as programmer any easier.
[ Damian, we didn't change our opinions a thing since the last debate
on this subject, last year, haven't we. Probably just another
holy war ]
Brian D Foy
<832f158a0706141722o5f...@mail.gmail.com>, Damian
Conway <dam...@conway.org> wrote:
> No. It's Pod. *Any* line that begins with '=begin' always starts a Pod
> block. Always.
As you know, one of the biggest complaints about Perl is that you have
to have a lot of special rules knowledge to figure some things out.
Whether that is true doesn't really matter: people still complain about
it.
In this case, it will actually be true. That a Perl 6 compiler might
actually decide that in the middle of a statement it isn't a statement
anymore but is Pod will cause some grief, not only in the here doc
example you show, but in things such as:
my $x
=begin();
This impacts Learning Perl 6 rather early because it's a rule that
people need to know at the same time that we tell them that whitespace
is insignificant. That's not really true anymore because a newline
followed by an = followed by begin is this special case, **no matter
how it shows up in the program**. Now there's this extra footnote to
explain this situation, and at the level of basic syntax, we have to
explain a lot more.
I realize that the motivation for this was to be able to scan a file
and extract the pod without parsing the Perl, but when the consequences
affect very basic language things, like where you put your whitespace
and operators, then you create more of a mess than you solve.
So, if this is the case, how will a new Perl 6 user debug a program
failure when part of their program mysteriously disappears because
they just happened to have =begin at the beginning of a line? And, is
the tradeoff in language complexity worth the extra trouble?
Also, doesn't this then limit Pod to Perl 6 (which I thought was not
the goal)? I doubt other languages will want to deal with this
situation.
Damian Conway
>> * Pod and Perl (or any other ambient source code) are to be syntactically
>> separated, even when there are semantic interconnections
>
> Why? Who says that?
Me. :-)
>> * Perl 6 will be able to access ambient Pod via $=POD (and other
>> $=WHATEVER variables)
>
> Cannot find anything about that in S26
Try S02.
> Yes, and that is one of the things which worries me most *You can*.
> It's full freedom, like XML, and without a convention everyone may
> think-up there own way of documenting the same kinds of code elements.
No. There *will* be conventions, which will be defined by the behaviour of the
standard documentation tools that we create.
> The whole point of the debate, is that IMO the focus (and only real
> challenge) is this task: combine information about the Perl program
> and the documentation about those elements in an as good as feasible
> way. All the rest is just syntax. On the moment, I am not convinced
> that this task sufficiently facilitated. Many features in POD6 will
> complicate this task too much.
Naturally, having spent a great deal of time to redesign Pod specifically to
facilitate better and easier documentation, I disagree. :-)
> And what I would like to see is that doc-tree and Perl6 AST are one.
And I am strongly opposed to that as the only alternative.
That said, it will certainly be *possible*.
> i.e. it is not a valid argument: expressions and regexes are integrated.
Sure, but you're arguing from a false analogy. Expressions and regexes are the
same kind of thing: executable specifications. So of course they're
integrated. Documentation is a different kind of thing...so naturally it
should be dis-integrated. ;-)
> IMO it is: the issue is to have sufficiently integrety in code and markup
> to be able to create documentation-generating tools which produce enough
> quality. And at the same time give the programmer the mimimal burden on
> writing documentation, to increase the chance that it is present and good.
> [this last sentence is my only design criterium]
And I claim the current design fully facilitates that.
>> Like so:
>>
>> class Mail::Message {
>> =PURPOSE Base class for message types
>>
>> has $msgid;
>> =for PURPOSE
>> The (internet wide) unique string which identifies this
>> message object. May be undef as long as the message is
>> begin composed.
>>
>> has $content_type = 'text/plain';
>> }
>
> This is just a syntax transformation, where I can live with. No
> problem. But it is not the whole story. "PURPOSE" is not in S26.
It is *now* ;-)
Remember, we're still designing and documenting that design. Hence the careful
wording of S26:
All other uppercase block typenames are reserved...
^^^
Standard semantic blocks include:
^^^^^^^
Ultimately, the complete set of semantic blocks will be defined by the scope
and behaviour of the documentation tools we create.
> In my "vision", the example is complete. Everything else is determined
> by the processing tools and style sheets.
Agreed.
> Do not understand me wrong: for the choosen approach to produce
> documentation, you did an excellent job with the specification. It
> is well written, well documented, and even implemented.
> But I do not see how this approach contributes to the homogeneous set
> of manual-pages for Perl modules that the end-user prefers.
It does so by providing a standard--and extensible--mark-up notation and a
well-structured document object model (which includes representations for
interspersed code), as well as standardized parsing tools from converting a
source document to an internal representation.
These features (along with Perl 6's ability to parse Perl 6 source) will make
it vastly easier to build man-page generators.
> [ Damian, we didn't change our opinions a thing since the last debate
> on this subject, last year, haven't we. Probably just another
> holy war ]
Yes. For that very reason I don't propose to keep arguing this issue.
I certainly respect your concerns and desires. I'm merely frustrated by the
fact that I can't seem to convince you that they're actually being addressed. :-)
Damian
Smylers
> In article
> <832f158a0706141722o5f...@mail.gmail.com>, Damian
> Conway <dam...@conway.org> wrote:
>
> > No. It's Pod. *Any* line that begins with '=begin' always starts a Pod
> > block. Always.
>
> As you know, one of the biggest complaints about Perl is that you have
> to have a lot of special rules knowledge to figure some things out.
Indeed. What's much nicer is to be able to state that a given rule
"always" applies.
Like Damian has just done here.
Saying that C<=> at the start of a line always means Pod is much simpler
than having a list of exceptions of places where it doesn't.
> Also, doesn't this then limit Pod to Perl 6 (which I thought was not
> the goal)?
I reckon the complete opposite.
> I doubt other languages will want to deal with this situation.
With these new Pod rules it's possible to entirely remove Pod from a
file without knowing _anything_ about the host language. (It could
straightforwardly be done as an editor macro, for example.) That
permits Pod to be used to document just about anything; all you need to
allow it is a filter that strips off all Pod before you do anything else
with the file.
If Pod were to take notice of the host language's context throughout the
file then this would not be possible: every language which wished to
have Pod support would require its own Pod parser embedded within the
languge parser. _That_ is orders of magnitude more complex than the
simplicity of filtering off all Pod first, and strikes me as something
other languages are much less likely to be bothered to do.
Smylers
Smylers
> * Damian Conway (dam...@conway.org) [070615 00:17]:
>
> > * Pod 6 is both a structural and a semantic scheme; you can specify
> > both the structure of a document, and the meaning of its various
> > components
>
> Yes, and that is one of the things which worries me most *You can*.
> It's full freedom,
You're concerned that an aspect of Perl 6 might have too much freedom?
Isn't Perl all about giving users freedom to choose their own way of
doing something?
> like XML, and without a convention everyone may think-up there own way
> of documenting the same kinds of code elements.
Yes. But in reality many people will follow what others do, or look to
follow best practices. With Perl 5 you have complete freedom as to the
names of C<=head1> sections in the Pod for modules, yet in browsing Cpan
it's clear that there are conventions and many people use the same
headings. So not mandating a convention isn't much of a problem.
Moreover, I reckon that not mandating a convention is essential. Look
at what's being done with Perl 5 at the moment (not specifically Pod,
just in the Perl 5 community in general) and the best practices that
have sprung up in recent years (and are still evolving). People are, of
their own accord, following conventions that nobody had even thought of
at the time Perl 5 was released; even at the time Perl 5.6, say, was
released.
> In this structure, the responsibility of how things get interpreted is
> not for the programmer, so consistent over all modules. We can make
> search.cpan.org and manual-pages with a consistent structure.
Do you really think that people can now, before Perl 6 has gained
anything approaching the usage we expect, make policy for how things
should be documented, such that that policy will be the best possible
way of documenting everything written in Perl 6, for ever? Or even a
good way?
That strikes me as incredibly shortsighted, verging on arrogance by
whoever comes up with the rules, and doomed to failure.
Rather than trying to map out the future in detail (which is tricky),
the best we can do is come up with things that are sufficiently flexible
that they're capable of being used in ways we haven't yet thought of.
Then when somebody, years from now, has a good idea, it will be possible
for that to be implemented (and followed by others), rather than tying
us to some convention set at an arbitrary point in the past.
> > * To summarize the summary:
> > - Use Perl for what Perl is good for (specification)
> > - Use Pod for what Pod is good for (commentary)
> - Use expressions where expressions are good for (calculation)
> - Use regexes where regexes are good for (matching)
> ...
>
> i.e. it is not a valid argument: expressions and regexes
> are integrated.
Yes, but on the other side of the argument coconuts and fax machines are
not integrated. I'm reasonably confident that for every pair of things
which you list as being integrated I can come up with a pair which
aren't; I doubt that will really assist the argument one way or t'other.
> the issue is to have sufficiently integrety in code and markup
> to be able to create documentation-generating tools which produce enough
> quality.
Damian's spec permits this.
> And at the same time give the programmer the mimimal burden on writing
> documentation, to increase the chance that it is present and good.
You should 'Perl 6 Documentation Best Practices', with guidelines for
how to use Pod. I'm sure many people would appreciate just being able
to follow a template rather than having to make decisions over the small
details of what to do.
That way we have a convention for those that want it, but also don't tie
ourselves into anything.
If a particular convention gains widespread approval then peer pressure
should encourage its use (in the same way that strict and warnings are
currently optional in Perl 5, but in the absence of a good reason it's
expected that they be used).
Smylers
Mark Overmeer
> > * Damian Conway (dam...@conway.org) [070615 00:17]:
> > > * Pod 6 is both a structural and a semantic scheme; you can specify
> > > both the structure of a document, and the meaning of its various
> > > components
> >
> > Yes, and that is one of the things which worries me most *You can*.
> > It's full freedom,
>
> You're concerned that an aspect of Perl 6 might have too much freedom?
> Isn't Perl all about giving users freedom to choose their own way of
> doing something?
Why treat documentation as a second-class citizen all the time? Why
have a standard syntax for regexes, and not for docs? Aren't you glad
that at last we get a standard for OO programming and named parameters?
The boundary between freedom and anacharchy is faint.
> Yes. But in reality many people will follow what others do, or look to
> follow best practices. With Perl 5 you have complete freedom as to the
> names of C<=head1> sections in the Pod for modules, yet in browsing Cpan
> it's clear that there are conventions and many people use the same
> headings. So not mandating a convention isn't much of a problem.
Well, the you are talking about the top three headers, the most. And
those have a good example in standard UNIX manual-pages. So: there
is a definitions for them, which most people have seen.
Look at how people document how methods should be called. We, as
programmers, don't worry: we can read between the lines. But most
"normal" programmers we hardly ever meet, have a hard time. Is
"freedom" (in some definition) so important that consistency must
suffer?
> Do you really think that people can now, before Perl 6 has gained
> anything approaching the usage we expect, make policy for how things
> should be documented, such that that policy will be the best possible
> way of documenting everything written in Perl 6, for ever? Or even a
> good way?
There is no need to think that a documentation syntax develops differently
than a programming language. So when Perl is developing, POD can develop
in parallel. And, of course, documentation is for most people a well
known area: at least, every program I wrote last 30 years had some form
of documentation. To find a kind of common base of features you need does
not require any knowledge about the language involved is quite simple.
> That strikes me as incredibly shortsighted, verging on arrogance by
> whoever comes up with the rules, and doomed to failure.
Sorry? Not only you insult me, but you also ignore all these other
languages which do have a nice and standard way of documenting.
Insignificant languages, like Java, which we tend to ignore. Also
Python, to name a closer neighbour.
It is incredibly shortsighted to think that any experienced programmer
would suggest a standard which cannot evolve, and adapt to variance
in needs.
What I suggest is at least some kind of suggestion how to denote the
things everyone certainly needs. As I understand now, Damian is working
on that.
> If a particular convention gains widespread approval then peer pressure
> should encourage its use (in the same way that strict and warnings are
> currently optional in Perl 5, but in the absence of a good reason it's
> expected that they be used).
Well, "man Perl/BUGS" says:
The -w switch is not mandatory.
So at least some people agree that there may need some more guidance.
Warnings default "on" is a good idea, just as giving people a good
default documentation strategy.
Mark Overmeer
>> I doubt other languages will want to deal with this situation.
* Smylers (Smy...@stripey.com) [070616 08:44]:
> With these new Pod rules it's possible to entirely remove Pod from a
> file without knowing _anything_ about the host language. (It could
> straightforwardly be done as an editor macro, for example.) That
> permits Pod to be used to document just about anything; all you need to
> allow it is a filter that strips off all Pod before you do anything else
> with the file.
And then the main point: if you write documentation which is not
related to Perl6 coding itself, do we really need to create just
another text processor? There are already so many sofisticated
text processors available!
Well, ok, if you "get it for free", like with POD(5), then take
that opportunity. But if the life of ordinary programmers who write
documentation is complicated just to provide this feature, we are IMO
on the wrong path.
And why do you want easy to remove docs? Perl6 is even nicer: you can
distribute it compiled; no program text and no docs! Docs in code
files are much less of a "burden" than in Perl5, so the need to strip
them from the code has deminished.
> If Pod were to take notice of the host language's context throughout the
> file then this would not be possible: every language which wished to
> have Pod support would require its own Pod parser embedded within the
> languge parser. _That_ is orders of magnitude more complex than the
> simplicity of filtering off all Pod first, and strikes me as something
> other languages are much less likely to be bothered to do.
Other languages already have their own documentation system. Why do you
expect them to use POD6? Why should we design POD6 with this endlessly
open requirement in mind? Let's rule the World!
All I try to achieve is the best possible set of documentation for
end-users: consistent in structure and correct in content. To achieve
this, I wish to keep the freedom and avoid the anacharchy. Correctness
automatically improves where redundancy is removed. But that requires
a closer doc/code entanglement than a lot of the experienced Perl people
like (==are used to).
--
Damian Conway
> As you know, one of the biggest complaints about Perl is that you have
> to have a lot of special rules knowledge to figure some things out.
> Whether that is true doesn't really matter: people still complain about
> it.
>
> In this case, it will actually be true.
I don't think that's the case. I think the new approach to Pod has
exactly the opposite effect: it *eliminates* the need for a lot of
special rules and contextual understandings.
> This impacts Learning Perl 6 rather early because it's a rule that
> people need to know at the same time that we tell them that whitespace
> is insignificant.
I'm pretty sure "Learning Perl 6" won't say that. Mainly because it's
not true. Whitespace in Perl 6 isn't always insignificant, even in the
simplest examples. Indeed, whitespace is significantly less
insignificant in Perl 6 than it was in Perl 5.
> That's not really true anymore because a newline followed by an =
> followed by begin is this special case, **no matter how it shows up
> in the program**. Now there's this extra footnote to explain this
> situation, and at the level of basic syntax, we have to explain a
> lot more.
I don't think so.
There are two distinct models we're considering in this thread. The
first model is that Pod is a layer entirely syntactically separate from
Perl, and which is preprocessed out of existence before the compiler
ever sees the source of a program. Call this the "Separation model".
The second model is that Pod is an integral syntactic feature of
Perl: effectively a special kind of comment. Call this the
"Integration model".
The rule you have to teach under the Separation model is:
"Any line that starts with an = is Pod...and not part of your program."
Whereas the rule you have to teach under the Integration model is:
"Any line that starts with an = is Pod...and not part of your program
...UNLESS it's part of an assignment or comparator or inside a string
in which case it's something else
...UNLESS it's also inside a code block inside that string,
in which case it's a Pod command again
...UNLESS it's part of a nested assignment or comparator
or doubly nested inside the nested string
...UNLESS it's in a doubly nested code block
inside that nested string
...etc.
...etc."
So, for example, consider the following lines:
$var
=comment Is this Pod?qq{
=comment Is this Pod?qq{
=comment Is this Pod?qq{
}}}
Which of those are Pod and which are Perl?
Under the Separation model, they're all Pod;
under the Integration model, only one of them is.
> So, if this is the case, how will a new Perl 6 user debug a program
> failure when part of their program mysteriously disappears because
> they just happened to have =begin at the beginning of a line?
The same way they debug it when part of their program mysteriously
disappears because they just happened to have # at the beginning of a
line: by learning to distinguish between code and commentary.
Except, of course, the Pod mysteriously vanishing will be considerably
easier to debug, because ALL lines starting with =begin vanish, whereas
only some lines beginning with # do.
> Also, doesn't this then limit Pod to Perl 6 (which I thought was not
> the goal)? I doubt other languages will want to deal with this
> situation.
As Smylers so ably pointed out, enabling Pod to be parsed independently
of the underlying language syntax actually makes it vastly easier to use
Pod with other languages. All you need to do is insert a Pod-stripping
preprocessor (already written, as it happens) before the language's
compiler...as, indeed, the Perl6::Pod does for Perl 5.
The bottom line is that separated Pod conforms to a single simple rule,
which makes it:
* easy to teach: "a = in column 1 is Pod"
* not context-sensitive: "a = in column 1 is ALWAYS Pod"
* easy for humans to identify: "is there an = in the first column?"
* easy for machines to parse: "is there an = in the first column?"
* easy to use: "put a = in column 1"
* easy to avoid "keep the = out of column 1"
I truly think it works better that way :-)
Damian
Mark Overmeer
* Damian Conway (dam...@conway.org) [070616 08:29]:
> No. There *will* be conventions, which will be defined by the behaviour
> of the standard documentation tools that we create.
man-page Perl6 secion BUGS:
The "Damian Documentation Conventions" should have been mandatory.
[Careful: this is intented to be a pun]
> >[ ... Probably just another holy war ]
> ... I'm merely frustrated by the fact that I can't seem to convince
> you that they're actually being addressed.
Well, let's have a drink in Vienna to ease our mutual frustrations ;-)
--
Brian D Foy
<dam...@conway.org> wrote:
[ First, I should note that whatever we end up with, that's the party
line and that's what I teach, but before we end up there, I know from
my years of experience teaching that certain sorts of questions are
going to come up. I'm looking at this from the perspective of the
student sitting in a class, not from the implementors perspective. ]
> brian wrote:
>
> > As you know, one of the biggest complaints about Perl is that you have
> > to have a lot of special rules knowledge to figure some things out.
> > Whether that is true doesn't really matter: people still complain about
> > it.
> >
> > In this case, it will actually be true.
>
> I don't think that's the case.
I'm speaking at the programmer level, not the implementor level. For the
guy in the trenches, this is a special case. This is more complexity
for the guy typing code, even if it's easier syntactically for the guy
writing the parser.
> > This impacts Learning Perl 6 rather early because it's a rule that
> > people need to know at the same time that we tell them that whitespace
> > is insignificant.
>
> I'm pretty sure "Learning Perl 6" won't say that.
Well, it won't say that if it's not true, but until pretty recently
it's been mostly true.
> The rule you have to teach under the Separation model is:
>
> "Any line that starts with an = is Pod...and not part of your program."
And that's something that now comes up very early in teaching the
assignment operator.
> Whereas the rule you have to teach under the Integration model is:
We don't teach any rule under this model, and it's been fine for over a
decade :)
When we do teach the current Pod, the simple rule is that Pod starts:
* when Perl is expecting a new statement
* there is a =something at the beginning of the line
Which is considerably simpler than the long rule list you used to say
the same thing (which seems a bit of FUD, honestly).
> > So, if this is the case, how will a new Perl 6 user debug a program
> > failure when part of their program mysteriously disappears because
> > they just happened to have =begin at the beginning of a line?
>
> The same way they debug it when part of their program mysteriously
> disappears because they just happened to have # at the beginning of a
> line:
> Except, of course, the Pod mysteriously vanishing will be considerably
> easier to debug, because ALL lines starting with =begin vanish, whereas
> only some lines beginning with # do.
That's not really the case. The # only affects one line, and a pound in
a literal string doesn't start a comment. People grok single line
comments very well.
The start of a Pod comment now affects it's line and the ones following
it.
>
> > Also, doesn't this then limit Pod to Perl 6 (which I thought was not
> > the goal)? I doubt other languages will want to deal with this
> > situation.
>
> As Smylers so ably pointed out, enabling Pod to be parsed independently
> of the underlying language syntax actually makes it vastly easier to use
> Pod with other languages.
Well, easy to use Pod with other languages until they try to use the
assignment operator at the beginning of the line, or a = in a literal
string at the beginning of a line.
I know you think it's easier to teach and explain, but that's because
you came up with it. The notion that a special character in a certain
column means something was tough to explain to other people in FORTRAN
too.
There are other things to consider, and to me it looks like this design
decision isn't based on what's easier for the Perl 6 programmer but
what's easier for the implementors. It's not that I don't understand
both sides, I just disagree about where the complexity should be.
Brian D Foy
<Smy...@stripey.com> wrote:
> brian d foy writes:
>
> > In article
> > <832f158a0706141722o5f...@mail.gmail.com>, Damian
> > Conway <dam...@conway.org> wrote:
> >
> > > No. It's Pod. *Any* line that begins with '=begin' always starts a Pod
> > > block. Always.
> >
> > As you know, one of the biggest complaints about Perl is that you have
> > to have a lot of special rules knowledge to figure some things out.
>
> Indeed. What's much nicer is to be able to state that a given rule
> "always" applies.
Well, now explain literal strings :) This isn't about one rule, it's
about an ecosystem.
The rules for Pod5 always applied too, so I don't see what we've gained
here as far as the simplicity of rules (but let's not go
round-and-round on that since we've both already explained our
positions).
> Like Damian has just done here.
This is a point where a lot of people will disagree, I suppose, and
it's a fundamental sort of disagreeement where neither side will really
be convinced otherwise. I don't think this is stubbornness either, but
reflects what people value most. That is, nobody is really wrong,
because we'd have to define some way to measure that, and we're really
disagreeing on the yard stick.
Putting aside this particular situation, the argument comes down to
where does the water balloon pooch out? We haven't lost any complexity,
it's just in different places. Maybe some places need less complexity
and some places could stand a little more.
Now, in this particular situation, Pod is much easier to extract, but
literal strings now have extra baggage to consider. That's not what
we're debating though. The real debate is whether you think moving the
complexity around like that is worth it.
Personally, extracting Pod from Perl 5 hasn't been a problem for me
(and I do a lot of Pod work and write lots of custom Pod translators),
so I don't think this re-distribution is worth it. I don't have to
answer many questions about extracting Pod, and I don't see many normal
people (meaning, not us) asking for easier ways to do this. I don't see
a motivation, for the perspective of normal people, for this. When we
through around terms like "natural language", normal people matter. :)
You and Damian have explained the other side very well. I understand
it, and it's very easy for me to understand and even deal with. We just
disagree on the consequences.
I tend to think that people like us are here to do the hard work so
other people don't have to think about this sort of stuff, so I don't
mind putting the complexity in the the parser if it takes it out of the
common program elements such as strings.
Darren Duncan
I agree with the camp that says an = at the start of a line is pod,
without exception, regardless of context.
I am also assuming that "start of the line" means there is no
whitespace to the left of the =.
I also recognize and agree with Perl 6 being sometimes whitespace sensitive.
I also found from my own experience a few months back (and I added a
test for it as t/syntax/parsing/pod_in_multi_line_exprs.t), that this
(represented with Pod 5) didn't work, but ought to (assume the 'my'
et al is flush left):
my $list = {
=pod foo docs
=cut
'foo' => sub {...},
=pod bar docs
=cut
'bar' => sub {...},
};
For technical reasons, I was constructing a bunch of routines
anonymized within a hash, rather than added to a package. A whole
file could be mostly a single hash declaration like this. I had been
expecting that I could put pod at the start of any line and it would
work, as is what seems the most intuitive. The camp that I support
would make this work.
Given this, there is an obvious (to me) solution for pod blocks in
the middle of expressions like:
my $foo
= $bar;
As the example shows, and I believe best practices espouse, you
*indent* the code line with a leading =.
This both makes the Perl code easier to understand (a continuation is
indented), and because there is leading whitespace, that = isn't
confused with pod, which is always not indented.
Simple.
And since Perl 6 does consider whitespace significant sometimes, this
is not unprecedented.
-- Darren Duncan
John Beppu
Function: Multiply
Multiplies two integers.
Parameters:
x - The first integer.
y - The second integer.
Returns:
The two integers multiplied together.
See Also:
<Divide>
http://www.naturaldocs.org/ (...which happens to be written in Perl.)
Jonathan Lang
> > Whereas the rule you have to teach under the Integration model is:
>
> We don't teach any rule under this model, and it's been fine for over a
> decade :)
>
> When we do teach the current Pod, the simple rule is that Pod starts:
>
> * when Perl is expecting a new statement
> * there is a =something at the beginning of the line
>
> Which is considerably simpler than the long rule list you used to say
> the same thing (which seems a bit of FUD, honestly).
Agreed about the parenthetical. From my perspective, the proponents
of the Separation Model are overstating the difficulties inherent to
the Integration model. That said:
By the rule you give above, Pod Sections cannot be embedded in the
middle of statements. For example, given:
if
a = 5
{
say "ouch!"
}
you couldn't insert a Pod Section between the first and second lines,
or between the second and third lines. From a guy-in-the-trenches
perspective, this isn't a problem; interrupting an instruction
mid-stream in order to talk about it is very bad form. Still, it's a
restriction that Damian's Separation model lacks.
> > > So, if this is the case, how will a new Perl 6 user debug a program
> > > failure when part of their program mysteriously disappears because
> > > they just happened to have =begin at the beginning of a line?
> >
> > The same way they debug it when part of their program mysteriously
> > disappears because they just happened to have # at the beginning of a
> > line:
>
> > Except, of course, the Pod mysteriously vanishing will be considerably
> > easier to debug, because ALL lines starting with =begin vanish, whereas
> > only some lines beginning with # do.
>
> That's not really the case. The # only affects one line, and a pound in
> a literal string doesn't start a comment. People grok single line
> comments very well.
With the advent of block comments, this isn't as true as it used to
be. In fact, I expect that some people will have to take a moment to
wrap their heads around the notion that you ignore everything that
follows the closing bracket on the last line of the block quote. That
said, I don't expect there to be much of a learning curve here.
Still, it may be worth it to state that a line that starts with '#'
followed by a closing bracket is _not_ commented out (although said
character sequence should be stripped out of the line). Less
cognitive dissonance that way.
> The start of a Pod comment now affects its line and the ones following
> it.
...also not much in the way of a learning curve; in fact, I expect
that people will grok Pod Sections more readily than they will grok
block comments as currently written.
> Well, easy to use Pod with other languages until they try to use the
> assignment operator at the beginning of the line, or a = in a literal
> string at the beginning of a line.
...or you're dealing with a language that assigns special meaning to
lines that begin with '='.
You are not going to be able to make Pod completely orthogonal to the
code that it's embedded in. You can come close (and IMHO you come
_very_ close with the Separation model that you're discussing), but
there will always be some sort of constraint placed on the code into
which you're embedding Pod.
> There are other things to consider, and to me it looks like this design
> decision isn't based on what's easier for the Perl 6 programmer but
> what's easier for the implementors. It's not that I don't understand
> both sides, I just disagree about where the complexity should be.
Thank you, Brian. That's exactly the sense that I've been getting
from the explanations being given.
Which is not to say that there isn't a time and place when ease of
implementation should trump ease of programming; taking an extreme
example, being able to write a program that consists of the single
line:
attend my every wish
is the ultimate in terms of ease of programming; it's also impossible
to implement, barring the invention of mind-reading hardware and
sapient artificial intelligence software. If a small increase in the
programmer's learning curve is the price that must be paid to achieve
a massive reduction in the implementor's workload, it may very well be
worthwhile to foist some of the complexity onto the programmer.
The question is whether or not this is one of those cases.
--
Jonathan "Dataweaver" Lang
Jonathan Scott Duff
> * Smylers (Smy...@stripey.com) [070616 09:09]:
> > > * Damian Conway (dam...@conway.org) [070615 00:17]:
> > > > * Pod 6 is both a structural and a semantic scheme; you can specify
> > > > both the structure of a document, and the meaning of its various
> > > > components
> > >
> > > Yes, and that is one of the things which worries me most *You can*.
> > > It's full freedom,
> >
> > You're concerned that an aspect of Perl 6 might have too much freedom?
> > Isn't Perl all about giving users freedom to choose their own way of
> > doing something?
>
> Why treat documentation as a second-class citizen all the time? Why
> have a standard syntax for regexes, and not for docs? Aren't you glad
> that at last we get a standard for OO programming and named parameters?
> The boundary between freedom and anacharchy is faint.
The docs *do* have a standard syntax. I think you've been arguing for
a more *specific* standard syntax and semantics.
You mention OOP. For Perl 5 we have a standard, if very general,
syntax and "open" semantics that have allowed people to implement OOP
in a variety of ways. This was all well and good for a while until we
realized that there should be some more reasonable defaults (in both
syntax and semantics) for common operations in OOP.
I think it's the same thing with POD6. It's "open" enough that many
documentation systems can be built from it (man pages, books, magazines,
wikis, etc.) For some of those documentation systems we'll have nice
conventions and other conventions will grow as needed. If we find that
convention isn't enough in specific areas, we'll start to grow
requirements for those cases. Requirements will be enforced by the
tools we use, not by the documentation specification (that way we can
use the same source document for multiple purposes with different
requirements).
Also, I don't think that documentation is being treated as
second-class at all. It's being treated as first-class but different.
To form a poor analogy, imagine threads woven together to make a
tapestry. The blue threads are just as important as the red threads,
but they each may have different purposes in the overall design.
my two cents,
-Scott
--
Jonathan Scott Duff <du...@pobox.com>
Jonathan Lang
> Which is not to say that there isn't a time and place when ease of
> implementation should trump ease of programming; taking an extreme
> example, being able to write a program that consists of the single
> line:
>
> attend my every wish
>
> is the ultimate in terms of ease of programming; it's also impossible
> to implement, barring the invention of mind-reading hardware and
> sapient artificial intelligence software. If a small increase in the
> programmer's learning curve is the price that must be paid to achieve
> a massive reduction in the implementor's workload, it may very well be
> worthwhile to foist some of the complexity onto the programmer.
>
> The question is whether or not this is one of those cases.
Addendum: let me note that Perl has a reputation for bending over
backward to accommodate the programmer; as such, the difference in
implementation workloads for the two models that Damian referenced is
going to have to be _very_ severe for "ease of implementation" to win
out over "ease of programming" - IMHO. Unfortunately for me, MHO
doesn't count as much as DHO. "Help, help! I'm being oppressed!"
--
Jonathan "Dataweaver" Lang
Moritz Lenz
I agree, and I want to make a point for the "ease of programming" point
of view.
If we have any kind of references to code, the POD parser must be able
to parse method/sub signatures. But since default values are allowed for
optional (and iirc named) arguments, it has to parse arbitrary complex
expressions anyway.
Which means that a complete POD parse will likely have to fall back to
STD.pm, and in that case it's ridiculous to argue about a line based
parser for POD anymore - when you can parse perl 6, you can just pick
the POD from the syntax tree.
Moreover part of the Perl philosophy has always been to provide the best
for the programmer, even if it makes implementation of the
interpreter/compiler rather hard - why should we stop with this
philosophy when talking about POD and POD-Parser?
I'd find it very annoying to have lines starting with '=' in a heredoc
beeing counted as POD. Humans don't think in terms of lines, but in
terms of visual blocks (at least I do), so if everything in heredoc goes
verbatim into a string, I'd expect _everything_ in that block to go into
that string. Everything else looks like an artificial exception to me.
Dave Whipp
> Given this, there is an obvious (to me) solution for pod blocks in the
> middle of expressions like:
>
> my $foo
> = $bar;
>
> As the example shows, and I believe best practices espouse, you *indent*
> the code line with a leading =.
I'd agree that indentation is good for readability. But I do know (and
regularly use) tools that will "helpfully" strip out leading whitespace.
For example, the bug tracking system I use at work drops all leading
whitespace; and if you put code in MS-Word using style-sheet indentation
then cut&paste will drop it. This would be unfortunate.
My problems with these tools would be reduced if the POD identification
rule was changed from /^=/ to /^=\w/. I.e. whitespace after the initial
"=" marks it as non-pod.
Dave.
Damian Conway
>> The rule you have to teach under the Separation model is:
>>
>> "Any line that starts with an = is Pod...and not part of your
>> program."
>
> And that's something that now comes up very early in teaching the
> assignment operator.
You know, that's a benefit I hadn't even considered. Documentation
*should* come up early! Thanks for pointing it out. ;-)
> When we do teach the current Pod, the simple rule is that Pod starts:
>
> * when Perl is expecting a new statement
> * there is a =something at the beginning of the line
Which means that the user has to understand Perl's statement syntax and
semantics, in order to write Pod. And the user also has to understand
that in order to even read Pod.
> Which is considerably simpler than the long rule list you used to say
> the same thing (which seems a bit of FUD, honestly).
Not FUD...just an partial enumeration of the full recursive complexity
of the "simple rule" regarding "is expecting a new statement".
> I know you think it's easier to teach and explain, but that's because
> you came up with it.
I hope I'm not that shallow. I would like to think I have enough experience
with both design and teaching not to fall prey to that--admittedly common--
mistake. I'm heartened by the fact that several other contributors to
this debate have agreed with me, but I certainly do have to acknowledge
the possibility that you (as an excellent and experienced teacher) are
right. Nevertheless I still believe that the new model, because it has
no exceptions or context dependency, will actually be much easier to
understand and to explain.
> The notion that a special character in a certain column means
> something was tough to explain to other people in FORTRAN too.
Sure. When the "certain column" was column 6! Surely you're not arguing
that:
"A = in the first column introduces some documentation"
is as hard to understand (or even just identify) as:
"A non-blank/non-zero in column 6 continues the previous line"
???
Perceptually and cognitively, the first column is both highly
distinctive and easily identified. For example, in my experience
teaching Fortran (back in the mid-80s), students had no problem
whatsoever learning that "A 'C' in the first column is a comment".
> There are other things to consider, and to me it looks like this
> design decision isn't based on what's easier for the Perl 6 programmer
> but what's easier for the implementors.
I assure you that that is categorically *not* the case (as I'll discuss
at length in a subsequent message).
Damian
Larry Wall
: My problems with these tools would be reduced if the POD identification
: "=" marks it as non-pod.
S26 says that ^^ '=' \s is for continuing a previous directive, and
therefore *must* not be interpreted as a new directive, so I think
you pretty much already have your wish, if we assume that a missing
previous directive implies that a "spaced" = isn't extending anything.
This interpretation would certainly cut down false matches way down
in any culture that conventionally puts spaces on both sides of an =
when used for assignment. (And if there isn't a space after, there
isn't likely to be a space before that would get wrapped.)
Larry
Mark Overmeer
> You mention OOP. For Perl 5 we have a standard, if very general,
> syntax and "open" semantics that have allowed people to implement OOP
> in a variety of ways. This was all well and good for a while until we
> realized that there should be some more reasonable defaults (in both
> syntax and semantics) for common operations in OOP.
OOP in Perl5 has a mechanism, but not a standard in the use of those
features. Full OO languages all choose a direction, but there are
serious culture differences. Perl uses them all. There are many ways
how to instantiate an object, which is lot of fun as programmer but a
hassle for "the average" programmer. Within one program, you may very
well need to use different instantiation techniques... Of course, this
could have been forseen (perl is not the first language which implements
OO), and some advice on the convention to use could have avoided wild
collection of approaches we see now.
This is also why "Perl Best Practices" is a good book [shameless plug]
although I would have welcomed it 11 years earlier.
> I think it's the same thing with POD6. It's "open" enough that many
> documentation systems can be built from it (man pages, books, magazines,
> wikis, etc.) For some of those documentation systems we'll have nice
> conventions and other conventions will grow as needed. If we find that
> convention isn't enough in specific areas, ...
Without any doubt, there are thousands of documentation systems around.
At least ten percent of them are considered "the best ever made" by
its developer or developer community. Just by simple math, the chance
that the system developed really is the best is less than one percent.
Gladly, there are many areas of application for documentation systems,
and there is a large variety in taste by the users. So per user/usage,
a different doc-system can be the best. Still, its a muddy terrain.
POD6 has the unique opportunity to become "the best documentation system"
for a specific large group of users: Perl programmers. Tenth of thousands
of people will use POD6 in their Perl6 programs, every day.
IMO, any argument that POD6 is good because it can be used to write books
or express complex mathematical expressions is really frightning me.
Damian is correctly avoiding this, in his argument. Is this another area
where our community is trying to incorporate every thinkable feature in
some design? And in the process, producing such a complex design that
it deserves an extra volume in "Programming Perl 6"? (Next to the 1000
page volumes for each of basic syntax, grammars, and OO).
IMO, the needs for POD6 are much simpler: a way to document the features
of the related Perl6 code, and smooth that into (parts of) manual pages.
If you write a book, than use pod6-to-docbook or pod6-to-pod, or whatever,
to get some fragments in your some documentation system which is
specialized in books.
Every single complication added to the doc syntax will make it not
to understand for a large percentage of the primar target community,
as every teacher can tell you from experience.
> Also, I don't think that documentation is being treated as
> second-class at all. It's being treated as first-class but different.
It's treated second-class in the Perl6 sense: the tools are not an
integral part of the Perl6 environemnt.
When I create a program, it starts with a goal. To reach that goal,
I have to write some code, some docs, some tests. For me, it is all
part of the program. I do not see one to be of more importance than
the other: all three are first-class sitizens in my program. And what
I really wish for is that those three work together in een KISS way;
to achieve my goal in no time so I can drink (more) beer. Yeh, on the
subject of tests,... :-b
--
Brian D Foy
<dam...@conway.org> wrote:
[writing publicly to head off any notions there's a personality problem
here]
> brian wrote:
> > I know you think it's easier to teach and explain, but that's because
> > you came up with it.
>
> I hope I'm not that shallow.
I didn't mean to imply anything about your character there, but that
naturally as thinking beings we all understand much better than anyone
else our own thoughts and architectures.
It's not that your shallow, it's that you've visited the deepest
trenches in the murky oceans and now have to figure out how to explain
the wonderful creatures you saw to people who don't want to get wet.
You're going to understand it much more because you've actually seen
those creatures. :)
> > There are other things to consider, and to me it looks like this
> > design decision isn't based on what's easier for the Perl 6 programmer
> > but what's easier for the implementors.
>
> I assure you that that is categorically *not* the case (as I'll discuss
> at length in a subsequent message).
I wasn't trying to assign any ethical baggage to that remark, and said
in an earlier message that it's a matter of philosophy about what
different people value most.
I still think it's a true statement though, and that the difference is
an honest disagreement about how the world should be.
Damian Conway
> [writing publicly to head off any notions there's a personality problem
> here]
I said I wasn't going to continue this discussion, and I'm not. But I
do want to agree publicly that there's no clash between brian and
myself. I have only the highest respect for brian: as a person, as an
educator, and as an expert on Perl. I respect his opinion (even
when--as now--I strongly disagree with it) and I respect his right to
hold and express that opinion.
It was a mistake to publicly express my private dismay. I apologize to
anyone who was hurt or upset by my having done so.
Damian
Brian D Foy
<brian...@gmail.com> wrote:
> There are other things to consider, and to me it looks like this design
> decision isn't based on what's easier for the Perl 6 programmer but
> what's easier for the implementors.
My comment here was offensive to Damian (and possibly others), and I
apologize for that.
I should have explicitly noted my context for this thought: some poor
programmer sitting alone with no knowledge of how Perl 6 got to be how
it is (that is, the audience for Learning Perl 6). I didn't mean to
apply this to Damian or anyone else specifically, and that we all know
each other makes my comment reasonably seem like I'm pointing at
specific people for this remark.
From this, Damian and I had a minor misunderstanding, and after a
couple of private emails we've cleared it up and are still friends. :)
I apologize for the inconvenience,
Chaddai Fouche
state that "if a line begin with "=" then it starts a POD section" is
way easier to understand than "a line beginning by = will start a POD
section except if it is in a Perl statement, or in a :to section, or
in a string literal, etc...". The "Learning Perl 6" argument seems
equally contrived to me since anyway you don't need POD to understand
programming in Perl and I never actually learned POD until I wanted to
do a real module and document my little console utilities in Perl. You
don't need to understand POD to read a program where POD is used :
it's usually quite clear from the content where it is POD and where it
is doc and each section that don't look like Perl is usually POD. Even
if you never heard of POD the first few samples would be a dead
giveaway that those weird "things" are actually documentation and not
code (it's my experience speaking here).
I seriously doubt most programmer will start including POD section in
confusing places because now they can do it, so the situation should
not be any different from before. And if some do it, hell, I seriously
doubt that their program would be in the scope of the beginning of
"Learning Perl 6" !! You didn't put "-+-" there in previous versions,
did you ?
The other "problem" is that if somehow a braindead guy (where would he
get the idea from, I never saw such a style) put his "=" in first
column expecting a assignment he won't get it... Seriously ? Are you
really allowing for such weirdness in introductory material to a
Language course ?
So in my opinion, it would be fine to let slip that you can also
create some kind of comment/doc by putting a = in the first column in
the first chapter, and let the subject of POD for a later chapter.
Of course some of you have far more experience teaching languages, but
as a language _student_ I don't feel this would be any inconvenience,
in fact I would have been very happy if all the Perl5 rules were that
easy.
--
Jedaï
Brian D Foy
<e9350eaf0706190959t269...@mail.gmail.com>, Chaddaï
Fouché <chaddai...@gmail.com> wrote:
> The "Learning Perl 6" argument seems
> equally contrived to me since anyway you don't need POD to understand
> programming in Perl and I never actually learned POD until I wanted to
> do a real module and document my little console utilities in Perl.
That's exactly my argument. You didn't need to learn that because you
weren't specifically doing anything with Pod. It doesn't even come up
because it's something you have to affirmatively do, and it's not
something that accidentally happens.
> And if some do it, hell, I seriously
> doubt that their program would be in the scope of the beginning of
> "Learning Perl 6" !! You didn't put "-+-" there in previous versions,
> did you ?
I'm not sure what you mean mean by "- + -".
Our current Learning Perl class starts off with a little Pod
manipulation because we know that everyone (should!) has the perldoc
along with their Perl. My actual experience trumps your serious doubt
:)
> The other "problem" is that if somehow a braindead guy (where would he
> get the idea from, I never saw such a style) put his "=" in first
> column expecting a assignment he won't get it... Seriously ? Are you
> really allowing for such weirdness in introductory material to a
> Language course ?
How is that braindead? It's perfectly fine, allowable, and intended
that a Perl 5 programmer can break statements over more than one line.
It's not weird at all.
I've taught a lot of beginner Perl classes, and people do all sorts of
things.
> So in my opinion, it would be fine to let slip that you can also
> create some kind of comment/doc by putting a = in the first column in
> the first chapter, and let the subject of POD for a later chapter.
The problem is that once you bring something up, people want to know
why you brought it up, and then they start playing with that point to
see what you meant. All of a sudden, you're explaining a lot of stuff
that doesn't get people any closer to completing a simple program.
As I've said previously, the rule for Pod looks simple, but the rules
for other things, such as strings, are now more complicated. However,
Larry mentioned that the Pod extractor may do what it likes, but Perl
shouldn't have to live with it's decisions about what is executable
code and what isn't, so it may still work out.
Smylers
> * Smylers (Smy...@stripey.com) [070616 08:44]:
>
> > With these new Pod rules it's possible to entirely remove Pod from a
> > file without knowing _anything_ about the host language. That
> > permits Pod to be used to document just about anything; all you need
> > to allow it is a filter that strips off all Pod before you do
> > anything else with the file.
>
> And then the main point: if you write documentation which is not
> related to Perl6 coding itself, do we really need to create just
> another text processor? There are already so many sofisticated
> text processors available!
Sure, but some people like Pod. Lots of things other than Perl 5 have
been documented with POD, so lets continue to make this kind of thing
easier.
> And why do you want easy to remove docs?
I didn't mean remove them for human (non-)consumption, but filtering
them out from, say, a shell script so that what remains is a
syntactically valid script which can be run in the shell.
> Other languages already have their own documentation system. Why do
> you expect them to use POD6?
Most people won't. But that's no reason to make it hard.
Smylers
Smylers
> * Jonathan Scott Duff (du...@pobox.com) [070616 20:15]:
>
> > You mention OOP. For Perl 5 we have a standard, if very general,
> > syntax and "open" semantics that have allowed people to implement
> > OOP in a variety of ways. This was all well and good for a while
> > until we realized that there should be some more reasonable defaults
> > (in both syntax and semantics) for common operations in OOP.
>
> OOP in Perl5 has a mechanism, but not a standard in the use of those
> features. Full OO languages all choose a direction, but there are
> serious culture differences. Perl uses them all. There are many ways
> how to instantiate an object, which is lot of fun as programmer but a
> hassle for "the average" programmer. Within one program, you may very
> well need to use different instantiation techniques... Of course,
> this could have been forseen (perl is not the first language which
> implements OO), and some advice on the convention to use could have
> avoided wild collection of approaches we see now.
But if this had been "forseen" and restrictions put in place, then we
wouldn't have had the recent advances in things like inside-out objects
and MOP. The flexibility and people doing different things allowed for
experimentation, and where those experiments were successful new ideas
gained community mindshare very quickly.
> This is also why "Perl Best Practices" is a good book [shameless plug]
Indeed. But it doesn't need to be part of the Perl core.
> although I would have welcomed it 11 years earlier.
But it extols many practices which weren't thought of 11 years
previously! Hence the need for flexibility, so as to allow time to
invent them ...
> > I think it's the same thing with POD6. It's "open" enough that many
> > documentation systems can be built from it (man pages, books,
> > magazines, wikis, etc.) For some of those documentation systems
> > we'll have nice conventions and other conventions will grow as
> > needed. If we find that convention isn't enough in specific areas,
> > ...
>
> Without any doubt, there are thousands of documentation systems
> around. At least ten percent of them are considered "the best ever
> made" by its developer or developer community. Just by simple math,
> the chance that the system developed really is the best is less than
> one percent.
Quite so -- which is why in my previous mail I thought it was
shortsighted to think we can create something now which we won't later
want to change.
> IMO, any argument that POD6 is good because it can be used to write
> books or express complex mathematical expressions is really frightning
> me.
POD (as in the version with Perl 5) wasnt designed for writing books or
whatever, yet people liked it enough they took it in directions that
nobody first expected. There's a good chance of that happening with Pod
6, even if it isn't an aim.
> Every single complication added to the doc syntax will make it not to
> understand for a large percentage of the primar target community, as
> every teacher can tell you from experience.
You could say the same thing about Perl 6 (or even earlier versions),
yet we cope, partly by ensuring that you don't need to learn all of it
in order to be able to use it. (In the case of Pod one of the things
that helps is its large overlap with plain text.)
> When I create a program, it starts with a goal. To reach that goal, I
> have to write some code, some docs, some tests. For me, it is all
> part of the program. I do not see one to be of more importance than
> the other: all three are first-class sitizens in my program.
Testing is another area which has had immense progress since the release
of Perl 5. Look at where Test::More, Test::Class, TAP::Parser and so on
are now; we certainly wouldn't want to be restricted to a standard of
best practices in testing from the early 1990s.
Smylers
Smylers
> In article <2007061608...@stripey.com>, Smylers
> <Smy...@stripey.com> wrote:
>
> > brian d foy writes:
> >
> > > In article
> > > <832f158a0706141722o5f...@mail.gmail.com>, Damian
> > > Conway <dam...@conway.org> wrote:
> > >
> > > > No. It's Pod. *Any* line that begins with '=begin' always starts a Pod
> > > > block. Always.
> > >
> > > As you know, one of the biggest complaints about Perl is that you have
> > > to have a lot of special rules knowledge to figure some things out.
> >
> > Indeed. What's much nicer is to be able to state that a given rule
> > "always" applies.
>
> Well, now explain literal strings :)
Fortunately Larry has found a way in which literal strings will still
work as you (and your students) expect.
> The rules for Pod5 always applied too, so I don't see what we've
> gained here as far as the simplicity of rules
Well the new is simpler.
> Putting aside this particular situation, the argument comes down to
> where does the water balloon pooch out? We haven't lost any
> complexity, it's just in different places. Maybe some places need less
> complexity and some places could stand a little more.
I agree with that.
I also think that this particular change is worth the risk of a learner
putting an assignment at the start of a line, especially given:
* With the exception for literal strings, this now pretty much only
applies to assignment.
* Because the C<=> has to be followed immediately by the Pod directive,
any C<=>s with space after them or assigning a quoted string will be
immune; only those assigning the return values of functions will be
affected.
That's slim. Even allowing for Sod's Law I would happily lead a
beginners' Perl course and chance not mentioning this exception
Smylers
Smylers
> I'm quite surprised by this debate... To me it seems a clear rule that
> state that "if a line begin with "=" then it starts a POD section" is
> way easier to understand than "a line beginning by = will start a POD
> section except if it is in a Perl statement, or in a :to section, or
> in a string literal, etc...".
It is. But that isn't the argument being made against it.
The point is that even knowing that simple rule is more complicated than
not having to know any rule at all. Even knowing that there _is_ a rule
requires knowing that Pod exists, which shouldn't be a prerequisite for
starting your first Perl program; you can't learn everything at once, in
parallel.
> The "Learning Perl 6" argument seems equally contrived to me since
> anyway you don't need POD to understand programming in Perl
Or rather, you didn't need that with Perl 5.
> and I never actually learned POD until I wanted to do a real module
> and document my little console utilities in Perl.
Sure. But the 'Learning Perl' argument is this one, which you mentioned
later in your message
> The other "problem" is that if somehow a braindead guy (where would he
> get the idea from, I never saw such a style) put his "=" in first
> column expecting a assignment he won't get it... Seriously ?
Yes. Personally I think that, on balance, the simplicity of the "what
is Pod?" rule is more of a gain than this awkwardness is a loss. But it
is a completely valid argument.
Remember that also until Larry's recent proposal, this affected string
constants too -- and somebody might have anything in a multi-line
string.
> Are you really allowing for such weirdness in introductory material to
> a Language course ?
The problem is that when teaching programming (I also used to be a Perl
5 trainer) you have to one part of the course first. And whatever that
is, it really should be usable without relying on material that hasn't
yet been covered.
This means that when teaching the very basics of Perl a trainer has to
do one of:
* State the exception that C<=> is special in the first column of the
source file -- thereby increasing the complexity of what users need to
grasp in one 'chunk' when first covering variables, and distracting
them from the core issue and sparking their curiosity with mention of
this documentation language.
* Not mention it, and hope that nobody formats a line in a way which
will be interpreted as Pod -- thereby causing much frustration in the
rare cases where this does happen, making the learner in question
irritated (with either the trainer for not mentioning it, or the
language for having this exception).
Neither of those are ideal.
> You don't need to understand POD to read a program where POD is used :
> it's usually quite clear from the content where it is POD and where it
> is doc and each section that don't look like Perl is usually POD.
Yes. But not if you've created unintentional Pod.
Smylers
[Apologies for the lag in this. I wrote 95% of the above 2 days ago --
before brian's similar message -- then got distracted and have only just
completed it. But I thought it worth mailing anyway, to point that even
though I've argued on the opposite side from brian on this, I completely
acknowledge he has a valid point.]
Smylers
> * Smylers (Smy...@stripey.com) [070616 09:09]:
> >
> > You're concerned that an aspect of Perl 6 might have too much
> > freedom? Isn't Perl all about giving users freedom to choose their
> > own way of doing something?
>
> Why treat documentation as a second-class citizen all the time?
I'm not suggesting we do; it's precisely _because_ Perl provides so much
freedom in things like coding style that I think it would be bizarre to
be so restrictive in the documenation format. Documentation should be a
first-class citizen, and therefore have as much freedom as coding.
> Why have a standard syntax for regexes, and not for docs?
We have a standard _syntax_ for docs; what you are additionally
proposing are standard _semantics_, which is something regeps (and other
areas of Perl) don't enforce -- they just provide the tools for
programmers to use them as they wish.
> Aren't you glad that at last we get a standard for OO programming and
> named parameters?
Yes. But I'm also glad that these have both carefully been done in ways
which don't enforce them, leaving programmers free to do things in other
ways.
> The boundary between freedom and anacharchy is faint.
Indeed. And I'd much rather we err on the side of anarchy. Many times
have I been frustrated by the limits of some software or programming
language. Far better to allow somebody enough freedom to create a poor
piece of software (since that's going to be possible anyway) than to go
too far in t'other direction and inadvertently prevent somebody in the
future from doing something really clever that none of us have yet
thought of.
> > Yes. But in reality many people will follow what others do, or look
> > to follow best practices. With Perl 5 you have complete freedom as
> > to the names of C<=head1> sections in the Pod for modules, yet in
> > browsing Cpan it's clear that there are conventions and many people
> > use the same headings. So not mandating a convention isn't much of
> > a problem.
>
> Well, the you are talking about the top three headers, the most.
Not true; there are several more which are shared between Cpan
distributions (by different authors).
> And those have a good example in standard UNIX manual-pages. So:
> there is a definitions for them, which most people have seen.
I am sceptical of your claim that most Cpan authors have seen any
official definitions of sections in Unix manual pages.
> > Do you really think that people can now, before Perl 6 has gained
> > anything approaching the usage we expect, make policy for how things
> > should be documented, such that that policy will be the best
> > possible way of documenting everything written in Perl 6, for ever?
> > Or even a good way?
>
> There is no need to think that a documentation syntax develops
> differently than a programming language.
I agree. And programming languages develop in hard-to-predict ways,
which is why Larry is putting lots of flexibility for things to be
redefined into Perl 6, so it can grow and adapt in line with future
developments. It would be good if Pod can keep up.
> So when Perl is developing, POD can develop in parallel.
Exactly!
> > That strikes me as incredibly shortsighted, verging on arrogance by
> > whoever comes up with the rules, and doomed to failure.
>
> Sorry? Not only you insult me,
Apologies, that wasn't supposed to be an insult. Note that it isn't
that I'm objecting to your rules (nor any particular rules), nor
doubting your abilities to come up with good rules; you'd be one of the
best people for coming up with some standards. I just don't believe
that _anybody_ can come up with rules that won't be bettered at some
point in the future.
> but you also ignore all these other languages which do have a nice and
> standard way of documenting. Insignificant languages, like Java,
> which we tend to ignore.
I've encountered Javadoc, and I really dislike it[*0]. The fact that a
standard exists does not make it a good one.
Smylers
[*0] Consider a function C<valid_postcode>. I'd document it along
the lines of:
valid_postcode
Returns whether the specified postcode is valid, for example:
if (valid_postcode $postcode) {
Javadoc-style systems seem to insist on documentation like:
valid_postcode
Description: Returns whether the specified postcode is valid.
Parameters:
$postcode: (string) The postcode to test for validity
Returns: (boolean) Whether $postcode is valid
Exceptions: none
Side Effects: none
Lots of structure and consistency, but in practical terms no
additional information compared to my informal doc -- and the
verbosity and repetition actually make it harder to spot the
information you want.
David Green
>Mark Overmeer writes:
> > The boundary between freedom and anacharchy is faint.
>Indeed. And I'd much rather we err on the side of anarchy.
I'd much rather we didn't err at all! Just because something isn't
"perfect" doesn't mean it's an error, or that it's not worth trying.
>[...] and inadvertently prevent somebody in the future from doing
>something really clever that none of us have yet thought of.
How does having some standards prevent anybody from doing anything?
Nobody's proposing Perl Documentation Police who will come and
kneecap you if you try to do something new and better.
> > And those have a good example in standard UNIX manual-pages. So:
>> there is a definitions for them, which most people have seen.
>I am sceptical of your claim that most Cpan authors have seen any
>official definitions of sections in Unix manual pages.
He only said they'd seen good examples. Which, really, are a kind of
standard. Even if all we did was to write down those standards
somewhere so that people weren't required to absorb them by osmosis,
that would be a good thing.
>I agree. And programming languages develop in hard-to-predict ways,
>which is why Larry is putting lots of flexibility for things to be
>redefined into Perl 6, so it can grow and adapt in line with future
>developments. It would be good if Pod can keep up.
Great, so let's have lots of rules and standards for documentation --
we'll just make sure they include formal ways to accommodate future
growth. Perl 6 is full of rules (it's software! all it is is a big
list of rules!), but it's still wonderfully flexible. I want lots of
doc-rules, but absolutely do I want them to be flexible too.
>I just don't believe that _anybody_ can come up with rules that
>won't be bettered at some point in the future.
Maybe; that doesn't mean we can't start off with some "really good"
rules today. Besides, insofar as that's true today, it will be just
as true in the future, so we'd be perpetually waiting for tomorrow's
Better Rules instead of actually doing something with today's Pretty
Good Rules.
>I've encountered Javadoc, and I really dislike it[*0]. The fact
>that a standard exists does not make it a good one.
Nope. But we're living in Javadoc's future, so let's learn from its
limitations and make P6doc better.
>[...]
> Exceptions: none
> Side Effects: none
>
>Lots of structure and consistency, but in practical terms no
>additional information compared to my informal doc -- and the
>verbosity and repetition actually make it harder to spot the
>information you want.
Well, clutter like "Blah: none" seems to me to be more the fault of
the doc-formatter for not hiding lines like that. But even though I
naturally tend to the lazy side myself, you have to admit there is
*some* info gained by requiring the "Blah: none" -- namely that there
really is no blah (as opposed to, "There might be, but I forgot to
say so.").
That makes me think of a practical example, though: Say you want to
find all the functions in your code that do have side-effects. Being
able to search for code that has an explicit "side-effects" tag with
a standard value of "none" (or undef or whatever) would be really
useful.
Actually, can't P6 tell whether code has side-effects or not? Which
is still a case for having structured docs; Perl could parse your
code and automatically insert a properly-formatted line to say so,
competently situated in the right place inside the rest of the
documentation you had to write manually. (And your local "style
sheet" could determine whether you see that line only when there are
side-effects, or only when there aren't, or never, or....)
The point isn't to have *only one* way to document your code; it's to
have *at least one*, that is, at least one way to build the FAD
(Fancy Advanced Documentation) tools of the future. I envision
things like "perldoc Test::BigHugeModule --sub=is" (sure beats
searching the page with ctrl-F "is"; and a pretty modest wish, I
think). Or perhaps I'm in the debugger, looking at the value of $foo
and wondering why it ==666 and whether that's a bad thing -- I could
hit a key and display the docs for that arg. Or I type "bar(" and my
text editor helpfully notices that I'm using the bar() function, asks
Perl for the specs, and pops up a handy summary of each parameter; it
could even see that bar() takes exactly one argument, of type
Mammal::African, and that I have exactly one such variable in scope,
$aardvark, and be ready to auto-complete it for me.
Actually, that last bit about detecting types in a signature or
variable is already possible, because Perl has formal rules for
specifying such things in your code; I'd like the same advantages
conferred on the documentation as well.
If you have a file with the text of the Magna Carta, a computer is
not going to understand it. You can add a bit of structure, like
headings and numbered lists, but we're a long way from software that
can interpret mediaeval politics. We do, however, have computers
that can interpret [and even compile =)] Perl... it has subs and
classes and lexicals and globals (and comments, and POD!). What a
shame it would be if all that structure and information that is
already available could not be put to good use by documentation tools.
Anyway, in case I'm sounding too polarised I should acknowledge that
I do agree with your underlying points about not overly restricting
things. I want rules *and* flexibility, and I believe we can have
both, just as Perl itself is very structured (more so than P5), but
is at the same time extremely flexible (also more so than P5!). I do
agree that docs should have as much freedom as code; I want them to
have as much structure too.
-David
Mark Overmeer
I fully agree with David's response to this mail. The only
thing I would like to add:
* Smylers (Smy...@stripey.com) [070621 18:02]:
> [*0] Consider a function C<valid_postcode>. I'd document it along
> the lines of:
>
> valid_postcode
>
> Returns whether the specified postcode is valid, for example:
>
> if (valid_postcode $postcode) {
>
> Javadoc-style systems seem to insist on documentation like:
>
> valid_postcode
> Description: Returns whether the specified postcode is valid.
> Parameters:
> $postcode: (string) The postcode to test for validity
> Returns: (boolean) Whether $postcode is valid
> Exceptions: none
> Side Effects: none
Of course, you can write horrible documentation in any syntax: that's
up to the authors. But now, just try to write above documentation in
the new POD6 syntax... in that case, it is not only horrible documentation,
but also 2 pages long.
In my idea, it suffices to write:
method isValidPostalCode(str $postalcode) returns bool {...}
By introspection during manual-page creation, it can collect
sufficient information to create this documentation item (controlled
by a (user-provided) template). With the POD back-end, something like
(blank lines removed)
=head1 METHODS
=over 4
=item $obj->isValidPostalCode(str $postalcode) returns bool
=back
Then, when you want to add some docs to the method, to help the
correct use, add it, for instance like:
method isValidPostalCode(str $postalcode) returns bool {...}
` Check wether the postal code confirms to the standards
`$postalcode: a string with blanks trimmed.
`return: the string is not cleaned-up.
or maybe (exact syntax open to discussion)
method isValidPostalCode(str $postalcode) returns bool {...}
#= Check wether the postal code confirms to the standards
#=$postalcode
#= a string with blanks trimmed.
#=return
#= the string is not cleaned-up.
or
method isValidPostalCode(str $postalcode) returns bool {...}
// Check wether the postal code confirms to the standards
//
// $postalcode a string with blanks trimmed.
// return the string is not cleaned-up, if you need
// that, call M<cleanupPostalCode()>.
or maybe at the bottom of your file, whatever you like
__DOC__
=doc isValidPostalCode
Check wether the postal code confirms to the standards
$postalcode a string with blanks trimmed.
return the string is not cleaned-up, if you need
that, call M<cleanupPostalCode()>.
There is so much user-friendliness to gain.
Very condensed documentation. Of course, you will get simple ways to
change the default mark-up of the parameters, for instance for the case
of MMD's, huge parameter lists, or where the parser cannot figure-out
info automatically.
--
Regards,
Damian Conway
> Then, when you want to add some docs to the method, to help the
> correct use, add it, for instance like:
>
> method isValidPostalCode(str $postalcode) returns bool {...}
> ` Check wether the postal code confirms to the standards
> `$postalcode: a string with blanks trimmed.
> `return: the string is not cleaned-up.
>
> or maybe (exact syntax open to discussion)
>
> method isValidPostalCode(str $postalcode) returns bool {...}
> #= Check wether the postal code confirms to the standards
> #=$postalcode
> #= a string with blanks trimmed.
> #=return
> #= the string is not cleaned-up.
Would the following syntax suffice?
method isValidPostalCode(str $postalcode) returns bool {...}
=PURPOSE Check weather the postal code confirms to the standards
=ARG $postalcode
a string with blanks trimmed.
=RETURN
the string is not cleaned-up.
Because you can already do precisely that in the current design of Pod 6.
And having done it, the current Pod::Parser will extract all of that
information (including the source code), preserving the sequential and
hierarchical relationships, and present you with an internal object
representation that your documentation tool can then analyze and
convert to whatever content and layout you prefer (including
non-semantic Pod that you can then feed into any of a dozen
back-ends).
Damian
Smylers
> Well, clutter like "Blah: none" seems to me to be more the fault of
> the doc-formatter for not hiding lines like that.
It's more the repetition in the lines you snipped that I really object
to: given the function's name, the name(s) of its parameter(s), and the
short description of what it does, it's obvious what the parameter(s)
are for and what it returns; repeating that information doesn't help
anybody.
(It may help computers, though.)
Smylers
Moritz Lenz
> Mark Overmeer wrote:
> Would the following syntax suffice?
>
> method isValidPostalCode(str $postalcode) returns bool {...}
> =PURPOSE Check weather the postal code confirms to the standards
> =ARG $postalcode
> a string with blanks trimmed.
> =RETURN
> the string is not cleaned-up.
>
> Because you can already do precisely that in the current design of Pod 6.
I really like that example.
And it makes me think that some of this discussion results from
insufficient Pod 6 knowledge, simply because we haven't read enough real
Perl 6 that is annotated with Pod 6.
So everybody reading this is now condemned to write an OO Perl 6 module,
and document it with Pod 6 ;-)
Mark Overmeer
> Mark Overmeer wrote:
> >Then, when you want to add some docs to the method, to help the
> >correct use, add it, for instance like:
> >
> > method isValidPostalCode(str $postalcode) returns bool {...}
> > ` Check wether the postal code confirms to the standards
> > `$postalcode: a string with blanks trimmed.
> > `return: the string is not cleaned-up.
> >
> >or maybe (exact syntax open to discussion)
> >
> > method isValidPostalCode(str $postalcode) returns bool {...}
> > #= Check wether the postal code confirms to the standards
> > #=$postalcode
> > #= a string with blanks trimmed.
> > #=return
> > #= the string is not cleaned-up.
>
> Would the following syntax suffice?
>
> method isValidPostalCode(str $postalcode) returns bool {...}
> =PURPOSE Check weather the postal code confirms to the standards
> =ARG $postalcode
> a string with blanks trimmed.
> =RETURN
> the string is not cleaned-up.
>
> Because you can already do precisely that in the current design of Pod 6.
This is syntactically what can be done with the current design,
but is semantically very different.
> And having done it, the current Pod::Parser will extract all of that
> information (including the source code), preserving the sequential and
> hierarchical relationships, and present you with an internal object
> representation that your documentation tool can then analyze and
> convert to whatever content and layout you prefer (including
> non-semantic Pod that you can then feed into any of a dozen
> back-ends).
Besides the point that I do not really like the YELLING all the time, it
still has the complication to reconstruct Perl code from the Pod::Parser
tree. The tools I wish for need to able to match the tree created by Perl
with the tree created by your Pod::Parser, in some safe and simple way.
The needs for my kind of documentation gerenators are very simple: the
Perl6 parser needs to collect all lines of docs which follow a certain
line with code into the AST node of that line or item. The Perl6
parser itself doesn't need to do anything more than that. Tools doing
introspection on the AST do the rest.
You gave the hint that comments are also in the parse tree. So, that
is suffiencent for me:
sub greetings() { say "Hello, World!" }
#= the standard first program <-- doc for user
# we like Perl6 new features <-- additional doc on internals.
or
#=sub greetings
#= the standard first program
sub greetings() { say "Hello, World!" }
# we like Perl6 new features <-- additional doc on internals.
[ I think that the doc block should always start with the item
what you are describing, so if the block is before the item (or
anywhere else in the file), then it needs to be referenced to (as
simple as possible) ]
[ now I have to study the introspection features of the AST ]
--
Mark Overmeer
> Damian Conway wrote:
> > Would the following syntax suffice?
> >
> > method isValidPostalCode(str $postalcode) returns bool {...}
> > =PURPOSE Check weather the postal code confirms to the standards
> > =ARG $postalcode
> > a string with blanks trimmed.
> > =RETURN
> > the string is not cleaned-up.
> >
> > Because you can already do precisely that in the current design of Pod 6.
>
> I really like that example.
> And it makes me think that some of this discussion results from
> insufficient Pod 6 knowledge, simply because we haven't read enough real
> Perl 6 that is annotated with Pod 6.
No, no... although you can create lines of text which are alike, the
whole meaning of these lines and needs for the processing tools behind
this is very different. Our discussion has always been about that:
how much info can the tools produce automatically: is POD6 integrated
with Perl6, or only (accidentally) sharing the same file.
--
Jonathan Lang
For the record, I find Damian's example to be considerably more readable.
> The needs for my kind of documentation generators are very simple: the
> Perl6 parser needs to collect all lines of docs which follow a certain
> line with code into the AST node of that line or item. The Perl6
> parser itself doesn't need to do anything more than that. Tools doing
> introspection on the AST do the rest.
Please forgive my ignorance: what does "AST" stand for?
--
Jonathan "Dataweaver" Lang
Mark Overmeer
> >> >Then, when you want to add some docs to the method, to help the
> >> >correct use, add it, for instance like:
> >> >
> >> > method isValidPostalCode(str $postalcode) returns bool {...}
> >> > ` Check wether the postal code confirms to the standards
> >* Damian Conway (dam...@conway.org) [070622 09:02]:
> >> method isValidPostalCode(str $postalcode) returns bool {...}
> >> =PURPOSE Check weather the postal code confirms to the standards
* Jonathan Lang (dataw...@gmail.com) [070622 10:41]:
> For the record, I find Damian's example to be considerably more readable.
Noted. Of course, it is just a syntactical difference, so not
really significant (not significant in the sense of our debate).
But, I think we should produce a few real code files and then compare
the visual friendliness of the various markup alternatives. CAPITALS
do not really attract me, but a design for Perl6 is not about me, but
about the average programmer (used to the DOS filesystem?)
> Please forgive my ignorance: what does "AST" stand for?
The Abstract Syntax Tree, the result of the code parser, un-interpreted.
--
Regards,
Damian Conway
>> Would the following syntax suffice?
>>
>> method isValidPostalCode(str $postalcode) returns bool {...}
>> =PURPOSE Check weather the postal code confirms to the standards
>> =ARG $postalcode
>> a string with blanks trimmed.
>> =RETURN
>> the string is not cleaned-up.
>>
>> Because you can already do precisely that in the current design of Pod 6.
>
> This is syntactically what can be done with the current design,
> but is semantically very different.
I don't think so. It's all just mark-up, no matter what the specific
syntax looks like. The semantics are provided by the behaviour of the tools
that parse and interpret that mark-up. If the tools treat:
=ARG $postalcode
a string with blanks trimmed.
the same as they treat:
`$postalcode: a string with blanks trimmed.
then semantically the two are exactly the same.
> Besides the point that I do not really like the YELLING all the time,
So you create a Pod module that defines "quieter" user-defined block names,
and write:
=use OOdoc:ver<6.0.0>
method isValidPostalCode(str $postalcode) returns bool {...}
=Purpose Check weather the postal code confirms to the standards
=Arg $postalcode
a string with blanks trimmed.
=Return
the string is not cleaned-up.
> You gave the hint that comments are also in the parse tree.
They can be. Better still, the (raw) Pod can also be kept in the parse
tree...since, like comments, the Perl parser still has to recognize it, even
when it's focusing on extracting Perl.
> So, that is sufficient for me:
And for others too, I hope.
Defining Perl 6 and Pod 6 independently opens up so many options for
building documentation tools:
* As you've observed, you can build them on top of the Perl 6
parser, using any mark-up syntax that will fit in a comment;
* And, as I've indicated, you can build them on top of
the Pod parser, using the standard Pod syntax;
* Or you can build them on top of the Perl 6 parser, but using the
standard Pod syntax...by parsing Pod from within the appropriate
nodes of the Perl AST using the Pod parser;
* Or you can build them on top of the Pod parser, using the standard
Pod syntax, by parsing any code-bearing "ambient" nodes within the
Pod DOM using the Perl 6 parser;
* Or you can even build them by using *both* parsers at once and
then walking the two resulting hierarchical representations (AST
and DOM) in parallel, since Perl 6 has very good support for such
concurrent tree traversals.
I don't believe any one of those alternatives will prove to be *the*
universal best approach for implementing all documentation tools, but I
do believe that having all those alternatives will make it as easy as
possible for us to collectively create the best tools for the each of
the many approaches to documentation that the Perl community is
ultimately going to want to support.
And that's what keeping Perl 6 and Pod 6 separate buys you: choice,
options, alternatives, the possibility of creating very different styles
of documentation for very different styles of programming. And for
different kinds of programmer: for the Busy Documentor, tools that
extract documentation automatically from code; for the Exacting
Documentor, a structural mark-up that allows precise manual control over
what's documented...and how; for the Pragmatic Documentor, tools that
allow structural mark-up and automatic extraction to be sensibly mixed.
And therefore I see the very fact that you don't like my design of Pod 6
as a strong argument in *favour* of that design, since that design aims
to provide the necessary syntactic extensibility and the essential
building blocks (parser, DOM, module support) required for the
proponents of each individual documentation philosophy to create a
mark-up system and supporting tools best suited to their specific needs
and objectives.
Damian
Jonathan Lang
> > You gave the hint that comments are also in the parse tree.
>
> They can be. Better still, the (raw) Pod can also be kept in the parse
> tree...since, like comments, the Perl parser still has to recognize it, even
> when it's focusing on extracting Perl.
...And since the Perl parser is required to include the Pod Parser (I
think), what gets included in the Perl parse tree for a Pod Section
will very likely be much more structured than what would get included
for a comment: the former will actually _have_ structure (and meaning)
which can be made available to introspection, while the latter will
continue to be merely an undifferentiated lump of text.
For example, I would expect semantic Pod Sections that appear within
class Foo's definition to be accessible via Foo.WHY, much like the
implementation details themselves show up in Foo.HOW.
> > So, that is sufficient for me:
>
> And for others too, I hope.
>
> Defining Perl 6 and Pod 6 independently opens up so many options for
> building documentation tools:
What sold me on it was Larry's suggestion that they only need to be
thought of separately from Pod's perspective. From Perl's
perspective, Pod can be thought of as part of the Perl language.
--
Jonathan "Dataweaver" Lang
Chas Owens
snipsnip
> > Please forgive my ignorance: what does "AST" stand for?
>
> The Abstract Syntax Tree, the result of the code parser, un-interpreted.
You mean it isn't Andrew S. Tanenbaum? Well, it would have been a lot
easier to write that parser then. You wouldn't believe how hard it is
to get a Perl script to output a Dutchman.
David Green
>David Green writes:
>> Well, clutter like "Blah: none" seems to me to be more the fault of
>> the doc-formatter for not hiding lines like that.
>
>It's more the repetition in the lines you snipped that I really object to:
Ah. (That was sneaky of me.) I agree with that -- the older I get,
the more allergic I become to typing the same thing more than once.
(That's why we like Perl, for it's DRY wit!)
>given the function's name, the name(s) of its parameter(s), and the
>short description of what it does, it's obvious what the
>parameter(s) are for and what it returns; repeating that information
>doesn't help anybody.
>(It may help computers, though.)
I'm wholly in favour of some syntactic sugar so such repetition isn't
necessary (which doesn't mean redefining Pod, of course, but rather
coming up with some alternative that would work this way).
-David