Re: Announcing the Perl 5 Documentation Team
Ævar Arnfjörð Bjarmason
> I am excited to announce the formation of the Perl 5 Documentation
> Team. Our goal is to have the best, most current, and easiest to use
> and understand documentation of any programming language. Why settle
> for small goals?
It's great that this is happening. And I hope that in general that
changes that this team makes get the benefit of the doubt. It's
somewhat important that everyone get a say and that every detail is
covered, but even more important IMO that our documentaion is being
*maintained*, which is hardly the case now.
> Goals to be met for the Perl 5.14.0 release include:
\o/
> * Extensive review and Modernization of all examples
One thing I'd like to suggest. I think it'd improve our code examples
a lot if we used a consistent style for them. By that I mostly mean
not coding style (we've had that giant flamewar), but the sort of
thing you find in "conventions used in this documentation" sections.
E.g. consistently showing the return values of expressions like the
Ruby and Python documentation do:
http://ruby-doc.org/docs/ProgrammingRuby/html/tut_classes.html
It's a lot easier to skim stuff like that than read I<The last call
for this function returns xyz> or something similar in prose.
> * Develop a plan to integrate with the [Pod2 translation project][4]
It'd be great if we could get multilingular documentation of the
ground. I think that for this to be fruitful we'd need a translation
system so that translated documentation doesn't suffer
bitrot. E.g. something similar to what translatewiki.org does to
translate content, where things are split into paragraphs or sections
and then assembled back again.
Is there something like that already that can be used or improved?
Chas. Owens
snip
>> * Updating the documentation so that all examples longer than one
>> line contain a `use` statement indicating the minimum version of
>> Perl 5 required to run the example
>
> I'm afraid that I don't think this is a sensible suggestion at all. I
> took a look at a few pages, almost at random. I looked at perldata,
> perlfaq3, perllol and perlcall. Adding C<use 5;>, as it would often be,
> a dozen or more times in a document adds little value but greatly
> detracts from the flow of the document.
>
> Is this really a problem. Are there many people out there trying to use
> qr// on 5.003, for example? I can see some value in noting that C<when>
> as a statement modifier is unavailable before 5.12, for example, but as
> a general policy I feel it is misguided. Please reconsider this idea.
snip
Failure to add a use 5.010; or use 5.012; limits the features that can
be used in examples. For instance, this example won't work even under
5.12 as it is written:
say "foo";
It is possible that we should only add use statements to examples that
need features in 5.10 or later.
--
Chas. Owens
wonkden.net
The most important skill a programmer can have is the ability to read.
Chas. Owens
snip
>> * Extensive review and Modernization of all examples
>
> One thing I'd like to suggest. I think it'd improve our code examples
> a lot if we used a consistent style for them. By that I mostly mean
> not coding style (we've had that giant flamewar), but the sort of
> thing you find in "conventions used in this documentation" sections.
>
> E.g. consistently showing the return values of expressions like the
> Ruby and Python documentation do:
> http://ruby-doc.org/docs/ProgrammingRuby/html/tut_classes.html
One of the things I have been doing is using a comment after a line
stating what the result of line is.
snip
>> * Develop a plan to integrate with the [Pod2 translation project][4]
>
> It'd be great if we could get multilingular documentation of the
> ground. I think that for this to be fruitful we'd need a translation
> system so that translated documentation doesn't suffer
> bitrot. E.g. something similar to what translatewiki.org does to
> translate content, where things are split into paragraphs or sections
> and then assembled back again.
>
> Is there something like that already that can be used or improved?
snip
These are the sort of things that cause this to be a long term goal.
One of my first thoughts was that we should just machine translate the
whole of the documentation into a bunch of languages and then let
people apply patches to correct the hilariously bad machine
translations.
Chas. Owens
> On Thu, Jul 1, 2010 at 7:11 PM, Chas. Owens <chas....@gmail.com> wrote:
>> Failure to add a use 5.010; or use 5.012; limits the features that can
>> be used in examples. For instance, this example won't work even under
>> 5.12 as it is written:
>>
>> say "foo";
>>
>> It is possible that we should only add use statements to examples that
>> need features in 5.10 or later.
>
> and only "intro" level material needs to be pedantic and include the
> "standard" pragmata list. (That could be one for your style guide --
> anything perl*intro|tut|quick should have them, everything else should
> not.)
>
> I just don't think every snippet of example needs to have "use 5.014"
> or "use strict" or "use warnings" or whatever. It just clutters the
> examples. Certainly, the introduction section of documentation files
> could make a point to say that examples assume those three pragmas,
> but the main content of the documents (even ones accessed via -f like
> perlfunc) don't need to be that pedantic if omitting make the
> documentation more streamlined.
>
> To the extent that certain features of something being documented are
> only available as of Perl version N, that could be noted in the text.
>
> -- David
>
It isn't just that say wasn't added until 5.10, it is that say doesn't
even work without a "use 5.010;" or similar statement. This means
that the person coming to the Perl for the first time will encounter
one of three scenarios:
1. None of the examples use features that are turned on by pragmas
2. Some of the examples use features that are turned on by pragmas,
but the pragmas are not visible in the example, so when the novice
tries to run the example he or she gets an error message that is not
very helpful ("Undefined subroutine &main::say called" not "To use the
say function use the feature pragma or add 'use 5.012;' to your
program"). This leads to confusion and thinking that the
documentation has broken examples.
3. Every non-trivial example contains a use statement that lets you
know what version of Perl this code will run on. This has the handy
side effect of turning on the features available in that version of
Perl.
While the third scenario is somewhat annoying to the more advanced
users, it is a lifesaver for novices.
A fourth scenario where each document has an intro section that
explains that the examples will be run under the following X pragmas
would be better than scenario 2, but only marginally. In my
experience, people just don't read those introductory statements. So,
all that would do is give experienced users the opportunity to say
"Did you read the introductory material? Well, no wonder the examples
didn't work, you need to add these X lines. The document tells you
that at the top." Adding an introductory section will also not help
people who are using perldoc -q, -f, -v, and the possible -O (for
operator help) or Padre's context sensitive help, since those only
spit out the section being referenced.
A fifth scenario where there is an introductory statement that
examples will work on 5.8 or later unless otherwise marked and then
pragmas are added to the examples that need features like say would
also work quite well. Heck, you wouldn't even need the introductory
statement. This really isn't about what versions of Perl the example
works on, it is about being able to use the features that are hidden
because of backwards compatibility concerns.
Chas. Owens
> On Fri, Jul 2, 2010 at 8:22 AM, Chas. Owens <chas....@gmail.com> wrote:
>> It isn't just that say wasn't added until 5.10, it is that say doesn't
>> even work without a "use 5.010;" or similar statement. This means
>> that the person coming to the Perl for the first time will encounter
>> one of three scenarios:
>
>
> The vast majority of the time, the documentation is being read by
> people who are *NOT* "coming to Perl for the first time" and thus I
> don't think that *all* documentation should be written for the
> absolute beginner.
>
> Intro-level documentation ("intro", "tutorial", "quick start",
> possibly "faq") should. Reference documentation should not. IMO,
> that includes things that would come from perldoc -f, etc.
>
> Yes, we want to make Perl easier to learn for the novice, but not at
> the cost of bloating the documentation for the average user.
>
> -- David
>
Lets take a quick look at what it would really look like. I chose
substr at random and updated it to fit scenario 3. It took four
version statements and a rewrite of the last example. I am not
satisfied with the last example, it is still very confusing. there
must be a better example of when you would use this feature of substr.
Also, and this shows my ignorance of the newer features, why doesn't
given work with that example? I thought it was supposed to alias $_
to its argument like for does.
Proposed style:
=item substr EXPR,OFFSET
Extracts a substring out of EXPR and returns it. First character is at
offset C<0>, or whatever you've set C<$[> to (but don't do that).
If OFFSET is negative (or more precisely, less than C<$[>), starts
that far from the end of the string. If LENGTH is omitted, returns
everything to the end of the string. If LENGTH is negative, leaves that
many characters off the end of the string.
use 5.005;
my $s = "The black cat climbed the green tree";
my $color = substr $s, 4, 5; # black
my $middle = substr $s, 4, -11; # black cat climbed the
my $end = substr $s, 14; # climbed the green tree
my $tail = substr $s, -4; # tree
my $z = substr $s, -4, 2; # tr
You can use the substr() function as an lvalue, in which case EXPR
must itself be an lvalue. If you assign something shorter than LENGTH,
the string will shrink, and if you assign something longer than LENGTH,
the string will grow to accommodate it. To keep the string the same
length, you may need to pad or chop your value using C<sprintf>.
If OFFSET and LENGTH specify a substring that is partly outside the
string, only the part within the string is returned. If the substring
is beyond either end of the string, substr() returns the undefined
value and produces a warning. When used as an lvalue, specifying a
substring that is entirely outside the string raises an exception.
Here's an example showing the behavior for boundary cases:
use 5.005;
my $name = 'fred';
substr($name, 4) = 'dy'; # $name is now 'freddy'
my $null = substr $name, 6, 2; # returns "" (no warning)
my $oops = substr $name, 7; # returns undef, with warning
substr($name, 7) = 'gap'; # raises an exception
An alternative to using substr() as an lvalue is to specify the
replacement string as the 4th argument. This allows you to replace
parts of the EXPR and return what was there before in one operation,
just as you can with splice().
use 5.005;
my $s = "The black cat climbed the green tree";
my $z = substr $s, 14, 7, "jumped from"; # climbed
# $s is now "The black cat jumped from the green tree"
Note that the lvalue returned by the 3-arg version of substr() acts as
a 'magic bullet'; each time it is assigned to, it remembers which part
of the original string is being modified; for example:
use 5.010;
my $x = '1234';
for my $temp (substr($x,1,2)) {
$temp = 'a'; say $x; # prints 1a4
$temp = 'xyz'; say $x; # prints 1xyz4
$x = '56789';
$temp = 'pq'; say $x; # prints 5pq9
}
Prior to Perl version 5.9.1, the result of using an lvalue multiple times was
unspecified.
Versus the original
=over 8
=item substr EXPR,OFFSET,LENGTH,REPLACEMENT
X<substr> X<substring> X<mid> X<left> X<right>
=item substr EXPR,OFFSET,LENGTH
=item substr EXPR,OFFSET
Extracts a substring out of EXPR and returns it. First character is at
offset C<0>, or whatever you've set C<$[> to (but don't do that).
If OFFSET is negative (or more precisely, less than C<$[>), starts
that far from the end of the string. If LENGTH is omitted, returns
everything to the end of the string. If LENGTH is negative, leaves that
many characters off the end of the string.
my $s = "The black cat climbed the green tree";
my $color = substr $s, 4, 5; # black
my $middle = substr $s, 4, -11; # black cat climbed the
my $end = substr $s, 14; # climbed the green tree
my $tail = substr $s, -4; # tree
my $z = substr $s, -4, 2; # tr
You can use the substr() function as an lvalue, in which case EXPR
must itself be an lvalue. If you assign something shorter than LENGTH,
the string will shrink, and if you assign something longer than LENGTH,
the string will grow to accommodate it. To keep the string the same
length, you may need to pad or chop your value using C<sprintf>.
If OFFSET and LENGTH specify a substring that is partly outside the
string, only the part within the string is returned. If the substring
is beyond either end of the string, substr() returns the undefined
value and produces a warning. When used as an lvalue, specifying a
substring that is entirely outside the string raises an exception.
Here's an example showing the behavior for boundary cases:
my $name = 'fred';
substr($name, 4) = 'dy'; # $name is now 'freddy'
my $null = substr $name, 6, 2; # returns "" (no warning)
my $oops = substr $name, 7; # returns undef, with warning
substr($name, 7) = 'gap'; # raises an exception
An alternative to using substr() as an lvalue is to specify the
replacement string as the 4th argument. This allows you to replace
parts of the EXPR and return what was there before in one operation,
just as you can with splice().
my $s = "The black cat climbed the green tree";
my $z = substr $s, 14, 7, "jumped from"; # climbed
# $s is now "The black cat jumped from the green tree"
Note that the lvalue returned by the 3-arg version of substr() acts as
a 'magic bullet'; each time it is assigned to, it remembers which part
of the original string is being modified; for example:
$x = '1234';
for (substr($x,1,2)) {
$_ = 'a'; print $x,"\n"; # prints 1a4
$_ = 'xyz'; print $x,"\n"; # prints 1xyz4
$x = '56789';
$_ = 'pq'; print $x,"\n"; # prints 5pq9
}
Prior to Perl version 5.9.1, the result of using an lvalue multiple times was
unspecified.
=back
Shawn H Corey
> Proposed style:
>
> =item substr EXPR,OFFSET
>
> Extracts a substring out of EXPR and returns it. First character is at
> offset C<0>, or whatever you've set C<$[> to (but don't do that).
> If OFFSET is negative (or more precisely, less than C<$[>), starts
> that far from the end of the string. If LENGTH is omitted, returns
> everything to the end of the string. If LENGTH is negative, leaves that
> many characters off the end of the string.
What the heck is LENGTH?
--
Just my 0.00000002 million dollars worth,
Shawn
Programming is as much about organization and communication
as it is about coding.
The secret to great software: Fail early & often.
Eliminate software piracy: use only FLOSS.
Chas. Owens
> On 10-07-02 09:30 AM, Chas. Owens wrote:
>>
>> Proposed style:
>>
>> =item substr EXPR,OFFSET
> What the heck is LENGTH?
My copy and paste left out
=item substr EXPR,OFFSET,LENGTH,REPLACEMENT
X<substr> X<substring> X<mid> X<left> X<right>
=item substr EXPR,OFFSET,LENGTH
In the first example.
Gabor Szabo
> On Fri, Jul 2, 2010 at 8:22 AM, Chas. Owens <chas....@gmail.com> wrote:
>> even work without a "use 5.010;" or similar statement. This means
>> that the person coming to the Perl for the first time will encounter
>> one of three scenarios:
>
>
> The vast majority of the time, the documentation is being read by
> people who are *NOT* "coming to Perl for the first time" and thus I
> don't think that *all* documentation should be written for the
> absolute beginner.
>
> Intro-level documentation ("intro", "tutorial", "quick start",
> possibly "faq") should. Reference documentation should not. IMO,
> that includes things that would come from perldoc -f, etc.
>
> Yes, we want to make Perl easier to learn for the novice, but not at
> the cost of bloating the documentation for the average user.
Interestingly I would have said that the documentation is mostly needed
by novices or by people who rarely use Perl. The difference might be
because I encounter so many people who are learning Perl.
What they usually would need, IMHO is an easy way to grab an example
of simple task.
"How can I open a file?"
or even
"How can I read from file?"
There are also many people familiar with other languages and they are looking
for solutions based on their language:
"How can I trim white spaces?"
or
"What is the equivalent of the 'trim' function?"
Then there are the people who inherit a piece of code without any
background in Perl.
They would need to get answer to the questions "What is this $_ ?",
"What is $_[0] ?" or
"What is ||= ?".
I don't know much about experts but I think we should make it easy for
people with no,
or little background in Perl to get simple answers to their questions
with pointers to
further details. I think the vast majority of people using Perl are in
this category.
Gabor
Ævar Arnfjörð Bjarmason
> Good luck with this. I'll just mention one thing that has bothered me since
> the documentation got all improved when moving from perl4 to perl5:
>
> The perl4 doc was all one giant man page, which made it really simple
> to search. It would be nice if there was a single document designed
> to be searched easily that at least had pointers to the right document
> where you'll find details of what you are searching for (in particular
> I never have any idea if something like -M is a "function" or a "operator" :-).
On my infinite todo list I have an item for adding a perl5 POD -> GNU
Texinfo conversion script. Then you can generate indexed docs you can
easily search through, dump the whole thing as a big man/HTML/PDF etc.
Or you could just concat all the existing core manpages now to get some of that.
Caleb Cushing
> be used in examples. For instance, this example won't work even under
> 5.12 as it is written:
>
> say "foo";
>
> It is possible that we should only add use statements to examples that
> need features in 5.10 or later.
>
use features 'say';
somewhere on the page
I guess it says...
This keyword is available only when the "say" feature is enabled: see feature.
--
Caleb Cushing
Caleb Cushing
> Interestingly I would have said that the documentation is mostly needed
> by novices or by people who rarely use Perl. The difference might be
> because I encounter so many people who are learning Perl.
depends on /what/ documentation you're talking about. you're talking
language basics documentation... however documentation goes way beyond
the basics. You really mean to say that you go look at the code every
time you need to know how to use a sub that you haven't used before
(or perhaps recently)?
# grumble about having to remember to use reply to all
Chas. Owens
This is mostly handled by perlfaq/perldoc -q QUESTION today. Long
term projects I am considering include a rosetta stone for
ruby/python/? and a cookbook (but this is already what the FAQs are to
a certain extent, si I need to think more about that).
Gabor Szabo
No, I meant to say that the people I encounter do that.
Gabor
Chas. Owens
That is in say's part of perlfunc, I am talking about being able to
use say in examples for other functions, operators, etc. For
instance, in substr it says
$x = '1234';
for (substr($x,1,2)) {
$_ = 'a'; print $x,"\n"; # prints 1a4
$_ = 'xyz'; print $x,"\n"; # prints 1xyz4
$x = '56789';
$_ = 'pq'; print $x,"\n"; # prints 5pq9
}
We could cut down the noise somewhat by saying
my $x = '1234';
for (substr($x,1,2)) {
$_ = 'a'; say $x; # prints 1a4
$_ = 'xyz'; say $x; # prints 1xyz4
$x = '56789';
$_ = 'pq'; say $x; # prints 5pq9
}
But now the example won't run in Perl 5.12. The example needs
use 5.010;
my $x = '1234';
for (substr($x,1,2)) {
$_ = 'a'; say $x; # prints 1a4
$_ = 'xyz'; say $x; # prints 1xyz4
$x = '56789';
$_ = 'pq'; say $x; # prints 5pq9
}
And this is a good idea anyway because this feature (substr's lvalue
being persistant) doesn't work in Perl 5.8.
Chas. Owens
> On Fri, Jul 2, 2010 at 9:30 AM, Chas. Owens <chas....@gmail.com> wrote:
>> Lets take a quick look at what it would really look like. I chose
>> substr at random and updated it to fit scenario 3. It took four
>
>
> * exec
> * localtime
> * split
>
> Below, I'll do an except bit of the latter. Consider these possible reactions:
>
> * What is "use 5.010"?
> * You mean I have to type "use 5.010" whenever I use "split"?
> * Why is it sometimes "use 5.010" and sometimes "use 5.005"?
>
> None of these have anything do with understanding 'split' -- so I
> don't think it adds value.
>
> Here is my example from 'split' in perlfunc:
> ...
> use 5.010;
> say join(':', split(/ */, 'hi there'));
>
> use 5.010;
> say join(':', split(//, 'hi there'));
>
> use 5.010;
> say join(':', split(/(?=\w)/, 'hi there!'));
>
> use 5.010;
> say join(':', split(//, 'hi there!', -1));
> say join(':', split(/\W/, 'hi there!', -1));
>
> use 5.005;
> ($login, $passwd, $remainder) = split(/:/, $_, 3);
>
> use 5.005;
> split(/([,-])/, "1-10,20", 3);
snip
The use statements should be explained at the top of the document (I
know I already argued that nobody reads those, but they are important
for explain things like style), and most of those examples would not
have a use statement under my proposal. Only examples that are longer
than one line are targeted for use statements. Also, most of them
could be rephrased to avoid the print or say in the first place
(removing the most pressing need for a use statement).
Shawn H Corey
> I don't know much about experts but I think we should make it easy for
> people with no,
> or little background in Perl to get simple answers to their questions
> with pointers to
> further details. I think the vast majority of people using Perl are in
> this category.
When experienced programmers make mistakes it's usually because they
assume the code is doing one thing when it's doing something else. For
example, not remembering that shift does not shift @_ outside a sub.
Since their confidence in their assumption is very high, they won't look
up what shift does until it is an act of desperation. That is, they
won't do it until they tried everything else they can think of and start
systematically reading the documentation for everything in the code.
To make things easier, perhaps a sub-section called "Special Cases"
might be added but if so, then remember that they will skip the rest of
the documentation and jump straight to it. That means it must be
comprehensive (it has to list every special case) though it need not be
thorough (it can simply say "see above").
Gabor Szabo
let me be even more specific. They will look it up in Google.
Chas, I am not sure:
$ perldoc -q trim
No documentation for perl FAQ keyword `trim' found
# perldoc -q open
gives a lot of FAQs but none of them gives the simple case
of
open my $fh, '<', $filename or die $!;
while (my $line = <$fh>) {
}
Though this is 5.10.1 here so it might have changed since then or I might
have misunderstood your comment.
Gabor
Chas. Owens
snip
> Chas, I am not sure:
>
> $ perldoc -q trim
> No documentation for perl FAQ keyword `trim' found
>
> # perldoc -q open
> gives a lot of FAQs but none of them gives the simple case
> of
>
> open my $fh, '<', $filename or die $!;
> while (my $line = <$fh>) {
> }
>
>
> Though this is 5.10.1 here so it might have changed since then or I might
> have misunderstood your comment.
No, you understood me, it is just that the FAQ doesn't do a good job
for those two questions. That is something we need to be doing
better.
brian d foy
<AANLkTimm-hIuF86eV9YkY...@mail.gmail.com>, Chas.
Owens <chas....@gmail.com> wrote:
> No, you understood me, it is just that the FAQ doesn't do a good job
> for those two questions. That is something we need to be doing
> better.
The FAQ doesn't do a good job of teaching you Perl because it's not the
FAQs job to teach you Perl. The FAQ is not there to teach you syntax or
programming.
As for the particular comment that's started this long thread, the idea
that we should add 'use XXX' to every snippet, I have two rule
* The documentation for a particular version of perl assumes that perl
is used for its examples.
* Never use features that distract the reader. Using 'use 5.010' just
to get say() is a distraction unless you are talking about say()
specifically.
* Only note the version when it really matters. For instance, you have
to somehow pull in the Switch statement.
* Note that since 5.10, 'use 5.xxx' now has side effects. Using that
blindly may cause other things in a program to break, although the
readers may not know why.
* Ignore any Perl released more than 10 years ago.
Chas. Owens
> In article
> <AANLkTimm-hIuF86eV9YkY...@mail.gmail.com>, Chas.
> Owens <chas....@gmail.com> wrote:
>
>> No, you understood me, it is just that the FAQ doesn't do a good job
>> for those two questions. That is something we need to be doing
>> better.
>
> The FAQ doesn't do a good job of teaching you Perl because it's not the
> FAQs job to teach you Perl. The FAQ is not there to teach you syntax or
> programming.
I think "How do I open a file?" and "How do I trim a string?" are
frequently asked questions. Certainly they are asked more often than
"How can I open a filehandle to a string?".
snip
> As for the particular comment that's started this long thread, the idea
> that we should add 'use XXX' to every snippet, I have two rule
>
> * The documentation for a particular version of perl assumes that perl
> is used for its examples.
snip
Yes an no. Between perldoc.perl.org defaulting to the latest version
and many production versions lagging behind what may be installed on
development machines, the version of Perl you are targeting might not
be the same as the version of Perl you are running (e.g. I have Perl
5.12.1 on my machine, but production is 5.8.8).
snip
> * Never use features that distract the reader. Using 'use 5.010' just
> to get say() is a distraction unless you are talking about say()
> specifically.
snip
For the most part I agree with this. In fact, I dislike print
statements in examples in general.
snip
> * Only note the version when it really matters. For instance, you have
> to somehow pull in the Switch statement.
snip
This is pretty much what my amended position is (the fifth scenario in
my earlier email). If the example needs a specific version, or the
feature only works on a specific version, then the pragma that turns
on the feature should be present. Similarly, if we are going to use a
function from a module, we should use the module.
snip
> * Note that since 5.10, 'use 5.xxx' now has side effects. Using that
> blindly may cause other things in a program to break, although the
> readers may not know why.
snip
I am actually counting on those side effects being present. On this
same topic, I have noticed that there doesn't seem to be an easily
discoverable way to find out what "use <VERSION>;" does. We are
training users to type "perldoc <thing after use>" to find out what
"<thing after use>" does. Typing "perldoc 5.012" yields 'No
documentation found for "5.012".'. Instead you have to type "perldoc
-f use" and then "perldoc feature". Should we add a symlink to the
delta for that version, or maybe a new document for each version that
explains what the consequences of specifying that version are?
snip
> * Ignore any Perl released more than 10 years ago.
snip
I think we are in agreement here.