What should Python apps do when asked to show help?

[Steven D'Aprano]

I have an application written in Python which accepts -h or --help to show
help. I can:

(1) print the help text to stdout;

(2) run the help text through a pager;

(3) do something else?


Many command line tools simply output help to stdout (or stderr, if they're
evil), which makes it easy to redirect the help to a file, pass it to grep,
etc. For example:

[steve@ando ~]$ wget --help | wc -l
136

Other tools automatically launch a pager, e.g. man. (Admittedly, man --help
does not use a pager.) The Python help system, pydoc, does the same.

I was thinking that my application could use pydoc:

def do_help():
import pydoc
pydoc.pager(HELPTEXT)

or just print the help text:

def do_help():
# Do I really need to show this???
print(HELPTEXT)


but I was thinking of doing both: give my application a subcommand or an
option to display help directly in a pager, while -h and --help print to
stdout as normal.

What do you think? Too clever?



--
Steven

[alister]

Send it to stdout, this gives the user the most choice.

if there is enough that I want it paged then I will pipe it to a pager.

do one job, do it well and don't reinvent the wheel unnecessarily.



--
Nobody shot me.
-- Frank Gusenberg, his last words, when asked by police
who had shot him 14 times with a machine gun in the Saint
Valentine's Day Massacre.

Only Capone kills like that.
-- George "Bugs" Moran, on the Saint Valentine's Day
Massacre

The only man who kills like that is Bugs Moran.
-- Al Capone, on the Saint Valentine's Day Massacre

[Ethan Furman]

On 04/28/2016 09:33 AM, Steven D'Aprano wrote:

> I have an application written in Python which accepts -h or --help to show
> help. I can:
>
> (1) print the help text to stdout;
>
> (2) run the help text through a pager;
>
> (3) do something else?

I think if the user is proficient enough to:

a) run the program from the command line, and

b) use the -h / --help switch

that they are proficient enough to efficiently use said help when output
to stdout.

--
~Ethan~

[Dan Strohl]

I would suggest using argparse https://docs.python.org/3/library/argparse.html as it handles all of that natively... including validating arguments, showing errors, help, etc... however, assuming you don't want to;

Send it to stdout, that allows the user to redirect it if they want to (and plays better with other dev-ops tools)

Don't run it through a pager, the user can do that if needed... HOWEVER
- If you have enough help that you need to page it, consider moving much of it to application documentation (either hosted, or as a doc / txt file). The command line help should simply be enough hints to remember what the various parameters are, not a full explanation about what it can do (unless the app is really simple).
- Alternatively, you can have a multi-level help approach where the user can type "app_name argument /help" and get more detailed information about that specific argument.

> --
> https://mail.python.org/mailman/listinfo/python-list

[John Wong]

On Thu, Apr 28, 2016 at 1:02 PM, Dan Strohl via Python-list <
pytho...@python.org> wrote:

> I would suggest using argparse
> https://docs.python.org/3/library/argparse.html as it handles all of that
> natively... including validating arguments, showing errors, help, etc...
> however, assuming you don't want to;
>

> Totally agree with this approach. Command line should stick with argparse.
Personally I'd stick with argparse and not other open source projects which
is built on argparse (or optparse, the one you don't want to use, but eh
some people decided to do that anyway because of some limitations in
argparse).

In fact you shouldn't need to implement -h/--help when you use argparse. If
a user is going to use the command line, you can almost always assume the
user can use -h/--help, or for those familiar with Linux just provide a man
page.

After all, what you need is a very clear documentation upfront prior to the
installation so your users can refer to that for ultimate help.

[Chris Angelico]

On Fri, Apr 29, 2016 at 2:33 AM, Steven D'Aprano <st...@pearwood.info> wrote:
> I have an application written in Python which accepts -h or --help to show
> help. I can:
>
> (1) print the help text to stdout;
>
> (2) run the help text through a pager;
>
> (3) do something else?
>
>
> Many command line tools simply output help to stdout (or stderr, if they're
> evil),

I'm not sure stderr is particularly more evil than stdout, but whatever :)

What you could do is run the help text through a pager only if stdout
is a TTY. That allows it to be grepped conveniently, but also browsed
easily. But I wouldn't worry about that unless you have at least 3-5
pages of screed - for most programs, just dumping the text to a
standard stream is usually enough.

ChrisA

[Irmen de Jong]

On 28-4-2016 18:33, Steven D'Aprano wrote:


> but I was thinking of doing both: give my application a subcommand or an
> option to display help directly in a pager, while -h and --help print to
> stdout as normal.
>
> What do you think? Too clever?

An idea: Use just one help option, then

if sys.stdout.isatty():
#....use a pager to display help text
else:
#....print all help text normally


Irmen

[Dan Strohl]

Yeah, if I am handling arguments from the command line, I use argparse, if I am doing a cli based app (so, going back and forth in interacting with the command line), I would look at clint (https://github.com/kennethreitz/clint)

As many people have said here, don’t reinvent the wheel.

Dan

From: John Wong [mailto:gokop...@gmail.com]
Sent: Thursday, April 28, 2016 10:06 AM
To: Dan Strohl <D.St...@F5.com>
Cc: alister <aliste...@ntlworld.com>; pytho...@python.org
Subject: Re: What should Python apps do when asked to show help?

[Ethan Furman]

On 04/28/2016 10:02 AM, Dan Strohl via Python-list wrote:

> I would suggest using argparse https://docs.python.org/3/library/argparse.html as it handles all of that natively...

On the other hand, if you feel that argparse is akin to using a canon to
kill a mosquito, you can try scription*:

- not argparse based
- simple to use
- supports flags, options, multioptions
- supports --help (and -h if no other parameter uses that abbreviation)
- supports --verbose (and -v of no other . . .)
- supports --version
- script parameters are global
- (sub)command parameters are local

--
~Ethan~

* https://pypi.python.org/pypi/scription

P.S. Yes, my package. ;) Created both for the learning, and because I
feel like argparse is overpowered, complicated, and un-fun for scripts.

[Jussi Piitulainen]

Keep --help, -h reasonably short and print it to stdout. That's what I
expect and then I know what to do with it.

An actual man page is the obvious place for more documentation.

The --help text can tell where more information is available.

[Random832]

On Thu, Apr 28, 2016, at 13:06, Chris Angelico wrote:
> On Fri, Apr 29, 2016 at 2:33 AM, Steven D'Aprano <st...@pearwood.info>
> wrote:

> > Many command line tools simply output help to stdout (or stderr, if they're
> > evil),
>

> I'm not sure stderr is particularly more evil than stdout, but whatever
> :)

When a tool has very long help and sends it to stderr:

$ foo --help
ok, it's scrolled off the top, and I could just scroll up, but it's so
long I'll have to search for what I want anyway.
$ foo --help | less
*help flashes on the screen, then blank ~~~~~~ screen from less*
#@!@#$#$!#$!@#$@#!$@!#$!@#$!@#$!@#$!@#$
$ foo --help 2>&1 | less

Basically the only reason this ever happens is that programmers get lazy
and use the same function for usage errors as for --help. A usage error
message *should* go to stderr (in order to show up if the program's
output has been redirected to null, and so as not to be silently piped
to something that expects the program's normal output), but it should be
short. Help should go to stdout because the user actually requested it.

The obvious exception, and probably the bad example that people are
following, is programs that don't actually *have* --help, but briefly
summarize all their options and the meanings of positional arguments in
the usage error. People start with that, and then expand it to a
full-blown GNU-style --help message, but continue sending it to stderr.

[Marko Rauhamaa]

Irmen de Jong <irmen....@xs4all.nl>:

> An idea: Use just one help option, then
>
> if sys.stdout.isatty():
> #....use a pager to display help text
> else:
> #....print all help text normally

I've seen that used, but I find it annoying:

========================================================================
$ man ls
WARNING: terminal is not fully functional
- (press RETURN)
========================================================================

========================================================================
$ git diff
WARNING: terminal is not fully functional
- (press RETURN)
========================================================================

when run in emacs' Shell Mode.

Even Ctrl-C doesn't get me out of it. I curse to myself and run again:

========================================================================
$ git diff | cat
...
========================================================================


Marko

[Dan Strohl]

I would hesitate to take this approach unless the tool was one that only I was going to be using, and I knew exactly what environments it was going to be in.

I know that many of the system items in python work differently in different operating systems, and different os's report things differently as well as different ways of launching things. So, for example, I have seen cases where tools such as webmin will try to run a command line tool, and end up hung because they are waiting on a keypress for the next screen...

While I might do it as a way of setting a default, I would still want to have the ability to force a pager, or force no pager. I've been bit too many times by programmers doing things automatically for me that I didn't want.

I think that most people who use command line tools these days will be able to figure out how to pipe it to "more" if needed (or whatever).

Dan

> -----Original Message-----
> From: Python-list [mailto:python-list-bounces+d.strohl=f5....@python.org]

> On Behalf Of Irmen de Jong
> Sent: Thursday, April 28, 2016 10:08 AM
> To: pytho...@python.org
> Subject: Re: What should Python apps do when asked to show help?
>

> --
> https://mail.python.org/mailman/listinfo/python-list

[Grant Edwards]

On 2016-04-28, Steven D'Aprano <st...@pearwood.info> wrote:

> I have an application written in Python which accepts -h or --help to show
> help. I can:
>
> (1) print the help text to stdout;

Yep: just write it to stdout.

> (2) run the help text through a pager;

If you do (1), and I can do that myself if that's what I want.

> (3) do something else?

Nope (at least not by default).

> Many command line tools simply output help to stdout (or stderr, if
> they're evil),

Yep, writing help output to stderr is annoying.

> which makes it easy to redirect the help to a file, pass it to grep,
> etc.

Exactly.

> but I was thinking of doing both: give my application a subcommand or an
> option to display help directly in a pager, while -h and --help print to
> stdout as normal.
>
> What do you think? Too clever?

As long as -? -h --help just write stuff to stdout you can add
whatever other options you like that run pagers, start up web
browsers, or show mp4 movies on the wall without annoying grouchy old
Unix users like me. ;)

--
Grant Edwards grant.b.edwards Yow! FOOLED you! Absorb
at EGO SHATTERING impulse
gmail.com rays, polyester poltroon!!

[Dan Strohl]

Yup.. another reason to use something like argparse... you define the argument descriptions, help, and when you raise an error, it automatically handles the output, sending it to the right place (stderr/stdout)... as well as allowing you to define different levels of verbosity easily... (or not if you don't want)

Argparse isn't the best tool in the world, and there are some things that annoy me about it sometimes, but it works pretty well within its boundaries.

(and if the code's already written, I'm not suggesting that it be re-written just to move it to a common library.

> -----Original Message-----
> From: Python-list [mailto:python-list-bounces+d.strohl=f5....@python.org]
> On Behalf Of Random832
> Sent: Thursday, April 28, 2016 10:30 AM
> To: pytho...@python.org
> Subject: Re: What should Python apps do when asked to show help?
>

> On Thu, Apr 28, 2016, at 13:06, Chris Angelico wrote:

> > On Fri, Apr 29, 2016 at 2:33 AM, Steven D'Aprano <st...@pearwood.info>
> > wrote:
> > > Many command line tools simply output help to stdout (or stderr, if
> > > they're evil),
> >

> > I'm not sure stderr is particularly more evil than stdout, but
> > whatever
> > :)
>
> When a tool has very long help and sends it to stderr:
>
> $ foo --help
> ok, it's scrolled off the top, and I could just scroll up, but it's so long I'll have to
> search for what I want anyway.
> $ foo --help | less
> *help flashes on the screen, then blank ~~~~~~ screen from less*
> #@!@#$#$!#$!@#$@#!$@!#$!@#$!@#$!@#$!@#$
> $ foo --help 2>&1 | less
>
> Basically the only reason this ever happens is that programmers get lazy and
> use the same function for usage errors as for --help. A usage error message
> *should* go to stderr (in order to show up if the program's output has been
> redirected to null, and so as not to be silently piped to something that
> expects the program's normal output), but it should be short. Help should go
> to stdout because the user actually requested it.
>
> The obvious exception, and probably the bad example that people are
> following, is programs that don't actually *have* --help, but briefly
> summarize all their options and the meanings of positional arguments in the
> usage error. People start with that, and then expand it to a full-blown GNU-
> style --help message, but continue sending it to stderr.

> --
> https://mail.python.org/mailman/listinfo/python-list

[Random832]

On Thu, Apr 28, 2016, at 13:33, Grant Edwards wrote:
> As long as -? -h --help just write stuff to stdout you can add
> whatever other options you like that run pagers, start up web
> browsers, or show mp4 movies on the wall without annoying grouchy old
> Unix users like me. ;)

One disadvantage is that you have to compose two forms of documentation.
I suspect this is why git [command] --help just opens the manpage. (It
helps that man itself just sends output to stdout when not outputting to
a tty... though it's very annoying that on Windows it opens it in a web
browser instead.)

[Grant Edwards]

On 2016-04-28, Random832 <rand...@fastmail.com> wrote:
>
>
> On Thu, Apr 28, 2016, at 13:33, Grant Edwards wrote:
>> As long as -? -h --help just write stuff to stdout you can add
>> whatever other options you like that run pagers, start up web
>> browsers, or show mp4 movies on the wall without annoying grouchy old
>> Unix users like me. ;)
>
> One disadvantage is that you have to compose two forms of documentation.

Only if you want two forms of documentation.

If you add an option to run the help info through a pager, I don't see
how that requires you to compose two forms for documentation. If you
want both plaintext and video help, then yes you'll have to prepare

two forms of documentation.

> I suspect this is why git [command] --help just opens the
> manpage. (It helps that man itself just sends output to stdout when
> not outputting to a tty... though it's very annoying that on Windows
> it opens it in a web browser instead.)

I don't understand.

If you want your comand-line help info to be in manpage format, that
doesn't mean you have to run it through a pager or open a new window
running the man command, just write the formatted text to stdout.

--
Grant Edwards grant.b.edwards Yow! My mind is making
at ashtrays in Dayton ...
gmail.com

[Marko Rauhamaa]

Grant Edwards <grant.b...@gmail.com>:

> On 2016-04-28, Random832 <rand...@fastmail.com> wrote:
>> One disadvantage is that you have to compose two forms of
>> documentation.
>
> Only if you want two forms of documentation.
>
> If you add an option to run the help info through a pager, I don't see
> how that requires you to compose two forms for documentation. If you
> want both plaintext and video help, then yes you'll have to prepare
> two forms of documentation.

I wouldn't want --help to output a reference manual.

Consider GNU tar:

tar --help ==> a list of options
man tar ==> a description of the command and the options
info tar ==> a user manual


Marko

[Grant Edwards]

That's fine. If you want two or three forms of documentation then you
prepare two or three forms of documentation.

Adding an option to run the default 'help' output through a pager or
display it in a web browser doesn't somehow force you "to compose two
forms of documentation" unless you want two forms of documentation.

--
Grant Edwards grant.b.edwards Yow! A dwarf is passing out
at somewhere in Detroit!
gmail.com

[Random832]

On Thu, Apr 28, 2016, at 15:39, Grant Edwards wrote:
> That's fine. If you want two or three forms of documentation then you
> prepare two or three forms of documentation.
>
> Adding an option to run the default 'help' output through a pager or

> display it in a web browser doesn't somehow force you "to compose two
> forms of documentation" unless you want two forms of documentation.

The point is that having "help" output at all (beyond either a cursory
"see the manpage") implies two forms of documentation (given you already
have a manpage), and some people choose not to do that, instead
launching directly into the manpage (which implies using a pager)

[Cameron Simpson]

On 29Apr2016 03:06, Chris Angelico <ros...@gmail.com> wrote:
>On Fri, Apr 29, 2016 at 2:33 AM, Steven D'Aprano <st...@pearwood.info> wrote:

>> I have an application written in Python which accepts -h or --help to show
>> help. I can:
>>
>> (1) print the help text to stdout;
>>

>> (2) run the help text through a pager;
>>

>> (3) do something else?

>>
>>
>> Many command line tools simply output help to stdout (or stderr, if they're
>> evil),
>
>I'm not sure stderr is particularly more evil than stdout, but whatever :)

I'm sure. If I _ask_ for the help information, it is not an error message.
Instead it is desired output. The logic is clear.

If I screw up and elicit an error and the usage message (which might be the
help text), that is an error message. But -h/--help? That is standard output,
even if it is the same usage text.

And on this topic, let me preemptively flame commands whose entire usage
message is "use -h/--help for usage". More evil.

>What you could do is run the help text through a pager only if stdout
>is a TTY. That allows it to be grepped conveniently, but also browsed
>easily. But I wouldn't worry about that unless you have at least 3-5
>pages of screed - for most programs, just dumping the text to a
>standard stream is usually enough.

+1 to this. I'm -0 on conditionally offering a pager based on isatty; Marko
Rauhamaa has illustrated one downside, and personally I find _unexpectedly_
paged output to be jarring, requiring me to modeshift my brain.

So I'm for keeping it simple and just sending it to stdout.

Cheers,
Cameron Simpson <c...@zip.com.au>

[Grant Edwards]

On 2016-04-28, Random832 <rand...@fastmail.com> wrote:

But I'm saying that having --help output the manpage should not imply
using a pager.

If --help is going to output the manpage, then I'm saying it can (and
should) be written to stdout without use of pager (and it shouldn't
have escape sequences or backspaces in it either). The Unix way is
that the output from whatever --help should be plain text written to
stdout (regardless of length).

If you want to output the man page, this can be accomplished simply by
doing someting like:

os.environ["GROFF_NO_SGR"] = "1";
os.system("man -Tascii mycmd | col -bx");

If I want the --help output run through a pager, I'll do it myself.

I agree that a good argument can be made that the --help output should
be a concise summary, and that a separate longer man page might be a
good idea. But whatever you decide to provide when invoked with
--help it should be plaintext written to stdout. No pagers, no
colors, no HTML.

[if you're really going to invoke the 'man' command, you should do it
with the subprocess module with shell=False -- but examples with
os.system() are easier to grok]

--
Grant Edwards grant.b.edwards Yow! Sometime in 1993
at NANCY SINATRA will lead a
gmail.com BLOODLESS COUP on GUAM!!

[Steven D'Aprano]

On Fri, 29 Apr 2016 07:08 am, Grant Edwards wrote:

> On 2016-04-28, Random832 <rand...@fastmail.com> wrote:
>> On Thu, Apr 28, 2016, at 15:39, Grant Edwards wrote:
>>> That's fine. If you want two or three forms of documentation then you
>>> prepare two or three forms of documentation.
>>>
>>> Adding an option to run the default 'help' output through a pager or
>>> display it in a web browser doesn't somehow force you "to compose two
>>> forms of documentation" unless you want two forms of documentation.
>>
>> The point is that having "help" output at all (beyond either a cursory
>> "see the manpage") implies two forms of documentation (given you already
>> have a manpage), and some people choose not to do that, instead
>> launching directly into the manpage (which implies using a pager)
>
> But I'm saying that having --help output the manpage should not imply
> using a pager.

What manpage? I don't have a manpage. The only help my application has is
whatever it outputs (which it gets from its docstring).

What is a good place where I can find out more about writing manpages?

> If --help is going to output the manpage, then I'm saying it can (and
> should) be written to stdout without use of pager (and it shouldn't
> have escape sequences or backspaces in it either). The Unix way is
> that the output from whatever --help should be plain text written to
> stdout (regardless of length).

Agree. The -h/--help option will output plain text to stdout.

> If you want to output the man page, this can be accomplished simply by
> doing someting like:
>
> os.environ["GROFF_NO_SGR"] = "1";
> os.system("man -Tascii mycmd | col -bx");

That doesn't work for me:

[steve@ando ~]$ man -Tasciii rm
man: invalid option -- T



However, this almost works:

man -Pcat rm

appears to output unformatted, plain text to stdout. However, if you capture
the output (say, to a file) you'll see it is full of backspaces, e.g.:

N^HNA^HAM^HME^HE
rm - remove files or directories

S^HSY^HYN^HNO^HOP^HPS^HSI^HIS^HS
r^Hrm^Hm [_^HO_^HP_^HT_^HI_^HO_^HN]... _^HF_^HI_^HL_^HE...


instead of

NAME
rm - remove files or directories

SYNOPSIS
rm [OPTION]... FILE...


Apparently `less` understands overstriking and displays it as bold (or
underlining in the case of _^H. That's very clever, but how do I get actual
plain text out of `man` without the backspaces?

> If I want the --help output run through a pager, I'll do it myself.

Easy enough to say for Linux/Unix users, but there are those who might not
have a pager that is easy to use, or at all. pydoc goes to considerable
trouble to work out the best pager available for your platform, falling
back to its own if needed. To use that is literally a two liner:

import pydoc
pydoc.pager(HELPTEXT)




--
Steven

[Paul Rubin]

Steven D'Aprano <st...@pearwood.info> writes:
> (1) print the help text to stdout;
> (2) run the help text through a pager;

Stdout unless the PAGER env var is set. Otherwise, I'd say still stdout
since the person can pipe it through a pager if they want, but you could
use the pager or be fancy and try to detect if stdout is a pty/tty
before using the pager. Git pages by default and it bugs me because I
tend to use git commands in emacs shell windows and the pager makes a
mess.

[Rustom Mody]

On Friday, April 29, 2016 at 7:45:35 AM UTC+5:30, Paul Rubin wrote:

Quite bewildered by this thread...
Are we in 2016?
[Reminds me of the bizarre claim that world has not moved on from text]
eg I am on on Ubuntu 16.4 (vanilla) right now.
This means that when I pull up a terminal I get gnome-terminal.
Which means that when the screen fills up I get a scroll bar
Now I can get perverse and use xterm/rxvt.
Or a console outside of X (ctrl-Alt-F[1-6] )
And even there Shift-pgup works

Of course there is the 1 in 100 times this does not hold;
- ssh to a remote machine
- Upgraded ubuntu and broke X
- etc

[What happens in windows on a cmd-box... not sure]

IOW pager functionality is quite builtin in shells nowadays
Why replicate and cause annoyance?

[Jussi Piitulainen]

Rustom Mody writes:

> On Friday, April 29, 2016 at 7:45:35 AM UTC+5:30, Paul Rubin wrote:
>> Steven D'Aprano writes:
>> > (1) print the help text to stdout;
>> > (2) run the help text through a pager;
>>
>> Stdout unless the PAGER env var is set. Otherwise, I'd say still stdout
>> since the person can pipe it through a pager if they want, but you could
>> use the pager or be fancy and try to detect if stdout is a pty/tty
>> before using the pager. Git pages by default and it bugs me because I
>> tend to use git commands in emacs shell windows and the pager makes a
>> mess.
>
> Quite bewildered by this thread...

Yet you seem to agree with everybody else in this thread? Including the
two persons you quote above, if I read them right :)

> Are we in 2016?

$ date '+Yes: %F %T'
Yes: 2016-04-29 08:11:07

> [Reminds me of the bizarre claim that world has not moved on from text]
> eg I am on on Ubuntu 16.4 (vanilla) right now.
> This means that when I pull up a terminal I get gnome-terminal.
> Which means that when the screen fills up I get a scroll bar
> Now I can get perverse and use xterm/rxvt.
> Or a console outside of X (ctrl-Alt-F[1-6] )
> And even there Shift-pgup works
>
> Of course there is the 1 in 100 times this does not hold;
> - ssh to a remote machine
> - Upgraded ubuntu and broke X
> - etc

Remote shell works fine.

I spend most of my time in a shell on a remote machine over ssh, without
X forwarding. I have turned scrollbars off.

If I need to scroll back the remote shell session a screen or more, I
just slide my index finger along the touchpad edge, or index and middle
finger over the touchpad on the other laptop.

In Emacs in that remote shell, including a shell session in the Emacs, I
use the normal keys to scroll back and forth. There the touchpad doesn't
scroll at all, and I don't expect it to, either.

> [What happens in windows on a cmd-box... not sure]

If that happens to be crippled, there must be other terminal programs
that work.

> IOW pager functionality is quite builtin in shells nowadays
> Why replicate and cause annoyance?

First, more and less do more than just scroll and they may be nicer even
for just scrolling. Second, if things are set up the way that just about
everybody in this thread has been advocating, you have full control over
your use of a pager. If it hurts, don't do it. Does it annoy you that
somebody else is doing it?

It's the *default* paging that is mildly annoying when it works and a
major pain when it doesn't. (Another major pain is a command-line tool
that tries to open a window and can't because I haven't enabled it.
Please don't make this the default either. :)

[Gregory Ewing]

> Irmen de Jong <irmen....@xs4all.nl>:

>
>>if sys.stdout.isatty():
>> #....use a pager to display help text
>>else:
>> #....print all help text normally

I think nowadays it's an anti-pattern for programs to
do their own pagination. Very often the "terminal" is
a GUI application with its own facilities for scrolling
and searching that are much nicer to use. I prefer the
program to just give me the output and get out of the
way. If I want a clunky tty-oriented pager I'll pipe
it to "more".

--
Greg

[Steven D'Aprano]

On Fri, 29 Apr 2016 03:00 pm, Rustom Mody wafted information-rich pheromones
into the air, where they diffused rapidly:

> Quite bewildered by this thread...
> Are we in 2016?
> [Reminds me of the bizarre claim that world has not moved on from text]

(For the benefit of those of us still stuck in the 2000s, I have transcribed
Rustom's pheromones into the obsolete medium of text.)

> eg I am on on Ubuntu 16.4 (vanilla) right now.
> This means that when I pull up a terminal I get gnome-terminal.
> Which means that when the screen fills up I get a scroll bar

Yes. So do I.

Of course, some applications (like screen) interfere with that, but even
when you have a scroll bar, having to scroll back more than a few pages is
a PITA. If you set the number of scrollback lines too low, you run out and
the things you are interested in disappears past the top... and if you set
it too high, it's too hard to locate the section that you actually want.

The bottom line is, screen real estate is a valuable resource, but even more
valuable is human attention, and pagers are a way of managing attention.
Screen space is not necessary rare, but neither is it something that we can
just consume thoughtlessly. Hence we have a whole range of options
available to us, to make the most of screen space:

- multiple windows;
- multiple tabs within each window;
- scroll bars within each tab;
- and pagers.

Think of a pager as being somewhat like a popup window for text: at the very
low cost of a single line of space in your terminal, you can "pop up" the
text, read it, and dismiss it again without using up any of the screen
space in your scrolling text view. (Apart from that one line.)

> IOW pager functionality is quite builtin in shells nowadays

Apart from those where they're not.

> Why replicate and cause annoyance?

If you don't want to use the functionality, don't.



--
Steven

[Rustom Mody]

On Friday, April 29, 2016 at 3:07:09 PM UTC+5:30, Steven D'Aprano wrote:
> On Fri, 29 Apr 2016 03:00 pm, Rustom Mody wafted information-rich pheromones
> into the air, where they diffused rapidly:

> > Why replicate and cause annoyance?
>
> If you don't want to use the functionality, don't.

You asked about calling up pagers!!

JFTR I find git behavior annoying -- as it seems do others -- but Ive figured
getting round it by calling it with -w (show in web browser) is ok (with me)

With python's help I find it annoying and Ive not figured out how to not get paging

[Martin A. Brown]

Hello,

>What is a good place where I can find out more about writing

>manpage?

I don't know of a single place where manpage authorship is
particularly documented. This seems to be one of the common target
links. In addition to introducing the breakdown of manpages by
type (section) and providing some suggestions for content, it
introduces the *roff markup:

http://www.schweikhardt.net/man_page_howto.html

It's been many years since I have written that stuff directly. I
prefer one of the lightweight, general documentation or markup
languages. So, below, I'll mention and give examples for creating
manpages from reStructuredtext, AsciiDoc and Plain Old
Documentation.

With the reStructuredText format [0] [1], you can convert an .rst
file to a manpage using two different document processors; you can
use sphinx-build from the sphinx-project [2] or rst2man from the
docutils project. The outputs are largely the same (as far as I
can tell).

There's also the AsciiDoc [3] format, which is a near to text and
reads like text, but has a clear structure. With the tooling
(written in Python), you can produce docbook, latex, html and a
bunch of other output formats. Oh, and manpages [4], too. There is
a tool called 'asciidoc' which processes AsciiDoc formats into a
variety of backend formats. The 'a2x' tool converts AsciiDoc
sources into some other (x) desired output.

If you don't like .rst or AsciiDoc, there's also the Plain Old
Documentation (POD) format. This is the oldest tool (of which I'm
aware) which other than the direct *roff processing tools. You run
'pod2man' (written in Perl) on your .pod file. POD is another dead
simple documentation language, supported by the pod2man [5] tool.
For more on the format, read also 'man 1 perlpod'.

sphinx-build: the sphinx documentation system is geared for
handling project-scoped documentation and provides many additional
features to reStructuredText. It can produce all kinds of output
formats, HTML single-page, help, multipage, texinfo, latex, text,
epub and ....oh, yeah, manpages. It's a rich set of tools.

If you wish to use sphinx, I can give you an example .rst file [6]
which I recently wrote and the following instructions for how to
process this with sphinx. When processing docs with sphinx, a
'conf.py' file is required. It can be generated with an ancillary
tool from the sphinx suite:

I know that I always find an example helpful. So, here are some
examples to help you launch.

mkdir sampledir && cd sampledir
sphinx-quickstart # -- and answer a bunch of questions
# -- examine conf.py and adjust to your heart's content
# confirm that master_doc is your single document for a manpage
# confirm that there's an entry for your document in man_pages
sphinx-build -b man -d _build/doctrees . _build/man

# -- or grab the files from my recent project [6] and try yourself

rst2man: even more simply, if you don't need the kitchen sink...

wget https://gitlab.com/pdftools/pdfposter/raw/develop/pdfposter.rst
rst2man < pdfposter.rst > pdfposter.1
# -- will complain about this, but still produces a manpage
# <stdin>:10: (ERROR/3) Undefined substitution referenced: "VERSION".
man ./pdfposter.1

asciidoc (a randomly selected example asciidoc file [7]):

wget https://raw.githubusercontent.com/DavidGamba/grepp/master/grepp.adoc
a2x -f manpage grepp.adoc
man ./grepp.1

perlpod:

wget https://api.metacpan.org/source/RJBS/perl-5.18.1/pod/perlrun.pod
pod2man --section 1 < perlrun.pod > perlrun.1
man ./perlrun.1

I know there are other tools for generating manpages; the
original *roff tools, visual manpage editors, DocBook,
help2man, manpage generators from argparse.ArgumentParser
instances,

And, of course, make sure to use version control for your
documentation. These git manpages may be helpful for the
uninitiated (joke, joke):

https://git-man-page-generator.lokaltog.net/ # -- humour!

Good luck,

-Martin

[0] http://docutils.sourceforge.net/docs/user/rst/quickref.html
[1] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
[2] http://www.sphinx-doc.org/en/stable/rest.html
[3] http://www.methods.co.nz/asciidoc/
[4] http://www.methods.co.nz/asciidoc/chunked/ch24.html
[5] http://perldoc.perl.org/pod2man.html
[6] https://raw.githubusercontent.com/tLDP/python-tldp/master/docs/ldptool-man.rst
[7] https://raw.githubusercontent.com/DavidGamba/grepp/master/grepp.adoc

--
Martin A. Brown
http://linux-ip.net/

[Ethan Furman]

Wow. Thank you for that very informative post!

--
~Ethan~

[Rustom Mody]

For emacs junkies there is also org-e-man


[1] http://orgmode.org/worg/org-tutorials/org-e-man-documentation.html
[2] https://github.com/tkf/org-mode/blob/master/contrib/lisp/org-e-man.el

[No I havent used this but I use org mode heavily]

[Steven D'Aprano]

On Fri, 29 Apr 2016 07:53 pm, Rustom Mody wrote:

> On Friday, April 29, 2016 at 3:07:09 PM UTC+5:30, Steven D'Aprano wrote:
>> On Fri, 29 Apr 2016 03:00 pm, Rustom Mody wafted information-rich
>> pheromones into the air, where they diffused rapidly:
>> > Why replicate and cause annoyance?
>>
>> If you don't want to use the functionality, don't.
>
> You asked about calling up pagers!!

Right. And options -h/--help will behave exactly as people have requested,
by printing to stdout.

> JFTR I find git behavior annoying -- as it seems do others

`git --help` behaves as the Unix standard: it prints help output to stdout.
Is that the annoying behaviour?

`git help <command>` and `git <command> --help` call `man`. Is that the
annoying behaviour? Then presumably `man` is also annoying, and the advise
I was given to just use man pages is bad advice.

> -- but Ive
> figured getting round it by calling it with -w (show in web browser) is ok
> (with me)

You would rather output be shown in a web browser than paged in your
terminal? That's weird, but okay, whatever floats your boat.

> With python's help I find it annoying and Ive not figured out how to not
> get paging

o_O

Okay, now I'm feeling as if you had said "I find it annoying to be fit and
healthy, I've not found a way to feel sufficiently sick, tired and
out-of-shape all the time."

But I see your point. The pydoc documentation itself is lacking. But from
reading the source code, I see that if you set the PAGER environment
variable to your preferred pager, it will use that. So setting it to "cat"
should work. I've just tested this under Linux, and it works for me:


[steve@ando ~]$ PAGER=cat python3.3
Python 3.3.0rc3 (default, Sep 27 2012, 18:44:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux
Type "help", "copyright", "credits" or "license" for more information.
py> help(len)
Help on built-in function len in module builtins:

len(...)
len(object) -> integer

Return the number of items of a sequence or mapping.

py>



--
Steven

[Ethan Furman]

On 04/29/2016 06:20 PM, Steven D'Aprano wrote:
> On Fri, 29 Apr 2016 07:53 pm, Rustom Mody wrote:


>> JFTR I find git behavior annoying -- as it seems do others
>
> `git --help` behaves as the Unix standard: it prints help output to stdout.
> Is that the annoying behaviour?

No.

> `git help <command>` and `git <command> --help` call `man`. Is that the
> annoying behaviour?

Yes.

> Then presumably `man` is also annoying,

No.

> and the advise I was given to just use man pages is bad advice.

The advice to call man from --help is bad; the advice to have a man page
for use with man is not.

>> With python's help I find it annoying and Ive not figured out how to not
>> get paging
>
> o_O
>
> Okay, now I'm feeling as if you had said "I find it annoying to be fit and
> healthy, I've not found a way to feel sufficiently sick, tired and
> out-of-shape all the time."

And exactly what is healthy and fit about calling "help(something)" and
then having that help disappear? I find that *extremely* annoying.

> But I see your point. The pydoc documentation itself is lacking. But from
> reading the source code, I see that if you set the PAGER environment
> variable to your preferred pager, it will use that. So setting it to "cat"
> should work. I've just tested this under Linux, and it works for me:

So I have to cripple my shell to get pydoc help to work nicely? Neat!
Actually, not so much. :(

--
~Ethan~

[Random832]

On Fri, Apr 29, 2016, at 22:09, Ethan Furman wrote:
> So I have to cripple my shell to get pydoc help to work nicely? Neat!
> Actually, not so much. :(

If you don't want a pager with pydoc, when exactly do you want it?

[Ben Finney]

"Martin A. Brown" <mar...@linux-ip.net> writes:

> Hello [Steven D'Aprano],
>
> >What is a good place where I can find out more about writing manpage?

Writing them directly in GNU troff markup is easy enough
<URL:http://liw.fi/manpages/>.

For writing manual pages programmatically via Python code, I am working
on a ‘manpage’ library. It is not yet at PyPI, but Steven may find
the existing library <URL:https://notabug.org/bignose/python-manpage>
useful. It's free software under the GNU GPL v3 or later.

Steven, if you want to use that library for making manual pages, I'd
love to get feedback. Currently it works on an ArgumentParser instance
(to get the program's name and option help), and a Distutils
distribution (to get the boader metadata about the package).

What ‘manpage’ doesn't yet have is a more generic way to specify the
metadata of a manual page. Everything is done by directly modifying a
Document instance. I have yet to decide a data format for the data which
specifies a manual page.

Getting several more code bases using this library would make it clearer
what use cases are important, and thereby inform a data format for the
specification. Please experiment!

--
\ “The process by which banks create money is so simple that the |
`\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It |
_o__) Came, Where It Went_, 1975 |
Ben Finney

[Rustom Mody]

- I start a python interpreter
- ... Am pottering around for a while
- ... Need help on something...
[So with Steven's solution of PAGER=cat Ive to restart python!!
Leaving that aside...]
- ... help can do one of two valid things [remember I am on a modern windowing
system]
-- start help inline, inband (cat)
-- start help out of band (browser, yelp, etc ie some other app)

Instead it does some ½-assed fall-between-the-stools of both

[Ben Finney]

"Martin A. Brown" <mar...@linux-ip.net> writes:

> Hello [Steven D'Aprano],
>

> >What is a good place where I can find out more about writing manpage?

Writing manual pages directly in standard GNU troff markup is simple
enough <URL:http://liw.fi/manpages/>. It's not a pretty markup language,
but it's workable.

For writing manual page documents programmatically via Python code, I am

working on a ‘manpage’ library.

It is not yet at PyPI, but Steven may find the existing library
<URL:https://notabug.org/bignose/python-manpage> useful. It's free
software under the GNU GPL v3 or later.

Steven, if you want to use that library for making manual pages, I'd
love to get feedback.

Currently it works on an ArgumentParser instance (to get the program's
name and option help), and a Distutils distribution (to get the boader
metadata about the package).

What ‘manpage’ doesn't yet have is a more generic way to specify the

metadata of a manual page; the above inputs are the only ones that I've
tested. Everything is done by directly modifying a Document instance. I

have yet to decide a data format for the data which specifies a manual
page.

If several more code bases could use this library to generate manual
pages, that would make it clearer what use cases are important, and

[Random832]

That doesn't answer the question of why, if you (Well, Ethan, but you're
taking the same position here) hate pagers so much, why you can't just
set PAGER=cat in your profile once, now, and never see one again. How
does this "cripple your shell"; if you hate pagers so much why do you
want man or git to use one any more than pydoc?

[Rustom Mody]

I dont get whats so hard to get in this:
When we need pagers we know where to get them
When we dont, please dont thump them on us
I dont know that anyone has taken the "I-HATE-pagers' view so much as
"Let me get them when I want them"

[Ben Finney]

Random832 <rand...@fastmail.com> writes:

> On Fri, Apr 29, 2016, at 22:27, Rustom Mody wrote:
> > Instead it does some ½-assed fall-between-the-stools of both
>

> That doesn't answer the question of why, if you (Well, Ethan, but
> you're taking the same position here) hate pagers so much

That's not a question relevant here; nobody inthe discussion has a
position fairly characterised as “hate pagers so much”. So you're
arguing against a straw man.

Rather, the position being argued is that they *do* like pagers; they
like pagers enough that they want the existing ‘PAGER’ environment
variable setting to remain untouched. And, simulatenously, they want
Python's help to not use a pager at the interactive prompt.

--
\ “The difference between religions and cults is determined by |
`\ how much real estate is owned.” —Frank Zappa |
_o__) |
Ben Finney

[Random832]

On Fri, Apr 29, 2016, at 22:46, Rustom Mody wrote:
> I dont get whats so hard to get in this:
> When we need pagers we know where to get them

And if you set PAGER=cat (thus "crippling your shell"), you will
likewise know where to get them when you want to page the output from
man or git. Why is pydoc less of a "place where a pager should be used
by default" than man or git (diff, log, etc), if you accept the latter?

> When we dont, please dont thump them on us
> I dont know that anyone has taken the "I-HATE-pagers' view so much as
> "Let me get them when I want them"

Sure. And if you set PAGER=cat, then you won't ever see a pager unless
you specifically request one. Everyone wins.

[Rustom Mody]

On Saturday, April 30, 2016 at 7:55:47 AM UTC+5:30, Ben Finney wrote:

> "Martin A. Brown" writes:
>
> > Hello [Steven D'Aprano],
> >
> > >What is a good place where I can find out more about writing manpage?
>
> Writing them directly in GNU troff markup is easy enough
> <URL:http://liw.fi/manpages/>.
>
> For writing manual pages programmatically via Python code, I am working
> on a 'manpage' library. It is not yet at PyPI, but Steven may find
> the existing library <URL:https://notabug.org/bignose/python-manpage>
> useful. It's free software under the GNU GPL v3 or later.
>
> Steven, if you want to use that library for making manual pages, I'd
> love to get feedback. Currently it works on an ArgumentParser instance
> (to get the program's name and option help), and a Distutils
> distribution (to get the boader metadata about the package).
>
> What 'manpage' doesn't yet have is a more generic way to specify the
> metadata of a manual page. Everything is done by directly modifying a
> Document instance. I have yet to decide a data format for the data which
> specifies a manual page.
>
> Getting several more code bases using this library would make it clearer
> what use cases are important, and thereby inform a data format for the
> specification. Please experiment!

Some general thoughts and comments:
Documentation in linux is a train-wreck in slow motion -- 30 years of slow motion!!

tl;dr
If you want to be helpful to the community please dont create a new format
Instead try to make existing formats interoperate/work better with each other

In more detail:
First there was man
Then gnu/rms created info to better man with hypertext capabilities
However another hypertext standard beat the gnu standard -- html
[No I am not talking of quality just general acceptance]
As with all things rms, its taking him decades to realize this defeat
[Latest makeinfo is 18 times slower than previous version!!
https://lists.gnu.org/archive/html/bug-texinfo/2013-01/msg00012.html
]

In the meantime the so called lightweight formats have multiplied:
rst, textile, markdown, asciidoc etc
with their own 'stacks'
Alongside siloed, incompatible help systems:
manpages, infodocs, yelp...

[Steven D'Aprano]

On Sat, 30 Apr 2016 12:49 pm, Ben Finney wrote:

> Random832 <rand...@fastmail.com> writes:
>
>> On Fri, Apr 29, 2016, at 22:27, Rustom Mody wrote:
>> > Instead it does some ½-assed fall-between-the-stools of both
>>
>> That doesn't answer the question of why, if you (Well, Ethan, but
>> you're taking the same position here) hate pagers so much
>
> That's not a question relevant here; nobody inthe discussion has a
> position fairly characterised as “hate pagers so much”. So you're
> arguing against a straw man.
>
> Rather, the position being argued is that they *do* like pagers; they
> like pagers enough that they want the existing ‘PAGER’ environment
> variable setting to remain untouched.

So they want the PAGER environment variable to specify what pager they
want...

> And, simulatenously, they want
> Python's help to not use a pager at the interactive prompt.

...so long as applications don't actually make use of that PAGER environment
variable to determine the pager they want to use.


(1) If you want man, and nothing else in the universe, to automatically use
a pager, then set MANPAGER="less" (or whatever you wish to use), and unset
PAGER.


(2) If you don't want *anything* to use a pager, then unset both MANPAGER
and PAGER. You may have to report a feature request / bug report for
applications which force their own pager.


(3) In Python specifically, you can trivially and easily tell help() to
output directly to stdout. (At least on Linux, I haven't tested it
elsewhere.) Simply use PAGER=cat on the command line you use to launch the
interactive environment. This will affect no other running or future
processes (apart from subprocesses launched from your interactive Python
session), allowing you to keep your PAGER for everything else.

(4) If you want more than that, then patches are welcome :-)


Seriously, I'm thinking that a keyword argument to help might be useful:

help(object, pager=None)

where:

- pager=None gives the current behaviour;

- pager="foo" calls out to the external program "foo";

- pager=callable passes the help text to callable().


pager=print would do exactly what people are asking for, and you could then
create your own wrapper to change the default:

help = functools.partial(builtins.help, pager=print)


I think that would make it easier to test help(). Thoughts?



--
Steven

[Ben Finney]

Steven D'Aprano <st...@pearwood.info> writes:

> So they want the PAGER environment variable to specify what pager they
> want...
>

> ...so long as applications don't actually make use of that PAGER
> environment variable to determine the pager they want to use.

This at least does not baldly misrepresent the position being made :-)

--
\ “My doctor told me to stop having intimate dinners for four. |
`\ Unless there are three other people.” —Orson Welles |
_o__) |
Ben Finney

[Paul Rubin]

Rustom Mody <rusto...@gmail.com> writes:
> As with all things rms, its taking him decades to realize this defeat
> [Latest makeinfo is 18 times slower than previous version!!
> https://lists.gnu.org/archive/html/bug-texinfo/2013-01/msg00012.html

Wait, what's it written in now?

> In the meantime the so called lightweight formats have multiplied:
> rst, textile, markdown, asciidoc etc

Pandoc can interconvert between most of those... maybe texinfo should
be folded into it if it's not already.

[Random832]

On Sat, Apr 30, 2016, at 00:06, Ben Finney wrote:
> Steven D'Aprano <st...@pearwood.info> writes:
>

> > So they want the PAGER environment variable to specify what pager they
> > want...
> >

> > ...so long as applications don't actually make use of that PAGER
> > environment variable to determine the pager they want to use.
>

> This at least does not baldly misrepresent the position being made :-)

I still don't understand how it's a misrepresentation. They said they
don't want stuff (where "stuff" includes pydoc but it's not clear why or
where it ends) to automatically use a pager without being requested, and
provided absolutely no criteria by which pydoc is special and shouldn't
behave the same as git and man do (since the entire reason PAGER=cat is
an unacceptable solution is that it "cripples" git and man).

[Rustom Mody]

Also environment variables are a nuisance and an antipattern:
http://peterlyons.com/problog/2010/02/environment-variables-considered-harmful

[Random832]

On Fri, Apr 29, 2016, at 23:23, Steven D'Aprano wrote:
> Seriously, I'm thinking that a keyword argument to help might be useful:
>
> help(object, pager=None)

I'd call it something more generic like "output".

> where:
>
> - pager=None gives the current behaviour;
>
> - pager="foo" calls out to the external program "foo";
>
> - pager=callable passes the help text to callable().
>
>
> pager=print would do exactly what people are asking for, and you could
> then
> create your own wrapper to change the default:
>
> help = functools.partial(builtins.help, pager=print)
>
>
> I think that would make it easier to test help(). Thoughts?

For testing purposes, help could return the result of the output
function, so that you can use lambda x: x to have it return the help
text.

More general thoughts: It might also be useful to move the pager
machinery from pydoc to an independent module. It's got, among other
things, a simple pager written in pure python, for use as a fallback.

There's a lot of stuff that could be improved in the pager stuff, while
we're at it. Probably needs a single function to handle "use external
program as a pager", rather than having a bunch of logic embedded in
getpager() which only works for os.environ['PAGER'].

The "pager" function doesn't behave "properly" (in its intended behavior
of calling the expensive getpager() once when it is first called and
subsequently reusing the cached value) if a reference to it is stored
elsewhere (e.g. by importing it to another module); it should store the
cached pager function somewhere else rather than by replacing itself.

The pure-python pager only works on unix-style ttys, an analogous
function could be written for windows using msvcrt.getwch.

[Rustom Mody]

On Saturday, April 30, 2016 at 9:36:42 AM UTC+5:30, Paul Rubin wrote:

> Rustom Mody writes:
> > As with all things rms, its taking him decades to realize this defeat
> > [Latest makeinfo is 18 times slower than previous version!!
> > https://lists.gnu.org/archive/html/bug-texinfo/2013-01/msg00012.html
>
> Wait, what's it written in now?

Have you any data that its moved on?

At that point what I gleaned was that original makeinfo was in C
New one was rewritten in perl.
[Yeah they could have taken a leaf out of pandoc -- written in haskell]

[Paul Rubin]

Rustom Mody <rusto...@gmail.com> writes:
> At that point what I gleaned was that original makeinfo was in C
> New one was rewritten in perl.

The previous one was definitely written in C and I've looked at the code
some. I hadn't known there was a new one. The C one was actually the
second one. The first one was written in Emacs Lisp but it was
painfully slow on the hardware of that era. The C version was
cumbersome by comparison. If they wanted to get rid of the C version
now that the hardware is 100x faster or more, it's amusing that they did
a Perl rewrite instead of reviving the Lisp version. Pandoc has a
significant user base now though, and it looks like it also supports
texinfo. So I wonder what the new Makeinfo does that pandoc doesn't.
Maybe pandoc doesn't support Info? That could be added, I'm sure.

[c...@zip.com.au]

On 30Apr2016 13:23, Steven D'Aprano <st...@pearwood.info> wrote:
>On Sat, 30 Apr 2016 12:49 pm, Ben Finney wrote:
>> Random832 <rand...@fastmail.com> writes:
>>> On Fri, Apr 29, 2016, at 22:27, Rustom Mody wrote:
>>> > Instead it does some ½-assed fall-between-the-stools of both
>>>
>>> That doesn't answer the question of why, if you (Well, Ethan, but
>>> you're taking the same position here) hate pagers so much
>>
>> That's not a question relevant here; nobody inthe discussion has a
>> position fairly characterised as “hate pagers so much”. So you're
>> arguing against a straw man.
>>
>> Rather, the position being argued is that they *do* like pagers; they
>> like pagers enough that they want the existing ‘PAGER’ environment
>> variable setting to remain untouched.
>
>So they want the PAGER environment variable to specify what pager they
>want...

_When_ they want a pager.

>> And, simulatenously, they want
>> Python's help to not use a pager at the interactive prompt.
>
>...so long as applications don't actually make use of that PAGER environment
>variable to determine the pager they want to use.

_When_ it is asked to use a pager.

What they (and, very often, me) want is that most things, including pydoc, to
_NOT_ invoke a pager automatically, _unasked_.

So tools which page without asking are unwelcome, because that requires a mode
switch when everything else (sed/grep/print/write/ls) do straight up unpaged
output.

>(1) If you want man, and nothing else in the universe, to automatically use
>a pager, then set MANPAGER="less" (or whatever you wish to use), and unset
>PAGER.
>
>(2) If you don't want *anything* to use a pager, then unset both MANPAGER
>and PAGER. You may have to report a feature request / bug report for
>applications which force their own pager.
>
>(3) In Python specifically, you can trivially and easily tell help() to
>output directly to stdout. (At least on Linux, I haven't tested it
>elsewhere.) Simply use PAGER=cat on the command line you use to launch the
>interactive environment. This will affect no other running or future
>processes (apart from subprocesses launched from your interactive Python
>session), allowing you to keep your PAGER for everything else.
>
>(4) If you want more than that, then patches are welcome :-)

This requires terrible domain specific knowledge. What about programs other
than man?

And setting PAGER=cat before invoking interactive python is no better, because
it will screw with $PAGER in any subprocess spawned from that environment.

What those of use in the "my terminal emulator scrolls nicely thank you" camp
want is that most command line things, when asked to simply print some help
text, _print_ it and do not page it. And often would like a way to tell pydoc
to not page, such as a pydoc specific switch (possibly an environment
variable).

>Seriously, I'm thinking that a keyword argument to help might be useful:
>
>help(object, pager=None)
>
>where:
>- pager=None gives the current behaviour;
>- pager="foo" calls out to the external program "foo";
>- pager=callable passes the help text to callable().

I'd be asking for pager=None to look at help.default_pager, which itself might
obey your rules above. That way help.default_pager can be a callable which
consults $PAGER or falls back to some inbuilt pager (presumably as now), but
which a user can sumply go:

help.default_pager=None

at the Python prompt and have unpaged output from there on.

>I think that would make it easier to test help(). Thoughts?

Yes it would. I'm +1 _provided_ the user can set a global to tune the default
mode, as otherwise it burdens the user with saying:

help(foo, stdout.write) # or "print" ?

whenever they want help. Not helpful!

Let me recite one of my favourite rules of thumb:

If it can't be turned off, it's not a feature. - Karl Heuer

Cheers,
Cameron Simpson <c...@zip.com.au>

[c...@zip.com.au]

On 29Apr2016 11:40, Steven D'Aprano <st...@pearwood.info> wrote:
>On Fri, 29 Apr 2016 07:08 am, Grant Edwards wrote:
>> On 2016-04-28, Random832 <rand...@fastmail.com> wrote:
>>> On Thu, Apr 28, 2016, at 15:39, Grant Edwards wrote:
>>>> That's fine. If you want two or three forms of documentation then you
>>>> prepare two or three forms of documentation.
>>>>
>>>> Adding an option to run the default 'help' output through a pager or
>>>> display it in a web browser doesn't somehow force you "to compose two
>>>> forms of documentation" unless you want two forms of documentation.
>>>
>>> The point is that having "help" output at all (beyond either a cursory
>>> "see the manpage") implies two forms of documentation (given you already
>>> have a manpage), and some people choose not to do that, instead
>>> launching directly into the manpage (which implies using a pager)
>>
>> But I'm saying that having --help output the manpage should not imply
>> using a pager.
>
>
>What manpage? I don't have a manpage. The only help my application has is
>whatever it outputs (which it gets from its docstring).
>
>What is a good place where I can find out more about writing manpages?

"man 5 man"? Describes the internal format of man pages (the [ntg]roff -man
macro set). Many people use an intermediate more human friendly format and use
a tool to transcribe -man format text. For standalone things I find Perl's
"POD" format fairly easy to use (the pod2man tool itself does have shortcomings
though).

>> If --help is going to output the manpage, then I'm saying it can (and
>> should) be written to stdout without use of pager (and it shouldn't
>> have escape sequences or backspaces in it either). The Unix way is
>> that the output from whatever --help should be plain text written to
>> stdout (regardless of length).
>
>Agree. The -h/--help option will output plain text to stdout.

Yay.

>> If you want to output the man page, this can be accomplished simply by
>> doing someting like:
>> os.environ["GROFF_NO_SGR"] = "1";
>> os.system("man -Tascii mycmd | col -bx");
>
>That doesn't work for me:
>[steve@ando ~]$ man -Tasciii rm
>man: invalid option -- T

Yes, it is nonportable. Presumes groff possibly, and GNU "man".

>However, this almost works:
>
>man -Pcat rm
>
>appears to output unformatted, plain text to stdout.

As should:

man rm | cat

without obscure options.

>However, if you capture
>the output (say, to a file) you'll see it is full of backspaces, e.g.:
>
>N^HNA^HAM^HME^HE
> rm - remove files or directories
>
>S^HSY^HYN^HNO^HOP^HPS^HSI^HIS^HS
> r^Hrm^Hm [_^HO_^HP_^HT_^HI_^HO_^HN]... _^HF_^HI_^HL_^HE...

Yes.

>instead of
>
>NAME
> rm - remove files or directories
[...]
>
>Apparently `less` understands overstriking and displays it as bold (or
>underlining in the case of _^H. That's very clever, but how do I get actual
>plain text out of `man` without the backspaces?

That is what Grant Edwards' "col -bx" does. (Actually, col can do more, because
tables also do funky things; let us not go there.)

But removing the overstriking is pretty easy even without col. My "unbs" script
does it, and its core operation is this:

s/[^^H]^H//g

Those ^Hs are literal backspace characters: remove any non-backspace followed
by a backspace. Full script here:

https://bitbucket.org/cameron_simpson/css/src/tip/bin/unbs

>> If I want the --help output run through a pager, I'll do it myself.
>
>Easy enough to say for Linux/Unix users, but there are those who might not
>have a pager that is easy to use, or at all. pydoc goes to considerable
>trouble to work out the best pager available for your platform, falling
>back to its own if needed. To use that is literally a two liner:
>
>import pydoc
>pydoc.pager(HELPTEXT)

Aye, but what is wanted by some of us is an easy incantation to tune that to
plain old sys.stdout.write in some flavour.

Cheers,
Cameron Simpson <c...@zip.com.au>

[Random832]

On Sat, Apr 30, 2016, at 19:51, c...@zip.com.au wrote:
> _When_ they want a pager.

Why would they need an environment variable at all in that case, rather
than explicitly invoking the pager by name?

To me, *not* having PAGER=cat signifies that someone *does* want a
pager. That may be a crappy convention, but it's the one we're stuck
with.

> And setting PAGER=cat before invoking interactive python is no better,
> because
> it will screw with $PAGER in any subprocess spawned from that
> environment.

But why would you not *also* want PAGER=cat in those subprocesses? You
don't want those things to spawn pagers that you haven't asked
explicitly for, either, right?

I don't get the "It's terrible if I run man or git it won't open a
pager, but I don't want pydoc to use a pager" viewpoint. It just doesn't
make any sense to me.

> Let me recite one of my favourite rules of thumb:
>
> If it can't be turned off, it's not a feature. - Karl Heuer

You turn it off by setting PAGER=cat. The fact that pydoc uses a pager
is the *same* feature as the fact that git and man do.

[Grant Edwards]

On 2016-05-01, Random832 <rand...@fastmail.com> wrote:
> On Sat, Apr 30, 2016, at 19:51, c...@zip.com.au wrote:
>> _When_ they want a pager.
>
> Why would they need an environment variable at all in that case, rather
> than explicitly invoking the pager by name?

We don't want to use a PAGER variable to specify when we want a pager
and when we don't want a pager. If we want a pager we append "| less"
to the command. If we don't want a pager, we don't append that to the
command.

For historical reasons, "man" invokes a pager by default. I'm not
crazy about that. If you want to argue that it's being inconsistent
then you're right: man shouldn't invoke a pager either.

--
Grant

[Random832]

On Sat, Apr 30, 2016, at 22:30, Grant Edwards wrote:
> We don't want to use a PAGER variable to specify when we want a pager
> and when we don't want a pager. If we want a pager we append "| less"
> to the command. If we don't want a pager, we don't append that to the
> command.

Setting PAGER=cat - permanently, in your profile, and never ever
changing it - gives this result, yet it's described by some people here
as "So I have to cripple my shell [...]?" What is crippled? Evidently
you're not "we", since this "crippled" state of affairs seems to be a
perfectly acceptable one to you.

[Steven D'Aprano]

On Sun, 1 May 2016 09:51 am, c...@zip.com.au wrote:

> Let me recite one of my favourite rules of thumb:
>
> If it can't be turned off, it's not a feature. - Karl Heuer

My microwave oven has a safety lock which prevents the mechanism from
operating (generating microwaves) while the door is open. Since I can't
turn it off, it is not a feature! What if I want to stick my head in the
microwave and cook it?

*wink*


Often-wanting-to-microwave-somebody-else's-head-ly y'rs,



--
Steven

[c...@zip.com.au]

This is an interesting argument.

Yes, PAGER=cat would make "man" also not page, and likely almost everything.
And yet I am unwilling to do so. Why?

On reflection, my personal problems with this approach are twofold:

- I want $PAGER to specify my preferred pager when I do want a pager, so
setting it to "cat" does not inform apps about my wishes

- in an interactive shell, yes I can type "| less"; it seems painful (oh for
the happy days of yore when this was spelt "|pg":-)

I am going to try PAGER=cat as an experiment to seem how it feels. Now, I _do_
wish "man" to page for a number of reasons (a) my brain expects it (b) I like
the nice colour highlighting of headings etc (c) manual pages and other
_lengthy_ documents _are_ better paged than not. This last is in contrast to
"help" messages, even lengthy usage messages: these are short enough that I
feel they should not be paged, and if I come across a glaring exception i can
always page them myself.

Fortunately for me, I am already in the position of mucking with "man"; my
despite for GNU info with its weird non-paging browser and the accompanying GNU
manual pages which all too frequently say "man is defunct, go look in info - we
don't bother with man" drove me to rewrite "man" to try to suck in info files
into flat pagable text. So telling my shell this:

man(){ PAGER=less command man ${1+"$@"}; }

is no hardship.

I'll give this a go and see how it feels.

Cheers,
Cameron Simpson <c...@zip.com.au>

[Chris Angelico]

On Sun, May 1, 2016 at 3:24 PM, <c...@zip.com.au> wrote:
> Yes, PAGER=cat would make "man" also not page, and likely almost everything.
> And yet I am unwilling to do so. Why?
>
> On reflection, my personal problems with this approach are twofold:
>
> - I want $PAGER to specify my preferred pager when I do want a pager, so
> setting it to "cat" does not inform apps about my wishes

So you expect the environment variable to say which of multiple pagers
you might want, but only when you already want a pager. Okay. How is
an app supposed to know whether or not to use a pager? How do you
expect them to mindread?

ChrisA

[c...@zip.com.au]

I think for several of us, we do not expect the app to mindread. Don't page for
short output!

As the rest of my article remarks, I at least think "man" should page on the
premise than manual pages will be long enough to benefit, as they should be.

Aside: especially if one uses "less" and includes the -d and -F options in the
$LESS envvar, which suppresses the warning about "dumb" terminals and autoquits
if the file fits on the screen - these two provide most of the painfree
behaviour for short outputs and embedded ttys at least.

We could fork a separate discussion on making pagers more seamless, and
terminal emulators with nice modes to reduce the need for pagers.

Cheers,
Cameron Simpson <c...@zip.com.au>

[alister]

all the discussion on the pager variable is interesting but it overlooks
what I consider to be a very important lesson on program output.

You have no way of knowing what the users output device is & have no
right to dictate what that should be.

alternative outputs for the command line could be.

a teletype printer
a text to speech reader
a Braille terminal
or a computer to (dead) parrot) interface

which is why mot of us here all agree, just output the data, let the end
users environment decide how to present it, that is the users choice not
the programmers.




--
Get in touch with your feelings of hostility against the dying light.
-- Dylan Thomas [paraphrased periphrastically]

[Steven D'Aprano]

Easy: if the READPAGERENVIRONVAR is set, then the application should read
the PAGER environment variable, otherwise it should ignore it.




--
Steven

[Steven D'Aprano]

On Sun, 1 May 2016 05:28 pm, c...@zip.com.au wrote:

> On 01May2016 16:44, Chris Angelico <ros...@gmail.com> wrote:

>>So you expect the environment variable to say which of multiple pagers
>>you might want, but only when you already want a pager. Okay. How is
>>an app supposed to know whether or not to use a pager? How do you
>>expect them to mindread?
>
> I think for several of us, we do not expect the app to mindread. Don't
> page for short output!

Is there an environment variable to tell the application what you
consider "short", or should it read your mind?

Personally, I'd rather use a pager for 3 lines than print 30 lines of help
text directly to the console, but others may feel differently.



--
Steven

[Chris Angelico]

On Sun, May 1, 2016 at 8:55 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> On Sun, 1 May 2016 05:28 pm, c...@zip.com.au wrote:
>
>> On 01May2016 16:44, Chris Angelico <ros...@gmail.com> wrote:
>
>>>So you expect the environment variable to say which of multiple pagers
>>>you might want, but only when you already want a pager. Okay. How is
>>>an app supposed to know whether or not to use a pager? How do you
>>>expect them to mindread?
>>
>> I think for several of us, we do not expect the app to mindread. Don't
>> page for short output!
>
> Is there an environment variable to tell the application what you
> consider "short", or should it read your mind?

How about $LINES? If it's less than that, it'll fit on one screen. Of
course, that still won't be perfect, but it's a definite improvement
over guessing.

ChrisA

[Grant Edwards]

On 2016-05-01, Chris Angelico <ros...@gmail.com> wrote:
> On Sun, May 1, 2016 at 3:24 PM, <c...@zip.com.au> wrote:
>> Yes, PAGER=cat would make "man" also not page, and likely almost everything.
>> And yet I am unwilling to do so. Why?
>>
>> On reflection, my personal problems with this approach are twofold:
>>
>> - I want $PAGER to specify my preferred pager when I do want a pager, so
>> setting it to "cat" does not inform apps about my wishes
>

> So you expect the environment variable to say which of multiple pagers
> you might want, but only when you already want a pager.

Yes!

Just like EDITOR specifies which editor to use _when_ _you_ _want_
_to_ _use_ _an_ _editor_. It doesn't tell programs to invoke an
editor all the time.

> Okay. How is an app supposed to know whether or not to use a pager?

Command line option.

> How do you expect them to mindread?

Nope, just recognize '-p' or somesuch.

--
Grant

[Marko Rauhamaa]

Grant Edwards <grant.b...@gmail.com>:

In discussions like these, it would be important to draw from
precedents. Are there commands that have such an option?

I could only find:

mysql --pager CMD

which seems sensible but nothing like an industry standard.

Personally, I wouldn't bother with builtin paging.


Marko

[Grant Edwards]

On 2016-05-01, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Grant Edwards <grant.b...@gmail.com>:
>
>> On 2016-05-01, Chris Angelico <ros...@gmail.com> wrote:
>>> Okay. How is an app supposed to know whether or not to use a pager?
>> Command line option.
>>
>>> How do you expect them to mindread?
>> Nope, just recognize '-p' or somesuch.
>
> In discussions like these, it would be important to draw from
> precedents. Are there commands that have such an option?

It's pretty rare. It is assumed that Unix uses can type " | less" if
they want to view the output of a program with a pager. That's
simpler and faster than spending time to try to figure out if and how
you tell some particular application to invoke a pager for you.

> I could only find:
>
> mysql --pager CMD
>
> which seems sensible but nothing like an industry standard.
>
> Personally, I wouldn't bother with builtin paging.

I agree completely. Builtin paging is pretty much pointless -- but if
you _are_ going to do, make it something that you invoke with a
command line option.

--
Grant

[Steven D'Aprano]

On Mon, 2 May 2016 02:30 am, Grant Edwards wrote:

>> In discussions like these, it would be important to draw from
>> precedents. Are there commands that have such an option?
>
> It's pretty rare.  It is assumed that Unix uses can type " | less"

Is nobody except me questioning the assumption that we're only talking about
Unix users?



--
Steven

[Grant Edwards]

Didn't the OP specify that he was writing a command-line utility for
Linux/Unix?

Discussing command line operation for Windows or OS-X seems rather
pointless.

--
Grant

[Gene Heskett]

linux, unix, mauche nichs. Are there others?>
>
> --
> Steven


Cheers, Gene Heskett
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page <http://geneslinuxbox.net:6309/gene>

[Random832]

On Sun, May 1, 2016, at 13:04, Grant Edwards wrote:
> On 2016-05-01, Steven D'Aprano <st...@pearwood.info> wrote:

> > Is nobody except me questioning the assumption that we're only
> > talking about Unix users?
>

> Didn't the OP specify that he was writing a command-line utility for
> Linux/Unix?

We've been talking about pydoc instead of the OP's program for a while
now.

[Ethan Furman]

Even Windows has "more".

--
~Ethan~

[c...@zip.com.au]

On 01May2016 20:55, Steven D'Aprano <st...@pearwood.info> wrote:
>On Sun, 1 May 2016 05:28 pm, c...@zip.com.au wrote:
>> On 01May2016 16:44, Chris Angelico <ros...@gmail.com> wrote:
>>>So you expect the environment variable to say which of multiple pagers
>>>you might want, but only when you already want a pager. Okay. How is
>>>an app supposed to know whether or not to use a pager? How do you
>>>expect them to mindread?
>>
>> I think for several of us, we do not expect the app to mindread. Don't
>> page for short output!
>
>Is there an environment variable to tell the application what you
>consider "short", or should it read your mind?

We're getting into matters of taste here. It shouldn't read my mind, but of
course when it differs it shows bad taste!

I am taking the line that usage and help messages should fall into the "short"
category, both simply by their nature and also as a design/style criterion for
program authors. Manuals, be they man pages or info or whatever, should be
"long", with specification and ideally explainations for rationale and some
examples.

>Personally, I'd rather use a pager for 3 lines than print 30 lines of help
>text directly to the console, but others may feel differently.

And I am very much the opposite. ["foo --help"; "types next command; huh? I'm
in a pager, not back at my prompt?"]

However, with "less" configured to quit if the text fits on the screen (which
is can usually determine by querying the terminal directly, no magic required),
I get the best of both wolds, possibly to the point that I have rarely noticed
that Python's help() pages.

And I've got mixed feelings about git. It seems that "git help" and "git
--help" produces sensible unpaged short help (42 lines of it, but that is ok to
me). It is "git help <subcommand>" which runs "man git-subcommand", and that is
somewhat defensible because most of the git subcommands have an outrageous
number of options. (And it really does just invoke "man" (by default - that is
also tunable I see); I can tell because it invokes my personal "man" command
instead of the system one or some internally sourced text.)

My constrast, Mercurial (hg) always produces unpaged output for "help" and
"help <subcommand>", and the "help <subcommand>" is succinct and fitting for a
help text. There is a single large "man hg" for when you want the detailed
manual. This approach is more to my liking.

Cheers,
Cameron Simpson <c...@zip.com.au>

[c...@zip.com.au]

On 01May2016 21:23, Chris Angelico <ros...@gmail.com> wrote:

>On Sun, May 1, 2016 at 8:55 PM, Steven D'Aprano <st...@pearwood.info> wrote:
>> Is there an environment variable to tell the application what you
>> consider "short", or should it read your mind?
>

>How about $LINES? If it's less than that, it'll fit on one screen. Of
>course, that still won't be perfect, but it's a definite improvement
>over guessing.

On terminal emulators you can normally query the terminal directly for its
current size (look at the output of "stty -a" for example), and pagers do. This
is better than $LINES, which is really a convenience thing presented by some
shells like bash and which won't magicly change if you resize your terminal
until bash gets another look.

If your pager can be told to autoquit if the output fits then this pain point
(whatever your preference) can be largely obviated.

Cheers,
Cameron Simpson <c...@zip.com.au>

[c...@zip.com.au]

On 01May2016 17:04, Grant Edwards <grant.b...@gmail.com> wrote:

>On 2016-05-01, Steven D'Aprano <st...@pearwood.info> wrote:
>> On Mon, 2 May 2016 02:30 am, Grant Edwards wrote:
>>

>>>> In discussions like these, it would be important to draw from
>>>> precedents. Are there commands that have such an option?
>>>

>>> It's pretty rare.  It is assumed that Unix uses can type " | less"
>>
>> Is nobody except me questioning the assumption that we're only
>> talking about Unix users?
>

>Didn't the OP specify that he was writing a command-line utility for
>Linux/Unix?
>

>Discussing command line operation for Windows or OS-X seems rather
>pointless.

OS-X _is_ UNIX. I spent almost all my time on this Mac in terminals. It is a
very nice to use UNIX in many regards.

Cheers,
Cameron Simpson <c...@zip.com.au>

Mac OS X. Because making Unix user-friendly is easier than debugging Windows.
- Mike Dawson, Macintosh Systems Administrator and Consultation.
mda...@mac.com http://herowars.onestop.net

[Steven D'Aprano]

On Mon, 2 May 2016 03:04 am, Grant Edwards wrote:

> On 2016-05-01, Steven D'Aprano <st...@pearwood.info> wrote:
>> On Mon, 2 May 2016 02:30 am, Grant Edwards wrote:
>>
>>>> In discussions like these, it would be important to draw from
>>>> precedents. Are there commands that have such an option?
>>>
>>> It's pretty rare.  It is assumed that Unix uses can type " | less"
>>
>> Is nobody except me questioning the assumption that we're only
>> talking about Unix users?
>
> Didn't the OP specify that he was writing a command-line utility for
> Linux/Unix?

*cough* I'm the OP, and no I didn't.

Obviously I'm a Linux user myself, but I'm presumptuous enough to hope that
when I release the utility publicly[1], others may find it of some small
use. Including Windows users.




[1] Real Soon Now.


--
Steven

[Terry Reedy]

As a Windows user in recent years, I expect -h to give me a list of
options, hopefully with some annotation beyond the bare bones, that give
the signature of the command (regarding it as a function call). 'python
-h' is pretty bare bones. 'python -m test -h' is much better. I expect
both to tell me how to properly pass a file argument. I don't expect
either to tell me how write a python or unittest file. I use the manual
for this.

--
Terry Jan Reedy

[Grant Edwards]

On 2016-05-01, c...@zip.com.au <c...@zip.com.au> wrote:

>>Didn't the OP specify that he was writing a command-line utility for
>>Linux/Unix?
>>

>>Discussing command line operation for Windows or OS-X seems rather
>>pointless.
>
> OS-X _is_ UNIX. I spent almost all my time on this Mac in terminals. It is a
> very nice to use UNIX in many regards.

I include what you're doing under the category "Unix". When I talk
about "OS X", I mean what my 84 year old mother is using. I assumed
everybody thought that way. ;)

--
Grant Edwards grant.b.edwards Yow! If I am elected no one
at will ever have to do their
gmail.com laundry again!

[c...@zip.com.au]

On 02May2016 14:07, Grant Edwards <grant.b...@gmail.com> wrote:
>On 2016-05-01, c...@zip.com.au <c...@zip.com.au> wrote:
>
>>>Didn't the OP specify that he was writing a command-line utility for
>>>Linux/Unix?
>>>
>>>Discussing command line operation for Windows or OS-X seems rather
>>>pointless.
>>
>> OS-X _is_ UNIX. I spent almost all my time on this Mac in terminals. It is a
>> very nice to use UNIX in many regards.
>
>I include what you're doing under the category "Unix". When I talk
>about "OS X", I mean what my 84 year old mother is using. I assumed
>everybody thought that way. ;)

Weird. My 79 year old mother uses "Apple". I can only presume there's no "OS X"
for her.

Cheers,
Cameron Simpson <c...@zip.com.au>