Re: questions about argparse
Ted Stern
>
> On Mon, Nov 8, 2010 at 11:38 PM, Ted Stern <dodec...@gmail.com> wrote:
>> I have some questions about argparse that don't seem to be covered by
>> the documentation.
>
> Probably best to send such emails to the argparse users list
> (http://groups.google.com/group/argparse-users) so others can have the
> benefit of the answers too. ;-)
Okay, I've subscribed (using another address) and am replying to the
list.
>
>> - With subparsers, the only option for help seems to be -h or --help.
>> I'd like to do, for example,
>>
>> mycommand.py mysubopt help
>>
>> instead. Is it possible to extend this?
>
> Maybe something like:
>
> class HelpAction(argparse.Action):
> def __call__(self, parser, namespace, values, option_string=None):
> if values is not None:
> parser.print_help()
> parser.exit()
>
> parser = argparse.ArgumentParser()
> subparsers = parser.add_subparsers()
> subopt = subparsers.add_parser('subopt')
> subopt.add_argument('help', nargs='?', action=HelpAction, choices=['help'])
> args = parser.parse_args()
>
> But that may or may not work depending on what other arguments
> "mysubopt" is supposed to take. For example, if "mysubopt" can also
> take positional arguments that aren't "help", it might be hard to do
> this in argparse.
I don't anticipate using positional arguments for subparsers, so this
is adequate. Thanks!
>
>> - I have a slightly different kind of subparsing that I currently do
>> with a class inherited from OptionParser:
>>
>> --suboption key1=val1,key2=val2 --suboption key3=val3
>>
>> In other words, I have an optional argument named --suboption, and I
>> can set up a set of keywords to be set.
>
> Assuming you want these key=val pairs set as attributes on the
> namespace returned by .parse_args() you can do this like:
>
> class KeywordsAction(argparse.Action):
> def __call__(self, parser, namespace, value, option_string=None):
> for key_val in value.split(","):
> key, val = key_val.split("=")
Small modification
# Split on the first equals sign only
key, val = key_val.split("=",1)
> setattr(namespace, key, val)
>
> parser = argparse.ArgumentParser()
> parser.add_argument('--suboption', action=KeywordsAction,
> dest=argparse.SUPPRESS)
> args = parser.parse_args()
Okay, I see how this works. It is part way to what I want but not all
the way there. Problems:
* what if you have more than one suboption, but in each one you have
same-named keys?
* How do you restrict the set of allowed keys for a suboption?
* How do you show Help for the available keys of a suboption?
What I have now with my SubOptionParser subclass of Optionparser is
actually another SubOptionParser as a component of the class. Then
the keys are actually double-hyphen optional arguments of that
suboption parser. After I parse the suboption arguments in the top
level parser, I add double hyphens before parsing the subparser. I
also subclass the Help function to strip out the double-hyphens if I
do
--suboption help
This enables me to have a special help listing for allowed keys under
the suboption, using the machinery already present for option parsing.
I could duplicate the same thing in argparse, but it seems like a
functionality that might be useful to others, and I have seen similar
types of options in non-Python software.
>
> Hope that helps,
>
> Steve
(Yes, it did)
Ted
--
Frango ut patefaciam -- I break so that I may reveal
Steven Bethard
>>> - I have a slightly different kind of subparsing that I currently do
>>> with a class inherited from OptionParser:
>>>
>>> --suboption key1=val1,key2=val2 --suboption key3=val3
>>>
>>> In other words, I have an optional argument named --suboption, and I
>>> can set up a set of keywords to be set.
>>
>> Assuming you want these key=val pairs set as attributes on the
>> namespace returned by .parse_args() you can do this like:
>>
>> class KeywordsAction(argparse.Action):
>> def __call__(self, parser, namespace, value, option_string=None):
>> for key_val in value.split(","):
>> key, val = key_val.split("=")
>
> Small modification
>
> # Split on the first equals sign only
> key, val = key_val.split("=",1)
>
>> setattr(namespace, key, val)
>>
>> parser = argparse.ArgumentParser()
>> parser.add_argument('--suboption', action=KeywordsAction,
>> dest=argparse.SUPPRESS)
>> args = parser.parse_args()
>
> Okay, I see how this works. It is part way to what I want but not all
> the way there. Problems:
>
> * what if you have more than one suboption, but in each one you have
> same-named keys?
What behavior do you want? The code above will overwrite the earlier
values for a key with later values.
> * How do you restrict the set of allowed keys for a suboption?
There are a few ways you could do this. A simple one might be to
create a custom type:
class SuboptionType:
def __init__(self, valid_keys):
self.valid_keys = valid_keys
def __call__(self, value):
for key_val in value.split(","):
key, val = key_val.split("=",1)
if key not in self.valid_keys:
raise TypeError("invalid key: " + key)
return value
You would then use this like:
parser.add_argument(..., type=SuboptionType(['key1', 'key2', ...]), ...)
> * How do you show Help for the available keys of a suboption?
I'm not sure what this means. Could you elaborate? I think maybe I
don't understand what you intend to do with the key/value pairs...
> What I have now with my SubOptionParser subclass of Optionparser is
> actually another SubOptionParser as a component of the class. Then
> the keys are actually double-hyphen optional arguments of that
> suboption parser. After I parse the suboption arguments in the top
> level parser, I add double hyphens before parsing the subparser. I
> also subclass the Help function to strip out the double-hyphens if I
> do
> --suboption help
>
> This enables me to have a special help listing for allowed keys under
> the suboption, using the machinery already present for option parsing.
>
> I could duplicate the same thing in argparse, but it seems like a
> functionality that might be useful to others, and I have seen similar
> types of options in non-Python software.
I'm still not sure I understand what the intended behavior is. Could
you give a couple examples of valid and invalid command lines and what
they should do?
That said, I'm certainly not opposed to patches that make whatever
you're trying to achieve easier in argparse. You can always submit
such feature requests to bugs.python.org. (Though with the Python 3.2
beta feature freeze coming up pretty soon here, it's likely the
feature wouldn't make it in until Python 3.3.)
Steve
--
Where did you get that preposterous hypothesis?
Did Steve tell you that?
--- The Hiphopopotamus
Ted Stern
Yes, that's the problem exactly. I would like to separate the key:val
pairs for each suboption in a subordinate namespace, not at the top
level. In other words, if you provide
--subopt1 a=a1,b=b1,c=c1 \
--subopt2 a=a2,b=b2,c=c2
you would be able to treat the a key of subopt1 differently from that
subopt2.
However, it would be natural to expect normal dict-style behavior, in
other words
--subopt1 a=a1,b=b1 \
--subopt1 a=a2,b=b2
should store subopt1['a'] as 'a2', not 'a1'. I'm using 'subopt1[...]'
as a placeholder for the actual dict variable until I know what it's
called :-).
>
>> * How do you restrict the set of allowed keys for a suboption?
>
> There are a few ways you could do this. A simple one might be to
> create a custom type:
>
> class SuboptionType:
> def __init__(self, valid_keys):
> self.valid_keys = valid_keys
> def __call__(self, value):
> for key_val in value.split(","):
> key, val = key_val.split("=",1)
> if key not in self.valid_keys:
> raise TypeError("invalid key: " + key)
> return value
>
> You would then use this like:
>
> parser.add_argument(..., type=SuboptionType(['key1', 'key2',
> ...]), ...)
Now we're getting closer! That's kind of what I want. Only, where
do the key,val pairs get stored?
>
>> * How do you show Help for the available keys of a suboption?
>
> I'm not sure what this means. Could you elaborate? I think maybe I
> don't understand what you intend to do with the key/value pairs...
Let's say I have a list of valid keys for a suboption, that I
initialize with the SuboptionType above. I then want to handle one
special key for the type, say 'help', so that if I do
--subopt1 help
I would see
Sub opt help usage: PROG [--subopt a=val[,b=val,...]]
Help string for --subopt1 suboption:
a=VAL a key help string
b=VAL b key help string
...
Is this possible? The way I was doing this with Optparse was with a
huge kluge involving a class that contained an option parser as a
component.
>
>> What I have now with my SubOptionParser subclass of Optionparser is
>> actually another SubOptionParser as a component of the class. Then
>> the keys are actually double-hyphen optional arguments of that
>> suboption parser. After I parse the suboption arguments in the top
>> level parser, I add double hyphens before parsing the subparser. I
>> also subclass the Help function to strip out the double-hyphens if I
>> do
>> --suboption help
>>
>> This enables me to have a special help listing for allowed keys under
>> the suboption, using the machinery already present for option parsing.
>>
>> I could duplicate the same thing in argparse, but it seems like a
>> functionality that might be useful to others, and I have seen similar
>> types of options in non-Python software.
>
> I'm still not sure I understand what the intended behavior is. Could
> you give a couple examples of valid and invalid command lines and what
> they should do?
I hope this makes it clear:
command --subopt1 a=a1,b=b1 \
--subopt1 c=c1,a=alternative \
--subopt2 a=a2,b=b2
should put the key/val pairs into two distinct dicts analogous to the
following:
subopt1[a]=alternative # overwritten on the cmdline
subopt1[b]=b1
subopt1[c]=c1
subopt2[a]=a2
subopt2[b]=b2
And you should be able to get help on the valid keys of the suboption
using something like this:
command --subopt1 help
returns "help" for the separate valid keys of subopt1. Here's an
example from a script called peg5run:
,----
| $ peg5run --pegasus help
|
| Usage: peg5run.py --pegasus [varabbrev=val[,varabbrev=val]]
|
| PEGASUS PBS batch environment variable settings.
|
| Options:
| -h, help show this help message and exit
| vendor_version=VENDOR_VERSION
| Sets environment variable PEGASUS_VENDOR_VERSION
| local_version=LOCAL_VERSION
| Sets environment variable PEGASUS_LOCAL_VERSION
| builder=BUILDER Sets environment variable PEGASUS_BUILDER
| precision=PRECISION Sets environment variable PEGASUS_PRECISION
| parallel=PARALLEL Sets environment variable PEGASUS_PARALLEL
| compiler=COMPILER Sets environment variable PEGASUS_COMPILER
| mpilib=MPILIB Sets environment variable PEGASUS_MPILIB
| config=CONFIG Sets environment variable PEGASUS_CONFIG
| version=VERSION Sets environment variable PEGASUS_VERSION
`----
If it's not obvious, I have some member functions that automate adding
the help text for a bunch of environment variable settings like this.
>
> That said, I'm certainly not opposed to patches that make whatever
> you're trying to achieve easier in argparse. You can always submit
> such feature requests to bugs.python.org. (Though with the Python 3.2
> beta feature freeze coming up pretty soon here, it's likely the
> feature wouldn't make it in until Python 3.3.)
I am considering using argparse-1.1 because it is able to work with
the disparate environment I have to support around my company, Python
versions 2.3.3 up to 2.7.
I would have to make sure any new versions were backward compatible in
the same way as argparse-1.1.
Would it be possible to make the SuboptionType an ArgumentParser?
Ted
>
> Steve
> --
> Where did you get that preposterous hypothesis?
> Did Steve tell you that?
> --- The Hiphopopotamus
--
Steven Bethard
> ,----
> | $ peg5run --pegasus help
> |
> | Usage: peg5run.py --pegasus [varabbrev=val[,varabbrev=val]]
> |
> | PEGASUS PBS batch environment variable settings.
> |
> | Options:
> | -h, help show this help message and exit
> | vendor_version=VENDOR_VERSION
> | Sets environment variable PEGASUS_VENDOR_VERSION
> | local_version=LOCAL_VERSION
> | Sets environment variable PEGASUS_LOCAL_VERSION
> | builder=BUILDER Sets environment variable PEGASUS_BUILDER
> | precision=PRECISION Sets environment variable PEGASUS_PRECISION
> | parallel=PARALLEL Sets environment variable PEGASUS_PARALLEL
> | compiler=COMPILER Sets environment variable PEGASUS_COMPILER
> | mpilib=MPILIB Sets environment variable PEGASUS_MPILIB
> | config=CONFIG Sets environment variable PEGASUS_CONFIG
> | version=VERSION Sets environment variable PEGASUS_VERSION
> `----
I see. That's clearer. Well you can get the parsing behavior you want
(parsing key-value pairs into sub-namespaces) using something like:
class StoreSubnamespaceAction(argparse.Action):
def __init__(self, valid_keys, **kwargs):
if 'default' not in kwargs:
kwargs['default'] = argparse.Namespace()
super(StoreSubnamespaceAction, self).__init__(**kwargs)
self.valid_keys = valid_keys
def __call__(self, parser, namespace, value, option_string=None):
namespace = getattr(namespace, self.dest)
for key_val in value.split(","):
key, val = key_val.split("=", 1)
if key not in self.valid_keys:
parser.error("invalid key: " + key)
setattr(namespace, key, val)
parser = argparse.ArgumentParser()
parser.add_argument(
'--pegasus',
action=StoreSubnamespaceAction,
required=True,
metavar='varabbrev=val[,varabbrev=val]',
valid_keys=['vendor_version', 'local_version', 'builder'])
args = parser.parse_args()
The "args" you get back will then have a sub-namespace based on the
argument, e.g. "args" would be something like
Namespace(pegasus=Namespace(local_version='3.5',
vendor_version='1.2'))
Getting the help formatted in the way you want is a bit more difficult
though - the help formatting of argparse isn't easily extendable (nor
are the formatting APIs public). You could always programmatically
generate the help and insert it into the parser as "description" or
"epilog", but I know that's not as satisfying as having argparse do it
for you.
>> That said, I'm certainly not opposed to patches that make whatever
>> you're trying to achieve easier in argparse. You can always submit
>> such feature requests to bugs.python.org. (Though with the Python 3.2
>> beta feature freeze coming up pretty soon here, it's likely the
>> feature wouldn't make it in until Python 3.3.)
>
> I am considering using argparse-1.1 because it is able to work with
> the disparate environment I have to support around my company, Python
> versions 2.3.3 up to 2.7.
>
> I would have to make sure any new versions were backward compatible in
> the same way as argparse-1.1.
Unfortunately python-dev folks have ripped out some of the backwards
compatibility bits when argparse was moved to Python trunk, so I think
you're stuck with argparse-1.1. However, the approach above should
still work fine with that version.
> Would it be possible to make the SuboptionType an ArgumentParser?
You could do something like that, though you'd probably have to resort
to the arg strings hacking a bit. Something like:
class ParserWrappingAction(argparse.Action):
def __init__(self, parser, **kwargs):
super(ParserWrappingAction, self).__init__(**kwargs)
self.parser = parser
def __call__(self, parser, namespace, values, option_string=None):
arg_strings = ['--' + arg for arg in values.split(',')]
args = self.parser.parse_args(arg_strings)
setattr(namespace, self.dest, args)
pegasus = argparse.ArgumentParser()
pegasus.add_argument('--vendor-version', type=int)
pegasus.add_argument('--mpilib')
parser = argparse.ArgumentParser()
parser.add_argument('--pegasus', action=ParserWrappingAction, parser=pegasus,
required=True, metavar='varabbrev=val[,varabbrev=val]')
args = parser.parse_args()
HTH,