feedback on changes to tools.cli

53 views
Skip to first unread message

gaz jones

unread,
Oct 13, 2011, 7:05:48 PM10/13/11
to cloju...@googlegroups.com
Hi,

I am updating the contrib project tools.cli to fix a couple of issues
that were pointed out to me by someone trying to use it, and would
appreciate some feedback on the the solution I have implemented before
i push it.

The main issue was the inability to provide arguments that you do not
want processed as "options". For example:

java -cp my.jar --foo bar myfile.txt

where "foo" is an option to be set to "bar" and "myfile.txt" is a
parameter that is not to be parsed, but is needed in the program
execution. My solution to this problem is to use the BASH standard of
"--" to signify the end of the options to the program:

java -cp my.jar --foo bar -- myfile.txt

This allows for a simple implementation and seems like a reasonable
thing to do, as the standard is already there and people are
(possibly) familiar with it. These arguments then get added to the
resulting command line args map assoc'd to a key of :trailing-args:

{:foo "bar"
:trailing-args ["myfile.txt"]}

The other option I was considering was to instead return a tuple of
[options-map, trailing-args] but this breaks the existing contract and
it didnt seem particularly clearer to me.

Any thoughts / suggestions are much appreciated :D

Cheers!

Gaz

Dave Ray

unread,
Oct 13, 2011, 9:19:21 PM10/13/11
to cloju...@googlegroups.com
Gaz,

I've used tools.cli recently and also noticed the same issue. I'd be
ok with the "--" solution, but I'd prefer support for getting trailing
arguments without it. Most CLI parsers I've used work that way. In
other news, it would also be nice if it was possible to customize the
failure behavior. I'd prefer cli throw an exception or call an error
handling function rather than just calling System/exit.

Cheers,

Dave

> --
> You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
> To post to this group, send email to cloju...@googlegroups.com.
> To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.
>
>

Stuart Halloway

unread,
Oct 13, 2011, 11:30:16 PM10/13/11
to cloju...@googlegroups.com
Gaz,

I've used tools.cli recently and also noticed the same issue. I'd be
ok with the "--" solution, but I'd prefer support for getting trailing
arguments without it. Most CLI parsers I've used work that way. In
other news, it would also be nice if it was possible to customize the
failure behavior. I'd prefer cli throw an exception or call an error
handling function rather than just calling System/exit.

Cheers,

Dave

Agreed with Dave, no System/exit. It would also be nice if the specification was made of data structures, instead of magic functions. (There could still be magic functions that emitted the data spec.)

Stuart Halloway
Clojure/core
http://clojure.com

Sean Corfield

unread,
Oct 14, 2011, 1:16:07 AM10/14/11
to cloju...@googlegroups.com
On Thu, Oct 13, 2011 at 4:05 PM, gaz jones <gareth....@gmail.com> wrote:
> My solution to this problem is to use the BASH standard of
> "--" to signify the end of the options to the program:
>
> java -cp my.jar --foo bar -- myfile.txt

I don't like this at all. I don't feel it's a standard at all. I can't
think of a single program I interact with that needs -- to indicate
the start of the "trailing" args.

IMO, the first argument that doesn't begin with "-" (or "--" of
course) should start the trailing args.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

Ben Smith-Mannschott

unread,
Oct 14, 2011, 4:34:22 AM10/14/11
to cloju...@googlegroups.com
On Fri, Oct 14, 2011 at 07:16, Sean Corfield <seanco...@gmail.com> wrote:
> On Thu, Oct 13, 2011 at 4:05 PM, gaz jones <gareth....@gmail.com> wrote:
>> My solution to this problem is to use the BASH standard of
>> "--" to signify the end of the options to the program:
>>
>> java -cp my.jar --foo bar -- myfile.txt
>
> I don't like this at all. I don't feel it's a standard at all. I can't
> think of a single program I interact with that needs -- to indicate
> the start of the "trailing" args.
>
> IMO, the first argument that doesn't begin with "-" (or "--" of
> course) should start the trailing args.

That can work, but that implies that the arity of options is known and
fixed. the example above can only be parsed unambiguously if we know
that --foo always takes exactly one argument ("bar") in this case. I
don't know tools.cli well enough to know if that's the case. (If it
isn't, perhaps it should be.)

The "--" is a common convention, but it's usually only used when one
of the trailing arguments might otherwise be confused with an option:
i.e. a file name that begins with a hyphen. In practice that's quite
rare.

// Ben

Cosmin Stejerean

unread,
Oct 14, 2011, 4:35:24 AM10/14/11
to cloju...@googlegroups.com
On Fri, Oct 14, 2011 at 1:16 PM, Sean Corfield <seanco...@gmail.com> wrote:
>> java -cp my.jar --foo bar -- myfile.txt

The only reason I've used -- to indicate end of arguments was to
specifically pass arguments that start with - to the underlying tool.
It doesn't feel natural to require it for the above example.


--
Cosmin Stejerean
http://offbytwo.com

Alan Malloy

unread,
Oct 14, 2011, 5:00:26 AM10/14/11
to Clojure Dev
At around the same time Gareth was starting on clargon (now
tools.cli), I built a mostly-working option parser named clojopts. I
tried to add too many features and configurable bits, so it's not
really ready for prime-time, but I think it's relevant here because
rather than rolling my own parsing I used the java port of GNU getopt,
which parses arguments in exactly the way everyone is used to. Gareth,
have you considered the possibility of using that? Basically I took a
map from the user, which I used to build a getopt string and then do
further clojure-oriented processing on the results from getopt.

On Oct 14, 1:35 am, Cosmin Stejerean <cos...@offbytwo.com> wrote:

Shantanu Kumar

unread,
Oct 14, 2011, 5:28:58 AM10/14/11
to cloju...@googlegroups.com
On Fri, Oct 14, 2011 at 2:30 PM, Alan Malloy <al...@malloys.org> wrote:
At around the same time Gareth was starting on clargon (now
tools.cli), I built a mostly-working option parser named clojopts. I
tried to add too many features and configurable bits, so it's not
really ready for prime-time, but I think it's relevant here because
rather than rolling my own parsing I used the java port of GNU getopt,
which parses arguments in exactly the way everyone is used to. Gareth,
have you considered the possibility of using that? Basically I took a
map from the user, which I used to build a getopt string and then do
further clojure-oriented processing on the results from getopt.

Last when I tried that I noticed the Java port was a signed JAR that didn't play well with "lein uberjar", so I gave up. Looks good to me otherwise.

Shantanu

Stefan Kamphausen

unread,
Oct 14, 2011, 6:02:41 AM10/14/11
to cloju...@googlegroups.com
Hi,

for the record: I use this functionality in grep a lot.  Where I work I often deal with a suite of programs all of which have been called using the option '-w'.  So to find out which programs are running I run grep (the program) in the following pipe 'ps aux | grep -- -w' to tell grep that '-w' is not an option but the search pattern.  Afterwards I use visual grep (the eyes) to find the running programs quickly.

Regards,
Stefan

gaz jones

unread,
Oct 14, 2011, 11:45:56 AM10/14/11
to cloju...@googlegroups.com
Thanks for all the feedback, glad I asked before going ahead with the
"--" approach as it is obviously not popular! Guess I'm one of the few
people who ends up using that regularly in bash.

Interestingly enough, the "--" is also used by GNU getopt java port to
signify end of arguments:
http://www.urbanophile.com/~arenn/hacking/getopt/gnu.getopt.Getopt.html#_top_
(not that I'm suggesting that is a reason to use "--").

I'll have a think, and try implementing some of the suggestions.

cheers,
gaz

> --
> You received this message because you are subscribed to the Google Groups
> "Clojure Dev" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/clojure-dev/-/akSLwM1S30IJ.

gaz jones

unread,
Oct 24, 2011, 11:15:25 AM10/24/11
to cloju...@googlegroups.com
hey, apologies this took a while i've been a bit ill with the
man-flu... anyway i took the feedback here and from the original
reporter of the bug into account and spent a few hours reworking this
last night. i have a branch on github if anyone has the time to look
and comment that would be great. the readme documents the new
interface fairly comprehensively:

https://github.com/clojure/tools.cli/tree/next

in summary:

* no more magic functions, plain vector to create a spec
* exceptions while parsing bubble up and need to be handled by the app
* grouping or 'nested' arguments removed to simplify and make less wacky
* bugs around boolean/flag handling no longer exist
* trailing args are collected and found under a :args key in the resulting map

feedback welcome...

Kevin Downey

unread,
Oct 24, 2011, 1:49:27 PM10/24/11
to cloju...@googlegroups.com
libraries that call System/exit are completely unacceptable.

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

Alan Malloy

unread,
Oct 24, 2011, 1:53:04 PM10/24/11
to Clojure Dev
I think everyone can get behind -- signifying end of options, but
having it be the *only* way to end options is pretty awful. Consider
these two command-lines:

$ myprog -n1 bar.clj -x # cmd 1

$ myprog -n1 -- bar.clj -x # cmd 2

GNU getopt (and the java port) parse (1) as having options -x and -n1,
and leave bar.clj in your args[], but cmd 2 as having options -n1, and
args = [bar.clj, -x].

On Oct 14, 8:45 am, gaz jones <gareth.e.jo...@gmail.com> wrote:
> Thanks for all the feedback, glad I asked before going ahead with the
> "--" approach as it is obviously not popular! Guess I'm one of the few
> people who ends up using that regularly in bash.
>
> Interestingly enough, the "--" is also used by GNU getopt java port to
> signify end of arguments:http://www.urbanophile.com/~arenn/hacking/getopt/gnu.getopt.Getopt.ht...

gaz jones

unread,
Oct 24, 2011, 2:12:15 PM10/24/11
to cloju...@googlegroups.com
so as it stands in the branch currently:

$ myprog -n1 bar.clj -x # cmd 1
$ myprog -n1 -- bar.clj -x # cmd 2

with a definition like:

(cli ["-n1" "bar.clj" "-x"]
["-n1" :flag true]
["-x" :flag true])

(1) results in {:n1 true :x true :args ["bar.clj"]} (matches GNU getopt)
(2) results in exception telling you "--" is not a valid argument.

adding support to make it match GNU getopt for "--" is trivial, and it
seems to make sense to do so.

On Mon, Oct 24, 2011 at 12:53 PM, Alan Malloy <al...@malloys.org> wrote:
> bar.clj

gaz jones

unread,
Oct 24, 2011, 2:19:31 PM10/24/11
to cloju...@googlegroups.com
suggestions for alternative solutions for the --help scenario are
completely acceptable. hook function with banner passed as arg?

i kind of feel like all i would do in the hook would be to print the
banner to stdout and then system/exit...

Paul Stadig

unread,
Oct 24, 2011, 2:41:18 PM10/24/11
to cloju...@googlegroups.com
On Mon, Oct 24, 2011 at 2:19 PM, gaz jones <gareth....@gmail.com> wrote:
suggestions for alternative solutions for the --help scenario are
completely acceptable. hook function with banner passed as arg?

i kind of feel like all i would do in the hook would be to print the
banner to stdout and then system/exit...

Something like that would be better. I think the case that Kevin is thinking about is if you have a long running JVM process that you attach to with a command-line tool (like we do at work). If someone is on a server and does `my-cool-server --help` it would kill the JVM process, which is bad.

Some level of hooking for the library consumer to choose what to do would be better than just calling System/exit.


Paul

gaz jones

unread,
Oct 29, 2011, 2:08:46 PM10/29/11
to cloju...@googlegroups.com
ok, you now get options, extra arguments, and a banner returned as a
vector. roll your own help if you want it:

(defn -main [& args]
(let [[options extra banner] (cli args
["-h" "--help" "HALP!" :default
false :flag true]
["-f" "--faux" "The faux du fafa"]
["-c" "--conchords" "How many
conchords?" :default 2 :parse-fn #(Integer. %)])]
(when (:help options)
(println banner)
(System/exit 0))
(println "options:" options)
(println "extra:" extra)))

i think this addresses all the concerns / original issues (no magic
functions, no System/exits, consistent access to trailing / extra
arguments, booleans) and im pretty happy with the interface so i will
likely merge this into master...

Reply all
Reply to author
Forward
0 new messages