Using the flag package to parse custom strings

2,162 views
Skip to first unread message

Alexander Orlov

unread,
May 17, 2011, 3:50:36 PM5/17/11
to golang-nuts
I want to extract commands and parameters from a string according to
the parsing schema of the flag package. But the string isn't provided
to the Go binary from the initialization on. I want to parse arbitrary
strings during runtime. However AFAIK the flag package has no function
to set a base string during runtime that can be parsed later?

Doing

var flagvar int
flag.IntVar(&flagvar, "flagname", 1234, "help message for flagname")

in my function I get a runtime error. Also it's not really what I
want. I want to

1. provide a base string (containing parameters with values and value-
less params/"switches")
2. get all parameters of this base string
3. get all values for each parameter
4. get the count of all params
5. get the count of all values of a single parameter

Can I do it using the flag package or is there another package you'd
recommend?

Kyle Lemons

unread,
May 17, 2011, 4:07:27 PM5/17/11
to Alexander Orlov, golang-nuts
On Tue, May 17, 2011 at 12:50 PM, Alexander Orlov <alexand...@loxal.net> wrote:
I want to extract commands and parameters from a string according to
the parsing schema of the flag package. But the string isn't provided
to the Go binary from the initialization on. I want to parse arbitrary
strings during runtime. However AFAIK the flag package has no function
to set a base string during runtime that can be parsed later?
The flag package uses os.Args for parsing.  I'm not sure what you mean by "the string isn't provided to the Go binary from the initialization on," but it is very standard for the command-line to already be split by the time a program starts executing (nearly every shell in existence does this before execvp).  If you are talking about non-flag parameter strings, then these are available after parsing via flag.Args.
 

Doing

var flagvar int
flag.IntVar(&flagvar, "flagname", 1234, "help message for flagname")

in my function I get a runtime error. Also it's not really what I
want. I want to
Provide the error message and we might be able to point you as to what's going wrong.  Also, most programs don't use the *Var mechanism; often the x := Type(...) is used.
 

1. provide a base string (containing parameters with values and value-
less params/"switches")
This is the only thing that isn't done by the flag package.  However, if you split your string yourself and assign it to os.Args, you can emulate this behavior.  The real question is WHY you need this.  Command-line flags are an efficient way of passing arguments to a program by hand, but not always the best way to get them from other places.
 
2. get all parameters of this base string
3. get all values for each parameter
4. get the count of all params
5. get the count of all values of a single parameter

Can I do it using the flag package or is there another package you'd
recommend?



--
~Kyle

"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?" 
— Brian Kernighan

Alexander Orlov

unread,
May 17, 2011, 10:02:10 PM5/17/11
to golang-nuts
On May 17, 10:07 pm, Kyle Lemons <kev...@google.com> wrote:
> On Tue, May 17, 2011 at 12:50 PM, Alexander Orlov <alexander.or...@loxal.net
> > 1. provide a base string (containing parameters with values and value-
> > less params/"switches")
>
> This is the only thing that isn't done by the flag package.  However, if you
> split your string yourself and assign it to os.Args, you can emulate this
> behavior.  The real question is WHY you need this.  Command-line flags are
> an efficient way of passing arguments to a program by hand, but not always
> the best way to get them from other places.

Why? I need it to handle user's command-line style input. Actually I
pass a r.URL.Raw that can look like "/command%20--param%20value%20--
param1%20value1%20--paramSwitch". I know that passing a JSON is a
better way to communicate with a RESTful API but for humans entering
JSON-wrapped commands isn't the best solution. So I want to parse this
r.URL.Raw or r.FormValue("commandWithParams") with the flag package
but modifying Appengine's os.Args isn't the best solution I guess. So
I can't use the flag package to parse this kind of commands?

Kyle Lemons

unread,
May 18, 2011, 11:49:10 AM5/18/11
to Alexander Orlov, golang-nuts
Sure you can. Do you know that the appengine sdk uses flag on its
own? If it doesn't, then you can do what I described: Split the
string by space (after urldecoding) and give it to os.Args.

It would also be a trivial CL to add functionality to the flag package
to add a flag.ParseStrings(args[] string). Or even a
flag.ParseString(cli string), but with IFS issues I think the former
would be preferential. I don't know if it would be accepted or not,
though.

The better way might be to sit down with your "/command"s and figure
out some logical "path names" for your use cases. This is similar,
but not identical to, designing a RESTful API (which makes the URIs
into entity names and uses the HTTP methods GET, POST, etc to modify
or query them). For example,
/users - list by default
/users/list - list explicitly
/users/list?from=20100408 - list users who registered on or after a date
/users/add?name=foo&password=bar - add a user

These URLs make a lot of sense to someone who's reading them, don't
have URLencoded spaces to deal with, and are not difficult to type.

--

Alexander Orlov

unread,
May 18, 2011, 12:43:44 PM5/18/11
to golang-nuts
On May 18, 5:49 pm, Kyle Lemons <kev...@google.com> wrote:
> It would also be a trivial CL to add functionality to the flag package
> to add a flag.ParseStrings(args[] string).  Or even a
> flag.ParseString(cli string), but with IFS issues I think the former
> would be preferential.  I don't know if it would be accepted or not,
> though.

Yep, that's what I've done. Also I can't understand why this
functionality isn't provided out-of-the-box. Or is the CLI not
considered as a good (web) UI replacement anymore...? I'm so fast
working with the CLI — why not bringing it to the web?

> /users - list by default
> /users/list - list explicitly
> /users/list?from=20100408 - list users who registered on or after a date
> /users/add?name=foo&password=bar - add a user

A RESTful API isn't the only thing I want. A RESTful API is a good
thing for machines and devs, not for non-devs. I take the command and
its params from http.Request.URL.Raw only in the first step. Later the
whole command string can come from anywhere. So I really need the flag
pkg and without the os.Args default.

André Moraes

unread,
May 18, 2011, 12:50:19 PM5/18/11
to golang-nuts
I wrote a very simple test program, that breaks the string into a
valid argument array.

Note that it break in spaces while it shoud check for arguments on
quotes, and it can't handle unix-like arguments.
But is a start. :)

/**
Author: Andre Luiz Alves Moraes
Barbacena - MG - Brasil
*/

package main

import (
"bytes"
"fmt"
)

func main() {
argument := "-param1 value -param2 other rest of the string"
argv := make([]string, 1)

buff := bytes.NewBufferString(argument)

for {
rune, _, _ := buff.ReadRune()
if rune == '-' {
arg, _ := buff.ReadString(' ')
val, _ := buff.ReadString(' ')
argv = append(argv, arg, val)
} else {
rest := string(rune) + buff.String()
argv = append(argv, rest)
break
}

if buff.Len() == 0 {
break
}
}

fmt.Printf("argv: %v\n", argv)

}


--
André Moraes
http://andredevchannel.blogspot.com/

André Moraes

unread,
May 18, 2011, 12:51:12 PM5/18/11
to golang-nuts
And I didn't check the errors from functions.

Alexander Orlov

unread,
May 18, 2011, 12:56:05 PM5/18/11
to André Moraes, golang-nuts
2011/5/18 André Moraes <and...@gmail.com>
I wrote a very simple test program, that breaks the string into a
valid argument array.



Very nice, thx. However to have a correspoinding functionality within Go's standard pkgs would be nice.

roger peppe

unread,
May 18, 2011, 1:01:25 PM5/18/11
to Alexander Orlov, André Moraes, golang-nuts
2011/5/18 Alexander Orlov <alexand...@loxal.net>:

strings.Fields provides the above functionality.
you might be able to get the scanner package to help
you parse quoted strings.

André Moraes

unread,
May 18, 2011, 1:08:21 PM5/18/11
to roger peppe, Alexander Orlov, golang-nuts
>>
>> Very nice, thx. However to have a correspoinding functionality within Go's
>> standard pkgs would be nice.
>
> strings.Fields provides the above functionality.
> you might be able to get the scanner package to help
> you parse quoted strings.
>

string.Fiels break only on the spaces.

My code break on spaces only for the arguments switches.
When the code don't find a "-" as the rune, it reads the rest of the
line as single string (maybe that isn't the best behavior) but is
different from string.Fields.

Also, scanner package definitely will be a best tool for this parsing job.

Reply all
Reply to author
Forward
0 new messages