Arguments with whitespace
Heiko Seeberger
--
Blog: heikoseeberger.name
Mark Harrah
On Thu, 4 Aug 2011 16:59:27 +0200
Heiko Seeberger <heiko.s...@googlemail.com> wrote:
> Hi,
>
> How can I pass arguments to the run method that contain a whitespace, e.g.
> run "Mark Harrah"
> where the expected result is *one* argument "Mark Harrah"?
Spec a cross-platform escaping strategy that sbt can use. (i.e., no \ allowed because of windows.)
-Mark
Doug Tangren
Hey Heiko,
Spec a cross-platform escaping strategy that sbt can use. (i.e., no \ allowed because of windows.)
On Thu, 4 Aug 2011 16:59:27 +0200
Heiko Seeberger <heiko.s...@googlemail.com> wrote:
> Hi,
>
> How can I pass arguments to the run method that contain a whitespace, e.g.
> run "Mark Harrah"
> where the expected result is *one* argument "Mark Harrah"?
Ha. I've ran into this to with an plugin that uses input tasks and requires free text input. If you need to do something custom you've got to write a parser for your arguments. Do not expect quoted arguments to be collected as one. Thanks windows >:(
Heiko Seeberger
Spec a cross-platform escaping strategy that sbt can use. (i.e., no \ allowed because of windows.)
Mark Harrah
That wasn't a hint, sorry. It was a comment that I'm happy to include a way to escape spaces if someone comes up with a way to do it that is cross platform. The unix way is to use quotes or precede the space with a backslash. You then have to be able to escape the quotes, which is usually done with a backslash. sbt can't use a backslash because it is the separator on windows.
-Mark
> Thanks,
>
> Heiko
>
Randall R Schulz
> On Fri, 5 Aug 2011 01:12:11 +0200
>
> >
> > Googleing for "cross-platform escaping strategy" doesn't yield
> > valuable results. Any pointers? Or examples?
>
> That wasn't a hint, sorry. It was a comment that I'm happy to
> include a way to escape spaces if someone comes up with a way to do
> it that is cross platform. The unix way is to use quotes or precede
> the space with a backslash. You then have to be able to escape the
> quotes, which is usually done with a backslash. sbt can't use a
> backslash because it is the separator on windows.
As is the forward slash. ... I think that's a hint.
> -Mark
Randall Schulz
Heiko Seeberger
That wasn't a hint, sorry. It was a comment that I'm happy to include a way to escape spaces if someone comes up with a way to do it that is cross platform. The unix way is to use quotes or precede the space with a backslash. You then have to be able to escape the quotes, which is usually done with a backslash. sbt can't use a backslash because it is the separator on windows.
Artem Khodyush
--
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To post to this group, send email to simple-b...@googlegroups.com.
To unsubscribe from this group, send email to simple-build-t...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en.
Mark Harrah
Artem Khodyush <gree...@gmail.com> wrote:
> On Thu, Aug 4, 2011 at 11:38 PM, Heiko Seeberger <
> heiko.s...@googlemail.com> wrote:
>
> > Hmm, what about quotes? And two quotes to escape them?
>
> Why two quotes? On my platform, if I have a quote in a file name, I escape
> it as \", not as "".
>
> See, any escape convention you come up with will be surprising to users from
> 'other platform' camp (or to both).
This is probably true. We need something, though. It is unlikely there is a solution that is intuitive to everyone, so we'll have to go for the next best thing. Further feedback and suggestions are welcome if doubling the quotes is not desirable.
-Mark
Frédérick Dubois
This probably fits in the "surprising to users of all platforms" though…
-Fred
-Fred
Derek Williams
How about backticks?
Paul Phillips
> Backticks are used in Linux (and most likely OSX) to use the output of a
> command as an arg, so I don't think that would work either.
That's bash (among other things, but application-level) and more than
likely it's equally true of bash on windows.
Suggestion: give up, this can't be done "cross-platform" except for that
too-common definition of cross-platform, "sucks for everyone." Let some
meta-mechanism establish the quoting scheme, and default it on each
platform to the sanest default for that platform.
It is a necessary and worthwhile fight to prevent elegance from becoming
the enemy of usefulness.
Artem Khodyush
--Derek Williams
Derek Williams
SBT in interactive mode has its own way of parsing command arguments - for example, 'run input > output' will call your program with 3 arguments - input, >, ouput. Which is unexpected, but reasonable - sbt is not your shell. If you want redirection, you have to run sbt from the shell like this
Stefan Langer
I'd suggest the percent sign as out if well understood by java and scala guys do to string formatting.
Mark Harrah
Artem Khodyush <gree...@gmail.com> wrote:
> On Fri, Aug 5, 2011 at 8:37 PM, Derek Williams <de...@fyrie.net> wrote:
>
> > 2011/8/5 Frédérick Dubois <fred....@gmail.com>
> >
> >> How about backticks?
> >>
> >>
> > Backticks are used in Linux (and most likely OSX) to use the output of a
> > command as an arg, so I don't think that would work either.
> >
>
> SBT in interactive mode has its own way of parsing command arguments - for
> example, 'run input > output' will call your program with 3 arguments -
> input, >, ouput. Which is unexpected, but reasonable - sbt is not your
> shell. If you want redirection, you have to run sbt from the shell like this
>
> sbt "run input" > output
>
> I think that using some non-traditional escape character will fit well into
> this picture, for example, with backtick the shell line becomes
>
> sbt "run input\` with\` spaces" > output
>
>
> which is not that bad. You can always refer to unix find as precedent - it
> also uses characters that have special meaning in shell, so everyone is
> forced to escape them (and swear at find authors, so maybe it's not really a
> good precedent :-)
You can also use single quotes to avoid the need to quote the backticks:
sbt 'run input` with` spaces' > output.
So far, this looks like the most reasonable option. If there are objections, please raise them now. How should the backticks be escaped? doubling?
-Mark
Paul Phillips
> You can also use single quotes to avoid the need to quote the backticks:
>
> sbt 'run input` with` spaces'> output.
>
> So far, this looks like the most reasonable option. If there are objections, please raise them now. How should the backticks be escaped? doubling?
I strongly object to the creation of brand new quoting mechanisms which
use characters with pre-established meanings in new ways. I find it
difficult to believe it is necessary, and if I have to spend one more
day of my life performing "quote tunneling" I'm going to lose it. Are
the known dimensions of this problem enumerated in one place somewhere?
Jason Zaugg
Doug Tangren
I'm going to chime in a second time here though I think the subject of this thread mislead me to thinking this was about both areas of user input.
The issue is not limited to external input, but also the way arguments are parsed for inputTasks. @file may be an option for the issue with external arguments but doesn't really help the same issue with inputTasks. I'd be one unhappy sailor of I had to store all combination's of arguments in an external file for reference :) I'd still recommend using targeted parser combinators instead of the default parser if you are arguments can be strings with spaces until a unified solution is available. Another coolish option would be to make argument parsing pluggable, which I think you can via settings, so that someone opt into using a custom argument parser for their os or style preference or be able to share theirs with others.
Thoughts?
-Doug Tangren
http://lessis.me
Paul Phillips
> Another coolish option would be to make argument parsing pluggable,
> which I think you can via settings, so that someone opt into using a
> custom argument parser for their os or style preference or be able to
> share theirs with others.
great, but it doesn't address the question of the defaults. As I find
less and less time to fiddle with things, the sanity of the default
settings takes an increasingly dominant role in whether software is
appealing or irritating. In this context that means the defaults should
be based on the platform conventions.
Paul Phillips
> if I have to spend one more day of my life performing "quote
> tunneling" I'm going to lose it.
Relevant: https://github.com/paulp/scalabash
Artem Khodyush
--
You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
To post to this group, send email to simple-build-tool@googlegroups.com.
To unsubscribe from this group, send email to simple-build-tool+unsubscribe@googlegroups.com.
Doug Tangren
Awesome. You should totally call that "sash".
Paul Phillips
> Awesome. You should totally call that "sash".
I think that's been taken by some shell. I thought scalabash sounded a
bit like a bacchanalia, something with which I prefer to associate.
eugene yokota
Is this discussion a settled matter?
Here's my suggestion:
1. double quote (") starts a double-quoted string.
1.1. within it, backslash (\) may start a Java escape sequence.
1.2. double quote (") terminates it.
2. single quote (') starts a single-quoted string.
2.1. within it, backslash (\) may start a Java escape sequence.
2.2. single quote (') terminates it.
3. at-double quote (@") starts an at-double-quoted string.
3.1 double quote (") terminates it.
* Java escape sequence:
http://java.sun.com/docs/books/jls/second_edition/html/lexical.doc.html#101089
As long as there's a way to suppress the escaping I don't see how \ is
such a big deal.
-eugene
Mark Harrah
eugene yokota <eed3...@gmail.com> wrote:
> Hi,
>
> Is this discussion a settled matter?
> Here's my suggestion:
>
> 1. double quote (") starts a double-quoted string.
> 1.1. within it, backslash (\) may start a Java escape sequence.
> 1.2. double quote (") terminates it.
> 2. single quote (') starts a single-quoted string.
> 2.1. within it, backslash (\) may start a Java escape sequence.
> 2.2. single quote (') terminates it.
> 3. at-double quote (@") starts an at-double-quoted string.
> 3.1 double quote (") terminates it.
>
> * Java escape sequence:
> http://java.sun.com/docs/books/jls/second_edition/html/lexical.doc.html#101089
>
> As long as there's a way to suppress the escaping I don't see how \ is
> such a big deal.
I've not seen @" anywhere else before. Is it from somewhere? Because backslashes are separators in paths on Windows, the unescaped quoting needs to be easily discoverable. Also, I'm not sure it is an improvement over not having any escaping by default because you can't mix Windows paths with escaping.
An interim solution to get something usable might be to allow a matched pair of one of " ' ` to quote a string and not worry about escaping yet.
-Mark
eugene yokota
> I've not seen @" anywhere else before. Is it from somewhere? Because backslashes are separators in paths on Windows, the unescaped quoting needs to be easily discoverable.
@"..." is the verbatim string literal from C#.
Because of work, I use Windows quite a bit, and I think Windows
programmers are use to escaping backslashes, substituting with forward
slash, or dig around for verbatim literals. For those who have done
.NET coding, @"..." would come very natural.
> An interim solution to get something usable might be to allow a matched pair of one of " ' ` to quote a string and not worry about escaping yet.
I guess that's fine, but saying quoting works like
Scala/Java/PHP/C++/C# etc I think is still along the line with
principle of least surprise.
Plus, realistically, if I were seriously using sbt and scala on
Windows, I'd run it on Cygwin anyway.
-eugene
Mark Harrah
eugene yokota <eed3...@gmail.com> wrote:
> On Wed, Sep 28, 2011 at 10:30 PM, Mark Harrah <dmha...@gmail.com> wrote:
> > I've not seen @" anywhere else before. Is it from somewhere? Because backslashes are separators in paths on Windows, the unescaped quoting needs to be easily discoverable.
>
> @"..." is the verbatim string literal from C#.
> Because of work, I use Windows quite a bit, and I think Windows
> programmers are use to escaping backslashes, substituting with forward
> slash, or dig around for verbatim literals. For those who have done
> .NET coding, @"..." would come very natural.
Thanks, I didn't know that. If we are going to have a verbatim string literal that is something other than the default "..." or '...', though, it should be Scala's """...""".
> > An interim solution to get something usable might be to allow a matched pair of one of " ' ` to quote a string and not worry about escaping yet.
>
> I guess that's fine, but saying quoting works like
> Scala/Java/PHP/C++/C# etc I think is still along the line with
> principle of least surprise.
> Plus, realistically, if I were seriously using sbt and scala on
> Windows, I'd run it on Cygwin anyway.
You might, but I doubt that is the case for everyone. The reason the escaping was backed out of the Process library was because of a bug report in the android plugin, for example.
-Mark
> -eugene
>
eugene yokota
>> @"..." is the verbatim string literal from C#.
I agree """..."" is the least-surprising route. But, I was thinking
about how `cmd` or `bash` expands `"` when you pass params from them.
(I know we are talking about InputTask or Commands, but it would be
nice to do that from outside too).
Here's a simple hello world that prints out only the first arg:
object Main {
def main(args: Array[String]) {
println("hello " + (args.toList match {
case Nil => "no args"
case x :: xs => x + "!"
}))
}
}
With bash:
#!/bin/bash
java -jar target/helloworld-assembly-0.1.jar "$@"
Now let's see the difference
$ ./test.sh """foo bar""" """baz"""
hello foo bar!
$ ./test.sh "\"\"\"foo bar\"\"\"" "\"\"\"baz\"\"\""
hello """foo bar"""!
$ ./test.sh @"foo bar" @"baz"
hello @foo bar!
$ ./test.sh "@\"foo bar\"" @\"baz\"
hello @"foo bar"!
With Windows,
@echo off
java -jar target/helloworld-assembly-0.1.jar %*
Here's the result
>test """foo bar""" """baz"""
hello "foo!
>test "\"\"\"foo bar\"\"\"" "\"\"\"baz\"\"\""
hello """foo bar"""!
>test @"foo bar" @"baz"
hello @foo bar!
>test "@\"foo bar\"" @\"baz\"
hello @"foo bar"!
Maybe allow both @"..." and """..."""?
-eugene
Doug Tangren
def delimitedStrings(isp: Parser[Char], notp: Parser[String])(display: String = "<arg>"): Parser[Seq[String]] =repsep(token(isp) ~> token(notp, display) <~ isp, SpaceClass.*)
-eugene
eugene yokota
> Am I seeing that the first example of triple quotes doesn't work on
> windows?
No, that wasn't what I tried to say. Sorry if
>test """foo bar""" """baz"""
hello "foo!
was misleading! It does look like I am in sbt shell, but I am in cmd
shell, as in:
F:\temp\>test.bat """foo bar""" """baz"""
hello "foo!
This is an acceptable behavior since cmd shell is expected to handle
`"`. My point is that to escape """...""" with cmd shell you have to
do:
F:\temp\>test.bat "\"\"\"foo bar\"\"\"" "\"\"\"baz\"\"\""
hello """foo bar"""!
As opposed to
F:\temp\>test "@\"foo bar\"" @\"baz\"
hello @"foo bar"!
, which is less typing. It's the same for bash
[temp]$ ./test.sh "\"\"\"foo bar\"\"\"" "\"\"\"baz\"\"\""
and
[temp]$ ./test.sh "@\"foo bar\"" @\"baz\"
> but I'm still not clear on the restriction windows has.
My theory is that there is no restriction, otherwise none of the
languages that uses backslash `\` as escaping would work, including
C/C++/C#/Scala.
So I decided to take a trip down to sbt's past.. The year 2010.
It was an unusually hot spring. Steve Thompson reports an unusual bug
to sbt list, titled ["Android Plugin Issue"][1].
>> under a Windows 7 box, and when I go to create a simple Android project with SBT, I get the following when I issue the 'sbt update package-debug' after running the generation script
Down the thread Mark replies
>> The Process API now uses the command line parser to split up arguments. This parser allows quoting arguments and backslashes to escape quotes. Obviously this is unix-style and doesn't work well with windows.
From the stack trace, the last line from android-plugin is this:
>> at AndroidProject$$anonfun$aaptGenerateTask$1.apply(AndroidProject.scala:97)
Here's the [line 97][2] from around that time:
def aaptGenerateTask = execTask {<x>
{aaptPath.absolutePath} package -m -M
{androidManifestPath.absolutePath} -S {mainResPath.absolutePath}
-I {androidJarPath.absolutePath} -J
{mainJavaSourcePath.absolutePath}
</x>} dependsOn directory(mainJavaSourcePath)
Notice there's no backslash here!
Let's see the rest of the stack trace from sbt 0.7.1:
java.lang.RuntimeException: Could not parse command: (1,3): Escape
sequences
can only occur in a quoted argument
at scala.Predef$.error(Predef.scala:76)
at sbt.Process$.apply(Process.scala:26)
at sbt.Process$.apply(Process.scala:15)
at sbt.Process$.apply(Process.scala:40)
I couldn't find the `Process.scala` from the time, but I did find
commented out code that looks to be [the culprit][3]:
/** create ProcessBuilder with working dir optionaly set to
File and extra environment variables */
def apply(command: String, cwd: Option[File], extraEnv:
(String,String)*): ProcessBuilder = {
apply(command.split("""\s+"""), cwd, extraEnv : _*)
// not smart to use this on windows, because CommandParser uses \ to escape ".
/*CommandParser.parse(command) match {
case Left(errorMsg) => error(errorMsg)
case Right((cmd, args)) => apply(cmd :: args, cwd, extraEnv : _*)
}*/
}
This was likely called by:
implicit def apply(command: scala.xml.Elem): ProcessBuilder =
apply(command.text.trim)
Maybe I am way off on this, but my theory is that the problem comes
from the fact that unescaping logic was applied to string values
stored in <x>...</x>. Since 0.7 used Scala code as the build
definition, the `\`-unescaping, if needed, has already performed.
And since `{androidManifestPath.absolutePath}` likely had Windows path
with \ in it like "C:\Program Files\Android SDK\bin\", that caused the
error.
This is a different situation as someone running `run` from sbt shell
as follows:
> run -m "foo\nbar\n" -d @"C:\temp\"
This is a string literal handling, which is unprocessed by scalac.
In other words, this is a shell-only issue.
[1]: http://groups.google.com/group/simple-build-tool/browse_thread/thread/29fe4de0309cc389
[2]: https://github.com/jberkel/android-plugin/blob/1dc54a32bc7a5519d6a5d730964193898712d259/src/main/scala/AndroidProject.scala#L97
[3]: https://github.com/harrah/xsbt/blob/3d736fcf66b05863061721ed59eac37d9d307588/sbt/src/main/scala/sbt/Process.scala#L23
-eugene
Doug Tangren
[1]: http://groups.google.com/group/simple-build-tool/browse_thread/thread/29fe4de0309cc389
[2]: https://github.com/jberkel/android-plugin/blob/1dc54a32bc7a5519d6a5d730964193898712d259/src/main/scala/AndroidProject.scala#L97
[3]: https://github.com/harrah/xsbt/blob/3d736fcf66b05863061721ed59eac37d9d307588/sbt/src/main/scala/sbt/Process.scala#L23
-eugene