Quote problem
jhogan
anywhere.
Can someone help me understand what I am seeing.
Lets start a session on cygwin...
set -x
# set $t to some sql preceded by a -Q flag. This will be passed to
another command obviously.
$ t=" -Q \"SELECT curriculum_last_update FROM application_settings\" "
+ t=' -Q "SELECT curriculum_last_update FROM application_settings" '
# So far so good
$ echo $t
+ echo -Q '"SELECT' curriculum_last_update FROM
'application_settings"'
-Q "SELECT curriculum_last_update FROM application_settings"
Notice the single-quote at the end of SELECT and at the beginning of
application_settings. Why does that happen?
# Make a call to sqlcmd. sqlcmd is a command line interface to sql
server. $t now contains the SQL
# that I would like to run, preceded by -Q which means "query"
$ sqlcmd -b -x -d mydb -S myserver $t
+ sqlcmd -b -x -d mydb -S myserver -Q '"SELECT'
curriculum_last_update FROM 'application_settings"'
Sqlcmd: 'SELECT" curriculum_last_update FROM "application_settings
\""': Unexpected argument. Enter '-?' for help.
sqlcmd seems to be getting a sql string that has a couple of
unnecessary quotes (not even sure if they are double or single
according to the output".
Oh well, I would love to know why this is happening. Thanks for your
help in advance.
Bill Marcum
> This has probably been asked before but I can't seem to find this
> anywhere.
>
> + echo -Q '"SELECT' curriculum_last_update FROM
> 'application_settings"'
> -Q "SELECT curriculum_last_update FROM application_settings"
>
> Notice the single-quote at the end of SELECT and at the beginning of
> application_settings. Why does that happen?
>
probably want:
eval echo $t
> # Make a call to sqlcmd. sqlcmd is a command line interface to sql
> server. $t now contains the SQL
> # that I would like to run, preceded by -Q which means "query"
> $ sqlcmd -b -x -d mydb -S myserver $t
$ eval sqlcmd -b -x -d mydb -S myserver $t
jhogan
> On 2009-11-24, jhogan <jessehogan1...@gmail.com> wrote:> This has probably been asked before but I can't seem to find this
Tremendous, that got my script working. Thanks for your help
Ben Finney
> This has probably been asked before but I can't seem to find this
> anywhere.
Short answer: In expanding the command line (the processing that occurs
before executing the command), parameter and variable expansion happens
before word splitting, which happens before quote removal.
<URL:http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06>
<URL:http://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions>
> set -x
>
> # set $t to some sql preceded by a -Q flag. This will be passed to
> another command obviously.
>
> $ t=" -Q \"SELECT curriculum_last_update FROM application_settings\" "
> + t=' -Q "SELECT curriculum_last_update FROM application_settings" '
>
> # So far so good
Right. The command above has one word, consisting of ‘t="[all the rest
in quotes]"’. At the end of the command-line expansion, the quoting
characters (‘\’, ‘"’, ‘'’) are removed. So, when the command executes,
the variable ‘t’ gets a value containing spaces, as you specified.
The trace output re-applies some quoting to show you the single word
which is the value assigned to ‘t’.
> $ echo $t
> + echo -Q '"SELECT' curriculum_last_update FROM
> 'application_settings"'
> -Q "SELECT curriculum_last_update FROM application_settings"
During expansion of this command line, variable substitution replaces
‘$t’ with the current value of the ‘t’ variable. That value contains
spaces, which are not quoted. When word splitting occurs, the resulting
command-line contains spaces, so they separate words in this command.
Then, the command runs, and as the trace output shows you, the command
has six words <URL:http://www.youtube.com/watch?v=JWi5jdgTUJs>:
'echo'
'-Q'
'"SELECT'
'curriculum_last_update'
'FROM'
'application_settings"'
since those are the words that were separated by spaces.
> Notice the single-quote at the end of SELECT and at the beginning of
> application_settings. Why does that happen?
Because the trace output of the shell is representing separate words.
Some of those words contain special characters (in this case, ‘"’). The
trace output knows only the arguments that resulted, and has no idea how
the words were actually typed into the command line, so it represents
one possible way the same argument could be generated: by surrounding
the word with single-quote characters.
> # Make a call to sqlcmd. sqlcmd is a command line interface to sql
> server. $t now contains the SQL
> # that I would like to run, preceded by -Q which means "query"
> $ sqlcmd -b -x -d mydb -S myserver $t
The same thing happens here: during expansion, the current value of ‘t’
is substituted in place of ‘$t’, and the result is more spaces which
separate words on the command line.
The solution is simple: if you have a value containing special
characters, consider for *every command line* whether its special
characters need to be quoted:
$ set -x
$ t=" -Q \"SELECT curriculum_last_update FROM application_settings\" "
+ t=' -Q "SELECT curriculum_last_update FROM application_settings" '
$ echo $t
+ echo -Q '"SELECT' curriculum_last_update FROM 'application_settings"'
-Q "SELECT curriculum_last_update FROM application_settings"
$ echo "$t"
+ echo ' -Q "SELECT curriculum_last_update FROM application_settings" '
-Q "SELECT curriculum_last_update FROM application_settings"
$ sqlcmd -b -x -d mydb -S myserver "$t"
+ sqlcmd -b -x -d mydb -S myserver ' -Q "SELECT curriculum_last_update FROM application_settings" '
--
\ “The greater the artist, the greater the doubt; perfect |
`\ confidence is granted to the less talented as a consolation |
_o__) prize.” —Robert Hughes |
Ben Finney
jhogan
David Combs
Interesting -- assuming I understood itg.
Could you elaborate a bit, on why you need the eval, what it
does for you?
THANKS!
---- SECONDARY question:
PS: Whenever I see a use of eval on groups like this, everyone
starts screaming about how DANGEROUS that is.
So, in this case, what is it that might bite you?
Thanks,
David
Kaz Kylheku
> In article <behtt6-...@marcumbill.bellsouth.net>,
> Bill Marcum <marcu...@bellsouth.net> wrote:
>>On 2009-11-24, jhogan <jesseho...@gmail.com> wrote:
>>> This has probably been asked before but I can't seem to find this
>>> anywhere.
>>>
>>> $ echo $t
>>> + echo -Q '"SELECT' curriculum_last_update FROM
>>> 'application_settings"'
>>> -Q "SELECT curriculum_last_update FROM application_settings"
>>>
>>> Notice the single-quote at the end of SELECT and at the beginning of
>>> application_settings. Why does that happen?
>>>
>>The echo command is treating each word as a separate argument. You
>>probably want:
>>eval echo $t
>>
>>> # Make a call to sqlcmd. sqlcmd is a command line interface to sql
>>> server. $t now contains the SQL
>>> # that I would like to run, preceded by -Q which means "query"
>>> $ sqlcmd -b -x -d mydb -S myserver $t
>> $ eval sqlcmd -b -x -d mydb -S myserver $t
>
> Interesting -- assuming I understood itg.
>
> Could you elaborate a bit, on why you need the eval, what it
> does for you?
Why do we need eval? Because we have a variable t which conatins
shell syntax!
t="-Q \"SELECT ....\""
This t is a fragment of a shell command line which needs to be subject
to word splitting, etc, just as if it were typed into the shell.
Eval is the shell's interface to its function that processes shell
syntax, right from the character level. eval is needed here to reduce a
meta-syntactic variable that contains code, into the corresponding code.
It's more correct to protect this variable with quotes when passing it
to eval. The best is to protect the entire syntax:
eval "sqlcmd -b -x -d mydb -S myserver $t"
I.e. the thing between the quotes is the piece of shell syntax we want
to pass to eval; our $t piece is substituted into this one big string.
If we just to this, it is wrong:
eval sqlcmd -b -x -d mydb -S myserver $t
At the very least we want to protect the variable. Now
eval gets "sqlcmd", "-b", ... "myserver" as separate arguments,
which are followed by the contents of t as one big argument.
eval sqlcmd -b -x -d mydb -S myserver "$t"
The contents of our variable t must not be subject to splitting prior to
being passed to eval. You see t could for instance contain significant
spaces:
t="arg1 arg2 \"arg with significant double spaces\""
These significant spaces will be lost if we do an unquoted eval.
# with above t
$ eval echo $t # oops, bug, where did our spaces go?
arg1 arg2 arg with significant double spaces
$ eval echo "$t" # aha!
arg1 arg2 arg with significant double spaces
As you can see, quoting preserves the syntax better through eval, so
that the ultimate command (in this case echo) gets the correct data.
> PS: Whenever I see a use of eval on groups like this, everyone
> starts screaming about how DANGEROUS that is.
>
> So, in this case, what is it that might bite you?
Well, see above: quoting can bite you. Unwanted evaluations can bite
you. Escapes can bite you.
Suppose your string contains a dollar sign followed by a word.
Under eval, that becomes a variable evaluation! In addition to taking
care of quoting, you must escape all characters that are special syntax
to the shell, unless you intend them to be treated that way.
Backslashes are super awkward, because they have a maening in the
syntax which are using to specify the syntax, as well as in the
syntax that will be valuated, and their interpretetation depends
on what kind of quotes you are in.
For instance if you write "\\" to the shell, that's actually a word
consiting of a single backslash. But '\\' codes for two backslashes.
If you're using single quotes to prepare synax for eval, and you want to
encode a single backslash, you sometimes have to type four backslashes!
# set up the shell syntax
$ shell_syntax="this is a backslash: \"\\\\\""
# now /show/ me my shell syntax
$ echo $shell_syntax
this is a backslash: "\\"
# now /interpret/ my shell syntax, as arguments to echo:
$ eval echo "$shell_syntax"
this is a backslash: \
The problem with multiple evaluation levels in the same code
is the close similarity between an evaluation level and its
metalevel. Meta-syntax is like shell syntax, but not exactly,
due to the extra escaping, which, to make things more
complicated, is selectively applied: some things you want
evaluated in this round, some in the next round, some in
the third round ...
David Combs
>>
>> Interesting -- assuming I understood itg.
>>
>> Could you elaborate a bit, on why you need the eval, what it
>> does for you?
>
>Why do we need eval? Because we have a variable t which conatins
>shell syntax!
>
> t="-Q \"SELECT ....\""
>
>This t is a fragment of a shell command line which needs to be subject
>to word splitting, etc, just as if it were typed into the shell.
>
>Eval is the shell's interface to its function that processes shell
>syntax, right from the character level. eval is needed here to reduce a
>meta-syntactic variable that contains code, into the corresponding code.
>
>It's more correct to protect this variable with quotes when passing it
...
...
THANKS!
Now I go print it out, put on my thinking cap (with lots of
caffein), and study it.
May take a day or two before I get back to the group --
especially with 6" forecasted for tonight and 12" for
tomorrow! (snow, shovel, snow-plows shoving it right
back into the driveway!)
David