Patch 7.3.443
Bram Moolenaar
Patch 7.3.443
Problem: MS-Windows: 'shcf' and 'shellxquote' defaults are not very good.
Solution: Make a better guess when 'shell' is set to "cmd.exe". (Ben Fritz)
Files: src/option.c, runtime/doc/options.txt
*** ../vim-7.3.442/src/option.c 2012-01-28 18:03:30.000000000 +0100
--- src/option.c 2012-02-12 23:17:55.000000000 +0100
***************
*** 3883,3889 ****
#if defined(MSDOS) || defined(WIN3264) || defined(OS2)
/*
! * Set 'shellcmdflag and 'shellquote' depending on the 'shell' option.
* This is done after other initializations, where 'shell' might have been
* set, but only if they have not been set before. Default for p_shcf is
* "/c", for p_shq is "". For "sh" like shells it is changed here to
--- 3883,3890 ----
#if defined(MSDOS) || defined(WIN3264) || defined(OS2)
/*
! * Set 'shellcmdflag', 'shellxquote', and 'shellquote' depending on the
! * 'shell' option.
* This is done after other initializations, where 'shell' might have been
* set, but only if they have not been set before. Default for p_shcf is
* "/c", for p_shq is "". For "sh" like shells it is changed here to
***************
*** 3920,3925 ****
--- 3921,3962 ----
# endif
# endif
}
+ else if (strstr((char *)gettail(p_sh), "cmd.exe") != NULL)
+ {
+ int idx3;
+
+ /*
+ * cmd.exe on Windows will strip the first and last double quote given
+ * on the command line, e.g. most of the time things like:
+ * cmd /c "my path/to/echo" "my args to echo"
+ * become:
+ * my path/to/echo" "my args to echo
+ * when executed.
+ *
+ * To avoid this, use the /s argument in addition to /c to force the
+ * stripping behavior, and also set shellxquote to automatically
+ * surround the entire command in quotes (which get stripped as
+ * noted).
+ */
+
+ /* Set shellxquote default to add the quotes to be stripped. */
+ idx3 = findoption((char_u *)"sxq");
+ if (idx3 >= 0 && !(options[idx3].flags & P_WAS_SET))
+ {
+ p_sxq = (char_u *)"\"";
+ options[idx3].def_val[VI_DEFAULT] = p_sxq;
+ }
+
+ /* Set shellcmdflag default to always strip the quotes, note the order
+ * between /s and /c is important or cmd.exe will treat the /s as part
+ * of the command to be executed. */
+ idx3 = findoption((char_u *)"shcf");
+ if (idx3 >= 0 && !(options[idx3].flags & P_WAS_SET))
+ {
+ p_shcf = (char_u *)"/s /c";
+ options[idx3].def_val[VI_DEFAULT] = p_shcf;
+ }
+ }
#endif
#ifdef FEAT_TITLE
*** ../vim-7.3.442/runtime/doc/options.txt 2011-06-26 05:36:07.000000000 +0200
--- runtime/doc/options.txt 2012-02-12 23:21:59.000000000 +0100
***************
*** 5880,5895 ****
security reasons.
*'shellcmdflag'* *'shcf'*
! 'shellcmdflag' 'shcf' string (default: "-c", MS-DOS and Win32, when 'shell'
! does not contain "sh" somewhere: "/c")
global
{not in Vi}
Flag passed to the shell to execute "!" and ":!" commands; e.g.,
"bash.exe -c ls" or "command.com /c dir". For the MS-DOS-like
systems, the default is set according to the value of 'shell', to
reduce the need to set this option by the user. It's not used for
! OS/2 (EMX figures this out itself). See |option-backslash| about
! including spaces and backslashes. See |dos-shell|.
This option cannot be set from a |modeline| or in the |sandbox|, for
security reasons.
--- 5899,5919 ----
security reasons.
*'shellcmdflag'* *'shcf'*
! 'shellcmdflag' 'shcf' string (default: "-c";
! Win32, when 'shell' is cmd.exe: "/s /c";
! MS-DOS and Win32, when 'shell' neither is
! cmd.exe nor contains "sh" somewhere: "/c")
global
{not in Vi}
Flag passed to the shell to execute "!" and ":!" commands; e.g.,
"bash.exe -c ls" or "command.com /c dir". For the MS-DOS-like
systems, the default is set according to the value of 'shell', to
reduce the need to set this option by the user. It's not used for
! OS/2 (EMX figures this out itself).
! On Unix it can have more than one flag. Each white space separated
! part is passed as an argument to the shell command.
! See |option-backslash| about including spaces and backslashes.
! Also see |dos-shell| for MS-DOS and MS-Windows.
This option cannot be set from a |modeline| or in the |sandbox|, for
security reasons.
***************
*** 5910,5918 ****
For Unix the default it "| tee". The stdout of the compiler is saved
in a file and echoed to the screen. If the 'shell' option is "csh" or
"tcsh" after initializations, the default becomes "|& tee". If the
! 'shell' option is "sh", "ksh", "zsh" or "bash" the default becomes
! "2>&1| tee". This means that stderr is also included. Before using
! the 'shell' option a path is removed, thus "/bin/sh" uses "sh".
The initialization of this option is done after reading the ".vimrc"
and the other initializations, so that when the 'shell' option is set
there, the 'shellpipe' option changes automatically, unless it was
--- 5934,5943 ----
For Unix the default it "| tee". The stdout of the compiler is saved
in a file and echoed to the screen. If the 'shell' option is "csh" or
"tcsh" after initializations, the default becomes "|& tee". If the
! 'shell' option is "sh", "ksh", "mksh", "pdksh", "zsh" or "bash" the
! default becomes "2>&1| tee". This means that stderr is also included.
! Before using the 'shell' option a path is removed, thus "/bin/sh" uses
! "sh".
The initialization of this option is done after reading the ".vimrc"
and the other initializations, so that when the 'shell' option is set
there, the 'shellpipe' option changes automatically, unless it was
***************
*** 6017,6024 ****
*'shellxquote'* *'sxq'*
'shellxquote' 'sxq' string (default: "";
! for Win32, when 'shell' contains "sh"
! somewhere: "\""
for Unix, when using system(): "\"")
global
{not in Vi}
--- 6043,6050 ----
*'shellxquote'* *'sxq'*
'shellxquote' 'sxq' string (default: "";
! for Win32, when 'shell' is cmd.exe or
! contains "sh" somewhere: "\""
for Unix, when using system(): "\"")
global
{not in Vi}
***************
*** 6026,6036 ****
the "!" and ":!" commands. Includes the redirection. See
'shellquote' to exclude the redirection. It's probably not useful
to set both options.
! This is an empty string by default. Known to be useful for
! third-party shells when using the Win32 version, such as the MKS Korn
! Shell or bash, where it should be "\"". The default is adjusted
! according the value of 'shell', to reduce the need to set this option
! by the user. See |dos-shell|.
This option cannot be set from a |modeline| or in the |sandbox|, for
security reasons.
--- 6052,6063 ----
the "!" and ":!" commands. Includes the redirection. See
'shellquote' to exclude the redirection. It's probably not useful
to set both options.
! This is an empty string by default on most systems, but is known to be
! useful for on Win32 version, either for cmd.exe which automatically
! strips off the first and last quote on a command, or 3rd-party shells
! such as the MKS Korn Shell or bash, where it should be "\"". The
! default is adjusted according the value of 'shell', to reduce the need
! to set this option by the user. See |dos-shell|.
This option cannot be set from a |modeline| or in the |sandbox|, for
security reasons.
*** ../vim-7.3.442/src/version.c 2012-02-12 20:13:55.000000000 +0100
--- src/version.c 2012-02-12 23:18:40.000000000 +0100
***************
*** 716,717 ****
--- 716,719 ----
{ /* Add new patch number below this line */
+ /**/
+ 443,
/**/
--
CVS sux, men don't like commitment
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
mattn
mattn
Bram Moolenaar
Yasuhiro Matsumoto wrote:
This is the opposite of what the patch was fixing.
What system is this on?
What were the option values before and after the patch?
Does it work OK without the /s argument?
--
hundred-and-one symptoms of being an internet addict:
51. You put a pillow case over your laptop so your lover doesn't see it while
you are pretending to catch your breath.
mattn
> This is the opposite of what the patch was fixing.
> What system is this on?
Windows XP. using cmd.exe
> What were the option values before and after the patch?
I don't set anything. using defaults.
> Does it work OK without the /s argument?
No.
RoDo
get "File Not Found"
So this patch solve the problem only if you use pipe.
Andy Wokula
cmd.exe sux. Why isn't there the opposite of "/s" to keep quotation marks?
Could somebody please make a patch for cmd.exe?
er, forgot, close source :-(
--
Andy
Ben Fritz
On Feb 12, 7:27 pm, mattn <mattn...@gmail.com> wrote:
> Sorry for delay for my checking this patch.
>
> This break many behaviors to call external program on windows.
> For example:
>
> Since 7.3.433:
>
> let command = 'openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/foo.tmp'
> let ret = system(command)
>
> This was expanded to:
>
> cmd /c openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/foo.tmp
>
> But After 7.3.433:
>
> let command = 'openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/foo.tmp'
> let ret = system(command)
>
> This will be expanded to:
>
> cmd /s /c "openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/foo.tmp"
>
to strip off the first and last " character before expanding, so the
command which actually gets executed is still:
openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/foo.tmp
"C:\Program Files\abc.exe" "some arg with spaces"
which the default cmd.exe /c will treat as:
C:\Program Files\abc.exe" "some arg with spaces
which is obviously wrong.
The default values in the patch first turn it into:
""C:\Program Files\abc.exe" "some arg with spaces""
which cmd.exe /s /c treats as:
"C:\Program Files\abc.exe" "some arg with spaces"
as intended.
See cmd.exe /?
If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic
is
used to process quote (") characters:
1. If all of the following conditions are met, then quote
characters
on the command line are preserved:
- no /S switch
- exactly two quote characters
- no special characters between the two quote characters,
where special is one of: &<>()@^|
- there are one or more whitespace characters between the
the two quote characters
- the string between the two quote characters is the name
of an executable file.
2. Otherwise, old behavior is to see if the first character is
a quote character and if so, strip the leading character and
remove the last quote character on the command line,
preserving
any text after the last quote character.
Ben Fritz
patch, i.e.:
gvim -N -u NONE -i NONE
:set shellcmdflag=/s\ /c
:set shellxquote=\"
and executing:
:echo system('dir "&"')
weird).
However, setting 'noshelltemp' as suggested by RoDo shows the
directory contents of '&' as expected.
Ben Fritz
On Feb 13, 9:41 am, Ben Fritz <fritzophre...@gmail.com> wrote:
>
> See cmd.exe /?
>
> If /C or /K is specified, then the remainder of the command line after
> the switch is processed as a command line, where the following logic
> is
> used to process quote (") characters:
>
> 1. If all of the following conditions are met, then quote
> characters
> on the command line are preserved:
>
> - no /S switch
> - exactly two quote characters
> - no special characters between the two quote characters,
> where special is one of: &<>()@^|
> - there are one or more whitespace characters between the
> the two quote characters
> - the string between the two quote characters is the name
> of an executable file.
> 2. Otherwise, old behavior is to see if the first character is
> a quote character and if so, strip the leading character and
> remove the last quote character on the command line,
> preserving
>
It appears that cmd.exe is NOT behaving as documented.
Take a look at this:
gvim -N -u NONE -i NONE
:set shellcmdflag=/s\ /c
:!echo "A&B"
The vimrun console pops up:
C:\WINDOWS\system32\cmd.exe /s /c "echo "A&B""
A
'B""' is not recognized as an internal or external command,
operable program or batch file.
shell returned 1
Hit any key to close this window...
According to the cmd.exe help, a command with the /s flag given
explicitly does NOT fall in the first category. It falls into the
second, "otherwise..." category. However, it is being treated as the
first category, first executing "echo "A and then executing B"".
Experimenting on the cmd.exe window confirms:
U:\>cmd /c echo A & pause
A
Press any key to continue . . .
U:\>cmd /c "echo A & pause"
A
Press any key to continue . . .
U:\>cmd /c echo "A & pause"
"A & pause"
U:\>cmd /s /c echo "A & pause"
"A & pause"
U:\>cmd /s /c "echo "A & pause""
A
'pause""' is not recognized as an internal or external command,
operable program or batch file.
Fantastic. I wonder whether the "special characters" text in the
cmd.exe help is to blame.
Note, this works:
:!echo "A^&B"
U:\>cmd /s /c "echo "A ^& pause""
"A & pause"
(in cmd.exe, the escape character is ^ for some crazy reason or
another)
Maybe Vim can automatically add this ^ character somehow? This seems
like a job for shellescape().
As far as the patch is concerned, I still think it the right thing to
do. But we must ask, which is more likely to occur in a call to an
external program? The '&' character in an argument, or space
characters in the command path and argument?
Ben Fritz
On Feb 13, 10:01 am, Ben Fritz <fritzophre...@gmail.com> wrote:
>
> Maybe Vim can automatically add this ^ character somehow? This seems
> like a job for shellescape().
>
> As far as the patch is concerned, I still think it the right thing to
> do. But we must ask, which is more likely to occur in a call to an
> external program? The '&' character in an argument, or space
>
Note in the original example:
let command = 'openssl dgst -binary -sha1 -hmac "A&B" < c:/temp/
foo.tmp'
let ret = system(command)
let command = '"C:\Program Files\OSSL\openssl" dgst -binary -sha1 -
hmac "A & B"'
let ret = system(command)
Then there is currently NO setting of shellxquote and shellcmdflag
that should work.
Does anybody see a problem with adding quoting of & with ^ in
shellescape? Some experimentation concerns me that things will break
if the user explicitly sets shellxquote back to empty, but if we
always use shellxquote=\" it will probably be OK:
U:\>echo "A^&B"
"A^&B"
U:\>cmd /c echo "A^&B"
"A^&B"
U:\>cmd /s /c "echo "A^&B""
"A&B"
U:\>cmd /c "echo "A^&B""
"A&B"
RoDo
But pipe work without ^.
It's weird.
Also set shellxquote=\" breaks gvim -d.
The diff mode command should be changed too.
Ben Fritz
On Feb 13, 10:23 am, RoDo <royad...@gmail.com> wrote:
> Never know ^ before.
> But pipe work without ^.
> It's weird.
> Also set shellxquote=\" breaks gvim -d.
> The diff mode command should be changed too.
>
long time now, and use vimdiff all the time. What's your diffexpr set
to, and from where? For a while there was one in vimrc_example.vim
which actually surrounded the entire command in quotes just like
shellcmdflag=\" does, which obviously caused problems when both were
done, but it seems to have been removed.
RoDo
I total forget about this.
I have a copy from vimrc_example which diffexpr set to MyDiff
function.
Remove it solve the problem. Thank you.
It's time to clean my _vimrc.
It's big and totally mess up.
Andy Wokula
>
> Patch 7.3.443
> Problem: MS-Windows: 'shcf' and 'shellxquote' defaults are not very good.
> Solution: Make a better guess when 'shell' is set to "cmd.exe". (Ben Fritz)
> Files: src/option.c, runtime/doc/options.txt
cmd.exe's quotes handling is broken, Vim cannot fix it.
(just my opinion)
But a cmdline can be enclosed in parens (...) which would prevent removal of quotes.
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/ntcmds_shelloverview.mspx
I just found it, not sure how well that works.
C:\>("C:\Program Files\abc.exe" "some arg with spaces")
If it works ok, a note in the help would be sufficient ...
--
Andy
Andy Wokula
> C:\>("C:\Program Files\abc.exe" "some arg with spaces")
er, I meant
C:\> cmd /c ("C:\Program Files\abc.exe" "some arg with spaces")
--
Andy
Ben Fritz
On Feb 13, 12:14 pm, Andy Wokula <anw...@yahoo.de> wrote:
> Am 12.02.2012 23:23, schrieb Bram Moolenaar:
>
>
>
> > Patch 7.3.443
> > Problem: MS-Windows: 'shcf' and 'shellxquote' defaults are not very good.
> > Solution: Make a better guess when 'shell' is set to "cmd.exe". (Ben Fritz)
> > Files: src/option.c, runtime/doc/options.txt
>
> cmd.exe's quotes handling is broken, Vim cannot fix it.
> (just my opinion)
>
enough to prevent this sort of thing, but that's apparently irrelevant
because cmd.exe doesn't follow it.
> But a cmdline can be enclosed in parens (...) which would prevent removal of quotes.http://www.microsoft.com/resources/documentation/windows/xp/all/prodd...
> I just found it, not sure how well that works.
>
> C:\>("C:\Program Files\abc.exe" "some arg with spaces")
>
> If it works ok, a note in the help would be sufficient ...
>
the parentheses are used "to group or nest multiple commands", so it's
not really the intended purpose, but perhaps it will work.
Also from the page you reference:
"The ampersand (&), pipe (|), and parentheses ( ) are special
characters that must be preceded by the escape character (^) or
quotation marks when you pass them as arguments."
Plugin authors should be correct in passing "A&B" to an external
command, without quoting. cmd.exe is doing the incorrect thing here, I
think. I think it best to try to get the command to work in the most
common cases.
So we have:
shellxquote=\" and shellcmdflag=/s\ /c (new default) breaks passing
arguments with & (and possibly |) unless the user knows to escape them
with ^.
shellxquote= and shellcmdflag=/c (previous default) breaks using
programs with a space in the path and a space in the argument, unless
the user knows to surround with parentheses (or quotes like
shellxquote does).
I'm not sure which use is more common. Either way, I agree adding a
help note may allow users and plugin authors to work around the issue.
Obviously the new value breaks backwards compatibility with plugins
which currently pass '&' inside arguments to external programs, even
if they have correctly quoted it. But, I've seen a few plugins in the
past (including, if I recall correctly, Taglist, and also
http://www.vim.org/scripts/script.php?script_id=2378) which fell
victim to the problem fixed by this patch.
It does look like using parentheses should allow the & character to be
used properly inside quoted arguments without escaping, also if the
paths or argument contains spaces.
I wonder how the shell redirection might play into using parentheses.
shellxquote includes the stuff added for redirection, whereas user-
added parentheses will not.
Bram Moolenaar
Ben Fritz wrote:
> On Feb 13, 12:14�pm, Andy Wokula <anw...@yahoo.de> wrote:
> > Am 12.02.2012 23:23, schrieb Bram Moolenaar:
> >
> >
> >
> > > Patch 7.3.443
> > > Problem: � �MS-Windows: 'shcf' and 'shellxquote' defaults are not very good.
> > > Solution: � Make a better guess when 'shell' is set to "cmd.exe". (Ben Fritz)
> > > Files: � � �src/option.c, runtime/doc/options.txt
> >
> > cmd.exe's quotes handling is broken, Vim cannot fix it.
> > (just my opinion)
> >
>
> Ugh, apparently so. I thought the documentation of cmd.exe was clear
> enough to prevent this sort of thing, but that's apparently irrelevant
> because cmd.exe doesn't follow it.
Never trust Microsoft documentation, always verify it actually works
that way. And then try it on different systems, they usually only think
of the latest. I would like to support back to Windows XP.
If we set shellxquote to '(', we could use the closing ')'
automatically. Does that solve most problems? Please try it out with
the examples previously given.
I do hope we can find a solution that doesn't break any existing
scripts.
A command may intentionally use & and | to separate commands. Always
escaping them will break this intentional use.
Unless we can find a solution soon, I think I'll revert that patch.
--
hundred-and-one symptoms of being an internet addict:
55. You ask your doctor to implant a gig in your brain.
mattn
RoDo
small patch:
--- misc2_org.c 2012-02-14 18:09:38 +0800
+++ misc2.c 2012-02-14 17:51:54 +0800
@@ -3230,7 +3230,13 @@
{
STRCPY(ncmd, p_sxq);
STRCAT(ncmd, cmd);
- STRCAT(ncmd, p_sxq);
+ if (STRCMP(p_sxq, "(") == 0)
+ {
+ STRCAT(ncmd, ")");
+ } else
+ {
+ STRCAT(ncmd, p_sxq);
+ }
retval = mch_call_shell(ncmd, opt);
vim_free(ncmd);
}
shellcmdflag = /c
shellxquote = (
echo system('dir "&") pass
echo system('echo "a&b") pass
system('"exe have space"') pass
system('"exe have sapce" "some arg with spaces"') pass
system('"exe have spce" "some arg with &"') pass
RoDo
should escape &
echo system('echo a ^& echo b') pass
Andy Wokula
> echo system('echo a & echo b') fail
> should escape &
> echo system('echo a ^& echo b') pass
omg:
C:\> echo a & echo b
a
b
C:\> cmd /c ( echo a & echo b )
b )
Looks like a bug with cmd.exe .
I guess it's specific to echo .
And it's quite an esoteric command line:
somecmd & echo b
--
Andy
Mike Williams
C:\>cmd /c ( echo a ) & ( echo b )
a
b
IIRC echo has "special" rules as to how it consumes the line.
Mike
--
5 out of 4 people have trouble with fractions.
mattn
Andy Wokula
> cmd /c "(echo a& echo b)"
Ok!
--
Andy
RoDo
add a os_win32 patch
--- os_win32_org.c 2012-02-14 21:18:54 +0800
+++ os_win32.c 2012-02-14 21:24:27 +0800
@@ -3909,6 +3909,7 @@
if (newcmd != NULL)
{
char_u *cmdbase = (*cmd == '"' ? cmd + 1 : cmd);
+ cmdbase = (*cmd == '(' ? cmd + 1 : cmd);
if ((STRNICMP(cmdbase, "start", 5) == 0) &&
vim_iswhite(cmdbase[5]))
{
Previous patch broke !start.
Now it works.
Ben Fritz
On Feb 14, 4:31 am, RoDo <royad...@gmail.com> wrote:
> echo system('echo a & echo b') fail
> should escape &
> echo system('echo a ^& echo b') pass
>
shellxquote, I like that option.
Since there are still some failures when using an unescaped & maybe we
should put a note in the help about how to escape it as well, probably
in :help :! and :help system() both.
Ben Fritz
On Feb 13, 10:04 pm, Bram Moolenaar <B...@Moolenaar.net> wrote:
> > Ugh, apparently so. I thought the documentation of cmd.exe was clear
> > enough to prevent this sort of thing, but that's apparently irrelevant
> > because cmd.exe doesn't follow it.
>
> Never trust Microsoft documentation, always verify it actually works
> that way. And then try it on different systems, they usually only think
> of the latest. I would like to support back to Windows XP.
>
Windows XP 64-bit at work, and Windows Vista. I even have used & to
intentionally separate commands. I never used it in an argument,
though, I guess I never thought to take apart the MS documentation and
test each little piece to make sure it actually behaves as stated.
> > It does look like using parentheses should allow the & character to be
> > used properly inside quoted arguments without escaping, also if the
> > paths or argument contains spaces.
>
> > I wonder how the shell redirection might play into using parentheses.
> > shellxquote includes the stuff added for redirection, whereas user-
> > added parentheses will not.
>
> If we set shellxquote to '(', we could use the closing ')'
> automatically. Does that solve most problems? Please try it out with
> the examples previously given.
>
> I do hope we can find a solution that doesn't break any existing
> scripts.
>
most common commands, so I think some change or another is needed to
the defaults. I like the shellxquote=( idea.
> A command may intentionally use & and | to separate commands. Always
> escaping them will break this intentional use.
>
config) with the shellxquote=" from the patch. I wasn't suggesting
always escaping them. I thought maybe escaping them in shellescape()
would be a good idea, since shellescape() is for command arguments;
however (according to the documentation anyway) surrounding it with
quotes is supposed to work, so escaping again is redundant (and
doesn't work correctly under normal circumstances). Using the
paretheses to surround the entire command appears to allow the
surrounding quotes to work just fine. I think the problem might be
that the shell calling cmd /c is parsing the quotes before the shell
created by cmd /c, e.g. the command executed for:
cmd /c "echo "A&B""
is not:
echo "A&B"
in a new cmd.exe as intended but rather:
cmd /c "echo "A
followed by:
B""
in the same cmd.exe.
Using paretheses instead of quotes to surround the whole thing should
avoid the calling shell trying to be too smart.
RoDo
Previous patch break shellxquote = \"
update:
--- os_win32_org.c 2012-02-14 21:18:54 +0800
if (newcmd != NULL)
{
char_u *cmdbase = (*cmd == '"' ? cmd + 1 : cmd);
if ((STRNICMP(cmdbase, "start", 5) == 0) &&
vim_iswhite(cmdbase[5]))
{
STRCPY(newcmd, cmdbase);
if (*cmd == '"' && *newcmd != NUL)
newcmd[STRLEN(newcmd) - 1] = NUL;
+ if (*cmd == '(' && *newcmd != NUL)
+ newcmd[STRLEN(newcmd) - 1] = NUL;
}
/*
Also remove ) while using !start command.
Bram Moolenaar
Andy Wokula wrote:
> Am 14.02.2012 13:20, schrieb mattn:
> > cmd /c "(echo a& echo b)"
>
> Ok!
Ehm, thus 'shellxquote' should be "(, and we should turn that into )"
for the tail? It's possible, but I wonder where it breaks now.
--
hundred-and-one symptoms of being an internet addict:
57. You begin to wonder how on earth your service provider is allowed to call
200 hours per month "unlimited."
Bram Moolenaar
Rodo wrote:
Good, but it can be done in one line:
STRCAT(ncmd, STRCMP(p_sxq, "(") == 0 ? (char_u *)")" : p_sxq);
Now let's test this some more.
--
Over the years, I've developed my sense of deja vu so acutely that now
I can remember things that *have* happened before ...
Bram Moolenaar
Ben Fritz wrote:
Using & in a command is unusual. Having the command work with spaces in
the command name and argument name is much more important.
But we would like to make every command work...
--
Creating the world with Emacs: M-x let-there-be-light
Creating the world with Vim: :make world
mattn
> for the tail? It's possible, but I wonder where it breaks now.
* What is a problem
* Why 7.3.443 is needed.
RoDo
Vim need to parse the command and understand it correctly, then escape
the special chars.
Let's give it back to user and cmd.exe. XD
On Feb 15, 11:06 am, Bram Moolenaar <B...@Moolenaar.net> wrote:
> Ben Fritz wrote:
> > On Feb 14, 4:31 am, RoDo <royad...@gmail.com> wrote:
> > > echo system('echo a & echo b') fail
> > > should escape &
> > > echo system('echo a ^& echo b') pass
>
> > It looks like most of the use cases pass when using the () patch for
> > shellxquote, I like that option.
>
> > Since there are still some failures when using an unescaped & maybe we
> > should put a note in the help about how to escape it as well, probably
> > in :help :! and :help system() both.
>
> Using & in a command is unusual. Having the command work with spaces in
> the command name and argument name is much more important.
>
> But we would like to make every command work...
>
> --
> Creating the world with Emacs: M-x let-there-be-light
> Creating the world with Vim: :make world
>
> /// sponsor Vim, vote for features --http://www.Vim.org/sponsor/\\\
> \\\ an exciting new programming language --http://www.Zimbu.org ///
mattn
Andy Wokula
Summary what I remember so far, and a suggestion at the end.
Please correct mistakes.
------------------------------------------
Bug (before the patch):
shellcmdflag: /c
shellxquote: empty
user executes :!"my editor.exe" "my file.txt"
vim executes cmd /c "my editor.exe" "my file.txt"
cmd executes my editor.exe" "my file.txt
(obviously wrong)
Problem: cmd sometimes strips outer quotes (and doesn't provide an
option to force keeping the quotes)
Solution: always add surrounding quotes and force removing them with /s
Patch:
shellcmdflag: /s /c
shellxquote: "
Yeah, this works now:
user executes :!"my editor.exe" "my file.txt"
vim executes cmd /s /c ""my editor.exe" "my file.txt""
cmd executes "my editor.exe" "my file.txt"
------------------------------------------
Bug (with the patch)
shellcmdflag: /s /c
shellxquote: "
user executes :!editor "my&file.txt"
vim executes cmd /s /c "editor "my&file.txt""
cmd executes "editor "my & file.txt""
(two commands, separated by `&')
Problem: cmd doesn't always strip *outer* quotes, but stops before `&'
Solution: make cmd's argument "atomic" by enclosing it in `('...`)'
Patch (by RoDo):
shellcmdflag: /c (no need to strip quotes)
shellxquote: ( (and automatically use `)' at the end)
Yeah, this works now (also did without patch 7.4.443):
user executes :!editor "my&file.txt"
vim executes cmd /c (editor "my&file.txt")
cmd executes (editor "my&file.txt")
------------------------------------------
Bug (with RoDo's patch)
shellcmdflag: /c
shellxquote: (
user executes :!echo a & echo b
vim executes cmd /c (echo a & echo b)
cmd executes (echo a & echo b)
-> output: two lines, `a' and `b)'
Problem: echo does not ignore the trailing `)'
Solution: enclose cmd's argument in `"('...`)"'
(I have no idea why this works)
Patch:
shellcmdflag: /c (not sure if `/s /c' would make a difference,
I'd expect `/c' to always remove outer quotes)
shellxquote: "(
or maybe:
shellxquote: "(,)" (new syntax!)
Yeah, this works now (also did without patch 7.3.443):
user executes :!echo a & echo b
--
Andy
mattn
> shellcmdflag: /c (not sure if `/s /c' would make a difference,
> I'd expect `/c' to always remove outer quotes)
> shellxquote: "(
> or maybe:
> shellxquote: "(,)" (new syntax!)
>
> Yeah, this works now (also did without patch 7.3.443):
> user executes :!echo a & echo b
C:\>copy c:\vim\vim.exe "my editor.exe"
C:\>cmd /c ("my editor.exe" "my filea.txt" a & dir)
# This is meaning that editing two files. If the editor exit, do dir. But...
C:\>cmd /s /c ("my editor.exe" "my filea.txt" a & dir)
'dir)' is not recognized as an internal or external command,
operable program or batch file.
Thus, I guess "/s" is needed.
C:\>cmd /s /c "("my editor.exe" "my filea.txt" a & dir)"
This is working well. :)
Andy Wokula
>> Patch:
>> shellcmdflag: /c (not sure if `/s /c' would make a difference,
>> I'd expect `/c' to always remove outer quotes)
>> shellxquote: "(
>> or maybe:
>> shellxquote: "(,)" (new syntax!)
>>
>> Yeah, this works now (also did without patch 7.3.443):
>
> I tried following.
>
> C:\>copy c:\vim\vim.exe "my editor.exe"
>
> C:\>cmd /c ("my editor.exe" "my filea.txt" a& dir)
I thought your suggestion was
C:\>cmd /c "("my editor.exe" "my filea.txt" a & dir)"
This works here.
> # This is meaning that editing two files. If the editor exit, do dir. But...
>
> C:\>cmd /s /c ("my editor.exe" "my filea.txt" a& dir)
> 'dir)' is not recognized as an internal or external command,
> operable program or batch file.
he, and (side note) the include-trailing-`)' bug is not specific to `echo' then.
> Thus, I guess "/s" is needed.
>
> C:\>cmd /s /c "("my editor.exe" "my filea.txt" a& dir)"
ah, you now add quotes. but for me this also works without /s .
> This is working well. :)
>
> Thanks.
--
Andy
RoDo
cmd /c ("my editor.exe" "my filea.txt" a ^& dir) as well.
If we find a & inside "" like "&", don't touch it.
Otherwise, we automatic add ^ before & or wrap entire command with
"()".
Right?
Andy Wokula
> You can execute
> cmd /c ("my editor.exe" "my filea.txt" a ^& dir) as well.
Surprise -- it indeed executes `dir'.
This suggests that special characters within `(' ... `)' should be
escaped with `^'.
The following works as well:
cmd /c (^"my editor.exe^" ^"my filea.txt^" a ^& dir)
I found that `"(' ... `)"' still doesn't work for redirection:
cmd /c "(cat cm.txt > "c&m.txt")"
it tries to execute `m.txt")"'.
Doesn't work either for `('...`)':
cmd /c (cat cm.txt > "c&m.txt")
it creates `c&m.txt)'.
But `('...`)' works when inner special chars are escaped:
cmd /c (cat cm.txt ^> ^"c^&m.txt^")
cmd /c (echo a ^& echo b)
cmd /c (^"my editor.exe^" ^"my filea.txt^")
Special characters according to
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/cmd.mspx
are
& < > ( ) @ ^ |
This list might not be complete, the website says "for example".
> If we find a& inside "" like "&", don't touch it.
> Otherwise, we automatic add ^ before& or wrap entire command with
> "()".
> Right?
No, we escape all special chars.
New settings:
shellcmdflag: /c
shellxquote: (
and escape special chars with `^'.
--
Andy
Ben Fritz
On Feb 15, 8:11 am, Andy Wokula <anw...@yahoo.de> wrote:
>
> Doesn't work either for `('...`)':
> cmd /c (cat cm.txt > "c&m.txt")
>
> it creates `c&m.txt)'.
>
> But `('...`)' works when inner special chars are escaped:
> cmd /c (cat cm.txt ^> ^"c^&m.txt^")
> cmd /c (echo a ^& echo b)
> cmd /c (^"my editor.exe^" ^"my filea.txt^")
>
> & < > ( ) @ ^ |
> This list might not be complete, the website says "for example".
>
> > If we find a& inside "" like "&", don't touch it.
> > Otherwise, we automatic add ^ before& or wrap entire command with
> > "()".
> > Right?
>
> No, we escape all special chars.
>
> New settings:
> shellcmdflag: /c
> shellxquote: (
> and escape special chars with `^'.
>
QUOTES, and surround in parentheses?
That's...special.
Andy Wokula
>> New settings:
>> shellcmdflag: /c
>> shellxquote: (
>> and escape special chars with `^'.
>
> So you're saying, we must escape all special characters, INCLUDING
> QUOTES, and surround in parentheses?
Yes, including quotes!
> That's...special.
But solves all problems so far.
--
Andy
Benjamin Fritz
>> So you're saying, we must escape all special characters, INCLUDING
>> QUOTES, and surround in parentheses?
>
>
> Yes, including quotes!
>
>> That's...special.
>
>
> But solves all problems so far.
Yes, but that doesn't mean I need to like it :-P
Bram, you said at one point you don't want to start escaping things.
Specifically, you wrote:
> A command may intentionally use & and | to separate commands. Always
> escaping them will break this intentional use.
It looks like using parentheses and always escaping everything
(including quotes) allows the intentional use to work. What do you
think of this? If it is an acceptable solution, how should we
accomplish the "always escaping"? Should Vim automatically escape & <
> ( ) @ ^ | and " whenever shell is cmd.exe and shellxquote is ( ? Or
should we add a new option (set by default)?
Benjamin Fritz
Parentheses are still weird :-(
C:\eclim-git\eclim>cmd /c echo abc)
abc)
C:\eclim-git\eclim>cmd /c (echo abc))
) was unexpected at this time.
C:\eclim-git\eclim>cmd /c (echo abc^))
) was unexpected at this time.
C:\eclim-git\eclim>cmd /c (echo abc^^))
abc)
C:\eclim-git\eclim>cmd /c (echo abc^(^))
) was unexpected at this time.
C:\eclim-git\eclim>cmd /c (echo ^"abc^)^")
"abc)"
C:\eclim-git\eclim>cmd /c (^(echo abc^) ^& ^(pause^))
abc
Press any key to continue . . .
Benjamin Fritz
> Parentheses are still weird :-(
>
> C:\eclim-git\eclim>cmd /c echo abc)
> abc)
>
I just realized this one is invalid anyway, since MS says you need to
escape a literal ) and here the user does not
> C:\eclim-git\eclim>cmd /c (echo abc))
> ) was unexpected at this time.
>
> C:\eclim-git\eclim>cmd /c (echo abc^))
> ) was unexpected at this time.
>
Hence, neither of these work. If the user is doing the right thing,
they will have escaped the ).
> C:\eclim-git\eclim>cmd /c (echo abc^^))
> abc)
>
If the user properly escapes the ), Vim will actually execute with ^^^)
> C:\eclim-git\eclim>cmd /c (echo abc^(^))
> ) was unexpected at this time.
>
Again, user ought to have escaped the () to begin with.
> C:\eclim-git\eclim>cmd /c (echo ^"abc^)^")
> "abc)"
>
The above command would result from :!echo "abc)" so we know quoting works!
> C:\eclim-git\eclim>cmd /c (^(echo abc^) ^& ^(pause^))
> abc
> Press any key to continue . . .
And so does the auto-escaping of parens intentionally used for grouping.
I think we should do the auto-escaping, with shellxquote=(, and
additionally add a help note by :! and system() that "According to the
Microsoft documentation for cmd.exe, you either need to escape (with a
^ character) any of these characters appearing literally in your
command, or surround them in quotes ("): > ( ) @ ^ |." Should we
mention that Vim will escape them again, and also escape "?
Bram Moolenaar
Ben Fritz wrote:
> I think we should do the auto-escaping, with shellxquote=(, and
> additionally add a help note by :! and system() that "According to the
> Microsoft documentation for cmd.exe, you either need to escape (with a
> ^ character) any of these characters appearing literally in your
> command, or surround them in quotes ("): > ( ) @ ^ |." Should we
> mention that Vim will escape them again, and also escape "?
Yes, I think the use should escape & and | when they appear in a
command. I haven't verified, but I believe they are illegal in a file
name.
Please try the patch below. I currently do not have a Windows machine
to try this out.
*** ../vim-7.3.444/src/misc2.c 2012-01-20 17:15:47.000000000 +0100
--- src/misc2.c 2012-02-16 05:34:37.000000000 +0100
***************
*** 3230,3236 ****
{
STRCPY(ncmd, p_sxq);
STRCAT(ncmd, cmd);
! STRCAT(ncmd, p_sxq);
retval = mch_call_shell(ncmd, opt);
vim_free(ncmd);
}
--- 3230,3240 ----
{
STRCPY(ncmd, p_sxq);
STRCAT(ncmd, cmd);
! /* When 'shellxquote' is ( append ).
! * When 'shellxquote' is "( append )". */
! STRCAT(ncmd, STRCMP(p_sxq, "(") == 0 ? (char_u *)")"
! : STRCMP(p_sxq, "\"(") == 0 ? (char_u *)")\""
! : p_sxq);
retval = mch_call_shell(ncmd, opt);
vim_free(ncmd);
}
*** ../vim-7.3.444/src/option.c 2012-02-12 23:23:25.000000000 +0100
--- src/option.c 2012-02-16 05:31:12.000000000 +0100
***************
*** 3935,3949 ****
*
* To avoid this, use the /s argument in addition to /c to force the
* stripping behavior, and also set shellxquote to automatically
! * surround the entire command in quotes (which get stripped as
! * noted).
*/
/* Set shellxquote default to add the quotes to be stripped. */
idx3 = findoption((char_u *)"sxq");
if (idx3 >= 0 && !(options[idx3].flags & P_WAS_SET))
{
! p_sxq = (char_u *)"\"";
options[idx3].def_val[VI_DEFAULT] = p_sxq;
}
--- 3935,3949 ----
*
* To avoid this, use the /s argument in addition to /c to force the
* stripping behavior, and also set shellxquote to automatically
! * surround the entire command in "(cmd)" (the double quotes get
! * stripped as noted).
*/
/* Set shellxquote default to add the quotes to be stripped. */
idx3 = findoption((char_u *)"sxq");
if (idx3 >= 0 && !(options[idx3].flags & P_WAS_SET))
{
! p_sxq = (char_u *)"\"(";
options[idx3].def_val[VI_DEFAULT] = p_sxq;
}
*** ../vim-7.3.444/src/os_win32.c 2011-08-27 15:10:00.000000000 +0200
--- src/os_win32.c 2012-02-16 05:47:36.000000000 +0100
***************
*** 3908,3915 ****
newcmd = lalloc(cmdlen, TRUE);
if (newcmd != NULL)
{
! char_u *cmdbase = (*cmd == '"' ? cmd + 1 : cmd);
if ((STRNICMP(cmdbase, "start", 5) == 0) && vim_iswhite(cmdbase[5]))
{
STARTUPINFO si;
--- 3908,3920 ----
newcmd = lalloc(cmdlen, TRUE);
if (newcmd != NULL)
{
! char_u *cmdbase = cmd;
+ /* Skip a leading ", ( and "(. */
+ if (*cmdbase == '"' )
+ ++cmdbase;
+ if (*cmdbase == '(')
+ ++cmdbase;
if ((STRNICMP(cmdbase, "start", 5) == 0) && vim_iswhite(cmdbase[5]))
{
STARTUPINFO si;
***************
*** 3953,3968 ****
* empty, keep the double quotes around the command.
* Otherwise remove the double quotes, they aren't needed
* here, because we don't use a shell to run the command. */
! if (*cmd == '"' && *p_sxq == NUL)
{
! newcmd[0] = '"';
! STRCPY(newcmd + 1, cmdbase);
! }
! else
! {
! STRCPY(newcmd, cmdbase);
! if (*cmd == '"' && *newcmd != NUL)
! newcmd[STRLEN(newcmd) - 1] = NUL;
}
/*
--- 3958,3983 ----
* empty, keep the double quotes around the command.
* Otherwise remove the double quotes, they aren't needed
* here, because we don't use a shell to run the command. */
! if (cmdbase > cmd)
{
! if (STRNCMP(cmd, p_sxq, cmd - cmdbase) != 0)
! {
! STRCPY(newcmd, cmd);
! }
! else
! {
! char_u *p;
!
! STRCPY(newcmd, cmdbase);
! /* Remove a trailing ", ) and )" if they have a match
! * at the start of the command. */
! p = newcmd + STRLEN(newcmd);
! if (p > newcmd && p[-1] == '"' && *cmd == '"')
! *--p = NUL;
! if (p > newcmd && p[-1] == ')'
! && (*cmd =='(' || cmd[1] == '('))
! *--p = NUL;
! }
}
/*
--
hundred-and-one symptoms of being an internet addict:
60. As your car crashes through the guardrail on a mountain road, your first
instinct is to search for the "back" button.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
Andy Wokula
> Ben Fritz wrote:
>
>> I think we should do the auto-escaping, with shellxquote=(, and
>> additionally add a help note by :! and system() that "According to
>> the Microsoft documentation for cmd.exe, you either need to escape
>> (with a ^ character) any of these characters appearing literally in
>> your command, or surround them in quotes ("):> ( ) @ ^ |." Should we
>> mention that Vim will escape them again, and also escape "?
>
> command. I haven't verified, but I believe they are illegal in a file
> name.
`|' is illegal in a file name, but `&' is not.
Characters with special meaning [1]: " < > | & ( ) @ ^
Characters that can't be used in file names: " < > | * ?
Characters to be escaped [2] in a file name: Space & ( ) @ ^
[1] Vim should escape these with `^'
[2] prepend `^' or enclose file name in quotes
> Please try the patch below. I currently do not have a Windows machine
> to try this out.
Does the patch escape special characters? I can't see code for it.
Again, suggested settings are:
shellcmdflag: /c
shellxquote: (
escape special characters with `^'
IMHO shellxquote = '"(' should not be supported, it was an
intermediate solution with its own problems.
> if (idx3>= 0&& !(options[idx3].flags& P_WAS_SET))
> {
> ! p_sxq = (char_u *)"\"";
> options[idx3].def_val[VI_DEFAULT] = p_sxq;
> }
>
> --- 3935,3949 ----
> *
> * To avoid this, use the /s argument in addition to /c to force the
> * stripping behavior, and also set shellxquote to automatically
> ! * surround the entire command in "(cmd)" (the double quotes get
> ! * stripped as noted).
> */
>
> /* Set shellxquote default to add the quotes to be stripped. */
> idx3 = findoption((char_u *)"sxq");
> if (idx3>= 0&& !(options[idx3].flags& P_WAS_SET))
> {
> ! p_sxq = (char_u *)"\"(";
> options[idx3].def_val[VI_DEFAULT] = p_sxq;
> }
>
> *** ../vim-7.3.444/src/os_win32.c 2011-08-27 15:10:00.000000000 +0200
> --- src/os_win32.c 2012-02-16 05:47:36.000000000 +0100
> ***************
> *** 3908,3915 ****
> newcmd = lalloc(cmdlen, TRUE);
> if (newcmd != NULL)
> {
> ! char_u *cmdbase = (*cmd == '"' ? cmd + 1 : cmd);
>
> if ((STRNICMP(cmdbase, "start", 5) == 0)&& vim_iswhite(cmdbase[5]))
> {
> STARTUPINFO si;
> --- 3908,3920 ----
> newcmd = lalloc(cmdlen, TRUE);
> if (newcmd != NULL)
> {
> ! char_u *cmdbase = cmd;
>
> + /* Skip a leading ", ( and "(. */
> + if (*cmdbase == '"' )
> + ++cmdbase;
> + if (*cmdbase == '(')
> + ++cmdbase;
> if ((STRNICMP(cmdbase, "start", 5) == 0)&& vim_iswhite(cmdbase[5]))
> ! if (p> newcmd&& p[-1] == '"'&& *cmd == '"')
> ! *--p = NUL;
> ! if (p> newcmd&& p[-1] == ')'
> ! && (*cmd =='(' || cmd[1] == '('))
> ! *--p = NUL;
> ! }
> }
>
> /*
--
Andy
Bram Moolenaar
I wrote:
> Ben Fritz wrote:
>
> > I think we should do the auto-escaping, with shellxquote=(, and
> > additionally add a help note by :! and system() that "According to the
> > Microsoft documentation for cmd.exe, you either need to escape (with a
> > ^ character) any of these characters appearing literally in your
> > command, or surround them in quotes ("): > ( ) @ ^ |." Should we
> > mention that Vim will escape them again, and also escape "?
>
> Yes, I think the use should escape & and | when they appear in a
> command. I haven't verified, but I believe they are illegal in a file
> name.
>
> Please try the patch below. I currently do not have a Windows machine
> to try this out.
I would like to send out this patch, but I first need to know if it
doesn't break the build. Please tell me if you successfully compiled
this.
--
hundred-and-one symptoms of being an internet addict:
67. Your hard drive crashes. You haven't logged in for two hours. You start
to twitch. You pick up the phone and manually dial your ISP's access
number. You try to hum to communicate with the modem. You succeed.
Bram Moolenaar
Andy Wokula wrote:
> Am 16.02.2012 05:54, schrieb Bram Moolenaar:
> > Ben Fritz wrote:
> >
> >> I think we should do the auto-escaping, with shellxquote=(, and
> >> additionally add a help note by :! and system() that "According to
> >> the Microsoft documentation for cmd.exe, you either need to escape
> >> (with a ^ character) any of these characters appearing literally in
> >> your command, or surround them in quotes ("):> ( ) @ ^ |." Should we
> >> mention that Vim will escape them again, and also escape "?
> >
> > Yes, I think the user should escape& and | when they appear in a
> > command. I haven't verified, but I believe they are illegal in a file
> > name.
>
> `|' is illegal in a file name, but `&' is not.
Weird.
> Characters with special meaning [1]: " < > | & ( ) @ ^
> Characters that can't be used in file names: " < > | * ?
> Characters to be escaped [2] in a file name: Space & ( ) @ ^
>
> [1] Vim should escape these with `^'
> [2] prepend `^' or enclose file name in quotes
>
>
> > Please try the patch below. I currently do not have a Windows machine
> > to try this out.
>
> Does the patch escape special characters? I can't see code for it.
> Again, suggested settings are:
> shellcmdflag: /c
> shellxquote: (
> escape special characters with `^'
I am against automatic escaping, because it makes it impossible to have
a & separate two commands. The user should do the escaping.
Generally, Vim should do as little as possible. We currently don't do
any of this ^ escaping, I am very hesitant to start this now.
Also because it is unclear to me where the un-escaping is done. The echo
example is confusing.
> IMHO shellxquote = '"(' should not be supported, it was an
> intermediate solution with its own problems.
I thought the conclusion was this was the best solution overall.
What are the problems?
--
hundred-and-one symptoms of being an internet addict:
68. Your cat always puts viruses on your dogs homepage
Andy Wokula
> Andy Wokula wrote:
>
>> Am 16.02.2012 05:54, schrieb Bram Moolenaar:
>>> Ben Fritz wrote:
>>>
>>>> I think we should do the auto-escaping, with shellxquote=(, and
>>>> additionally add a help note by :! and system() that "According to
>>>> the Microsoft documentation for cmd.exe, you either need to escape
>>>> (with a ^ character) any of these characters appearing literally in
>>>> your command, or surround them in quotes ("):> ( ) @ ^ |." Should
>>>> we mention that Vim will escape them again, and also escape "?
>>>
>>> Yes, I think the user should escape& and | when they appear in a
>>> command. I haven't verified, but I believe they are illegal in a file
>>> name.
>>
>> `|' is illegal in a file name, but `&' is not.
>
> Weird.
>
>> Characters with special meaning [1]: "< > |& ( ) @ ^
>> Characters that can't be used in file names: "< > | * ?
>>
>> [1] Vim should escape these with `^'
>> [2] prepend `^' or enclose file name in quotes
>>
>>
>>> Please try the patch below. I currently do not have a Windows
>>> machine to try this out.
>>
>> Does the patch escape special characters? I can't see code for it.
>> Again, suggested settings are:
>> shellcmdflag: /c
>> shellxquote: (
>> escape special characters with `^'
>
> I am against automatic escaping, because it makes it impossible to
No, it's different between `('...`)':
An escaped `&' separates two commands, for example:
cmd /c (command1 ^& command2)
To get a literal `&', one has to write `^^^&':
cmd /c (notepad my^^^&file.txt)
(the file name is `my&file.txt')
> Generally, Vim should do as little as possible. We currently don't do
> any of this ^ escaping, I am very hesitant to start this now.
Ok, but the only thing that worries me at the moment is to get the right
set of special characters.
The user executes in Vim
:!notepad my^&file.txt & dir
Vim executes
cmd /c (notepad my^^^&file.txt ^& dir)
cmd.exe does the unescaping while parsing the argument, and finally executes
notepad my^&file.txt & dir
The same with quotes:
:!notepad "my&file.txt" & dir
-> cmd /c (notepad ^"my^&file.txt^" ^& dir)
-> notepad "my&file.txt" & dir
> Also because it is unclear to me where the un-escaping is done. The
> echo example is confusing.
cmd.exe unescapes text after `('.
>> IMHO shellxquote = '"(' should not be supported, it was an
>> intermediate solution with its own problems.
>
> I thought the conclusion was this was the best solution overall.
> What are the problems?
Above example (= mattn's original request) doesn't work:
:!notepad "my&file.txt"
-> cmd /s /c "(notepad "my&file.txt")"
Error: command "file.txt")"" not found
--
Andy
Andy Wokula
> to try this out.
I had success with applying the patch.
:set shcf? sxq?
shellcmdflag=/s /c
shellxquote="(
:!"my cat.exe" "file with spaces.txt"
-> cmd.exe /s /c "("my cat.exe" "file with spaces.txt")"
OK
--
Andy
Edwin Steiner
> Bram Moolenaar wrote:
> > Andy Wokula wrote:
> >
> > > Am 14.02.2012 13:20, schrieb mattn:
> > > > cmd /c "(echo a& echo b)"
> > >
> > > Ok!
One caveat that I learned the hard way:
Do not trust "echo" too much when testing shell quoting on Windows!
On Windows the separation of the command line into words (i.e. argv)
is not done by the shell but is left to the invoked program.
Most programs (not "echo"!) will rely on the Microsoft C library for that which
contains its own un-quoting with its own insanities and bugs.
So you have to fight at least two levels when quoting.
I found it most reliable for testing to write a simple C program
that prints its argv[] array.
best regards
Edwin
--
Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
John Beckett
> I found it most reliable for testing to write a simple C
> program that prints its argv[] array.
I did that a few years ago, and in case anyone is interested,
here is what I used.
/* Echo arguments in one line.
* John Beckett 2007/05/25
* This is to see what is passed to program by Vim system().
*
* If name of program contains "echoarg", program runs
* and terminates.
* Otherwise, program prompts user to press Enter so the window
* can be seen, for example, if invoked with Vim system().
*/
#include <stdio.h>
#include <string.h>
#include <windows.h>
#define MAXLN 2000
int main(int argc, char *argv[])
{
char buf[MAXLN+10+1] = "";
char *p = buf;
int j;
int wait = (strstr(argv[0], "echoarg") == NULL);
printf("Raw command line:\n%s\n\n", GetCommandLine());
printf("Arguments passed to argv:\n");
for ( j = 0; j < argc; ++j ) {
if ( (p + strlen(argv[j])) > (buf + MAXLN) )
break;
p += sprintf(p, "[%d %s]", j, argv[j]);
}
printf("%s\n", buf);
if ( wait ) {
printf("\nEchoarg will terminate when you press Enter\n");
getchar();
}
return 0;
}
John
George V. Reilly
> Edwin Steiner wrote:
> > I found it most reliable for testing to write a simple C
> > program that prints its argv[] array.
>
> I did that a few years ago, and in case anyone is interested,
> here is what I used.
I fought with all this shellquote nonsense long ago and am probably
partly responsible for the current tangle. I mostly use Macs and Linux
nowadays, so I'm no longer particularly invested in the Windows
version of Vim.
Python's subprocess package takes an interesting approach. You pass it
a list of arguments, which is easy if slightly tedious for the user to
get right, and there's no need to worry about quoting individual
arguments. At the last minute, it carefully concatenates the list into
a correctly quoted string before calling CreateProcess.
See the implementation at
http://hg.python.org/cpython/file/b8593ec7e8c5/Lib/subprocess.py
--
/George V. Reilly geo...@reilly.org Twitter: @georgevreilly
http://www.georgevreilly.com/blog http://blogs.cozi.com/tech
Edwin Steiner
As a further data point: I have found the following algorithm for
Windows shell quoting quite reliable under the following conditions:
* Windows 2000, Windows XP, or Windows 7
* the assembled command is executed using "system"
* both parent and child programs compiled by MinGW
I tested this algorithm quite thoroughly, but your mileage may vary.
In particular for redirecting output I have found it most reliable
to put the >"file name" in _front_ of the command. See quoteRedirectedCommand below.
And now be prepared for (counter-)insanity and have fun!
best regards
Edwin
Special characters mentioned:
double_quote "
backslash \
caret ^
whitespace any ch with isspace(ch)
special_char any of these: less (<), greater (>), pipe (|),
ampersand (&), percent (%), caret (^)
Notation:
+= and + are used for string concatenation
Usage is like this:
system(double_quote + quoteCommand(double_quote + program_unquoted + double_quote + space separated arguments quoted individually with quoteArgument) + double_quote)
or
system(double_quote + quoteRedirectedCommand(double_quote + program_unquoted + double_quote + space separated arguments quoted individually with quoteArgument, filename_unquoted) + double_quote)
Here is the algorithm in pseudo-code:
String quoteArgument(String arg)
{
if (arg is the empty string)
{
quoted = double_quote + double_quote
}
else
{
quoted = empty string
pos = 0
while (pos < length of arg)
{
n = 0
while (arg[pos + n] is whitespace or backslash)
n = n + 1
if (n > 0)
{
quoted += double_quote
for each character ch in arg[pos..pos+n-1]
{
if (ch is backslash)
quoted += backslash + backslash
else
quoted += ch
}
quoted += double_quote
pos = pos + n
}
else
{
if (arg[pos] is double_quote)
{
quoted += backslash + double_quote
}
else
{
quoted += arg[pos]
}
pos = pos + 1
}
}
}
return quoted
}
String quoteCommand(String cmd)
{
even_number_of_quotes = true
quoted = empty string
for each character ch in cmd
{
if (ch is double_quote)
{
even_number_of_quotes = !even_number_of_quotes
}
else if ((ch is special_char) && even_number_of_quotes)
{
# quote characters that are special to cmd.exe:
quoted += caret
}
quoted += ch
# protect against expansion of %FOO% by turning it into %""FOO%:
if (ch is '%' and the following character is not double_quote or backslash)
quoted += double_quote + double_quote
}
return quoted
}
String quoteRedirectedCommand(String cmd,
String filename)
{
return ">" + quoteCommand(double_quote + filename + double_quote + space + cmd)
Edwin Steiner
> # protect against expansion of %FOO% by turning it into %""FOO%:
>
> if (ch is '%' and the following character is not double_quote or
> backslash)
> quoted += double_quote + double_quote
only insert the "" if there actually *is* a following character in cmd.
Reason: To avoid sequences of three double quotes (""") which I found to trigger bugs in the un-quoting.
Andy Wokula
Please, somebody with a Windows 7 / Vista / Win 2003 machine:
Does the following edit new&file.txt ?
cmd /c (notepad ^"new^&file.txt^")
It works here on good ol XP.
I suspect that shell quoting has changed since:
Is it ok to escape the double quotes?
And, similar,
Does the following output the contents of new&file.txt ?
cmd /c (type ^"new^&file.txt^")
or does it output something like
Error, not a command: "file.txt")"
The error case would be awful.
--
Andy
Ingo Karkat
Works on both Vista Professional 32-bit and 7 Ultimate 64-bit, both in a command
prompt and from the Start > Run... dialog.
I feel your pain; cmd.exe [quoting] is awful. Thanks for putting so much energy
into it!
-- regards, ingo
Jürgen Krämer
Hi,
Andy Wokula wrote:
> Am 15.02.2012 19:36, schrieb Andy Wokula:
>> Am 15.02.2012 17:22, schrieb Ben Fritz:
>>>> New settings:
>>>> shellcmdflag: /c
>>>> shellxquote: (
>>>> and escape special chars with `^'.
>>>
>>> So you're saying, we must escape all special characters, INCLUDING
>>> QUOTES, and surround in parentheses?
>>
>> Yes, including quotes!
>>
>>> That's...special.
>>
>> But solves all problems so far.
>
> Please, somebody with a Windows 7 / Vista / Win 2003 machine:
>
> Does the following edit new&file.txt ?
>
> cmd /c (notepad ^"new^&file.txt^")
>
> It works here on good ol XP.
on a German Windows 7 64-bit it does, too. Notepad opens, showing a
dialog with
Die Datei new&file.txt kann nicht gefunden werden.
M�chten Sie eine neue Datei erstellen?
Ja Nein Abbrechen
which translates to
The file new&file.txt could not be found.
Do you want to create a new file?
Yes No Cancel
Answering "Ja" puts a new file named new&file.txt in the current
directory.
> I suspect that shell quoting has changed since:
> Is it ok to escape the double quotes?
>
> And, similar,
> Does the following output the contents of new&file.txt ?
> cmd /c (type ^"new^&file.txt^")
>
> or does it output something like
> Error, not a command: "file.txt")"
>
> The error case would be awful.
No error here, the file is correctly displayed. Also deleting it with
cmd /c (del ^"new^&file.txt^")
works.
Regards,
J�rgen
--
Sometimes I think the surest sign that intelligent life exists elsewhere
in the universe is that none of it has tried to contact us. (Calvin)
Andy Wokula
> works.
Thanks :-)
--
Andy
Benjamin Fritz
> Please try the patch below. I currently do not have a Windows machine
> to try this out.
I feel like an idiot, but I cannot get the patch to apply cleanly; and
importing into mq just says the patch is empty!
Can someone see what I'm doing wrong? I admit I've only successfully
used the command-line patch tool once or twice ever in the past.
F:\Documents\Ben\vim>patch --version
patch 2.5.9
Copyright (C) 1988 Larry Wall
Copyright (C) 2003 Free Software Foundation, Inc.
This program comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of this program
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.
written by Larry Wall and Paul Eggert
F:\Documents\Ben\vim>patch -i ..\vim-patches\bram_bad_shellxquote.diff -p2
missing header for context diff at line 3 of patch
patching file src/misc2.c
Hunk #1 FAILED at 3230.
1 out of 1 hunk FAILED -- saving rejects to file src/misc2.c.rej
missing header for context diff at line 26 of patch
patching file src/option.c
Hunk #1 FAILED at 3935.
1 out of 1 hunk FAILED -- saving rejects to file src/option.c.rej
missing header for context diff at line 61 of patch
patching file src/os_win32.c
Hunk #1 FAILED at 3908.
Hunk #2 FAILED at 3958.
2 out of 2 hunks FAILED -- saving rejects to file src/os_win32.c.rej
F:\Documents\Ben\vim>hg revert --all
reverting src\misc2.c
reverting src\option.c
reverting src\os_win32.c
F:\Documents\Ben\vim>patch -i ..\vim-patches\bram_bad_shellxquote.diff -p1
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|*** ../vim-7.3.444/src/misc2.c 2012-01-20 17:15:47.000000000 +0100
|--- src/misc2.c 2012-02-16 05:34:37.000000000 +0100
--------------------------
File to patch:
F:\Documents\Ben\vim>REM convert to unix format in gvim...
F:\Documents\Ben\vim>patch -i ..\vim-patches\bram_bad_shellxquote.diff -p2
patch: **** malformed patch at line 12: 3240 ----
Andy Wokula
> On Wed, Feb 15, 2012 at 10:54 PM, Bram Moolenaar<Br...@moolenaar.net> wrote:
>> Please try the patch below. I currently do not have a Windows machine
>> to try this out.
>
> I feel like an idiot, but I cannot get the patch to apply cleanly; and
> importing into mq just says the patch is empty!
>
> Can someone see what I'm doing wrong? I admit I've only successfully
> used the command-line patch tool once or twice ever in the past.
On Windows, the 'fileformat' of the patchfile must be "dos".
Otherwise the patch program cries errors.
From the vim source code root dir, I did (IIRC):
patch -p0 < path\to\patchfile
--
Andy