exec argument protection -- patch available
Andreas Leitgeb
There have been many threads about changing the command
"exec" such, that it would be possible to pass arguments
that look like redirections verbatim to the external command.
I'm now coming up with a patch, that I do not yet consider
mature for a TIP, because it contains a very ugly (but sound)
hack. I hope to find a nicer alternative with your help.
The patch (for tcl8.4.3-src.tgz) can be found at:
<http://www.logic.at/people/avl/tcltk/tcl-4.3.0-exec.patch>
The syntax that it introduces is rather minimal, but sufficient
to be 100% compatible and to principially allow for arbitrary
arguments:
Let's start with an example:
set string "<hello>"
exec -quote -- echo '$string > 'foo ;# writes "<hello>\n" to file "'foo".
set string "'<hello>" ;# *any* user-specified string
exec -quote echo '$string '| >foo ;# writes "'<hello> |\n" to file "foo".
without option "-quote", behaviour is just like it always has been.
The name of the option and the special char ' have been arbitrarily
chosen. the char can be anything except |<>2& but must be constant,
as it is not possible to pass more than a few yes/no -informations
down to the nested functions.
The option -quote does nothing else than remove at most one leading
single-quote-char (') from every argument to exec, but only after
all the redirections and pipes have been parsed. Since "'" cannot
be the start of any redirection-meta-argument, special treatment
is bypassed. If for a redirection the argument is separate (as in
exec ... > $filename) then $filename is NOT subject to '-removal!
The mechanics of the patch:
starting from the "exec"-command procedure, which now recognizes
the "-quote" option, this boolean piece of information needs to
be passed into two other functions, whose (both) interface cannot
be enhanced by an extra argument.
Tcl_OpenCommandChannel already has an argument "flags" which is a
set of OR'd constants defined in tcl.h
The patch uses a new flag TCL_QUOTE now defined in tcl.h to
pass through the information.
TclCreatePipeline has no clean option for an extra bit of info.
To work around that, I currently misuse the "argc":
if quote is set, argc will be mirrored to negative:
argc= -1 -argc;
inside TclCreatePipeline a negative argc is recognized,
mirrorred back, and the local flag set.
An alternative (not much nicer) would be to overwrite
argv[argc] which is by definition always NULL.
It could be replaced by a static char-pointer pointing to
an statically defined empty string, and inside TclCreatePipeline
it would be compared to that (using ==, not strcmp!) and
handled correctly (flag set and argv[argc] reset to NULL)
Because the string would only be visible in that module,
no other user of TclCreatePipeline would know this pointer.
I'm truely hoping that this might soon close this very unhappy
un-orthogonality of exec.
PS: currently, the patch does not yet include user-doku but only
local comments. Once this patch is made TIP-ready, I'll add the
patches for user-doku. (exec.n and OpenFileChnl.3)
PS: open "|..." is not (yet) affected by the change, but could be
made to make use of it, if someone comes up with a compatible
syntax to allow explictly requesting this behaviour. (I didn't
find one, yet)
PS: It worked nicely for my own tests under linux.
--
Nichts ist schlimmer als zu wissen, wie das Unheil sich entwickelt,
und voll Ohnmacht zusehn muessen - das macht mich voellig krank...
-- (Musical Elisabeth; "Die Schatten werden laenger")
steve bold
|<>. One use for such an exec is to allow execution of a program
where a string, specified at runtime by the user, is passed as a
command line argument. Unfortunately, this quoted exec still has
' as a special character and so can't fully handle this case.
I had been toying with a similar idea in which all the arguments
following the switches are taken as literal strings. You'd then need
another switch to support redirection, e.g. '-file stdout /tmp/myOutputFile'.
Unfortunately, it wouldn't allow pipelines with arguments containing
literal <>|, but that's probably a very rare case.
Andreas Leitgeb
> Certainly it would be useful to have an exec that supports literal
> |<>. One use for such an exec is to allow execution of a program
> where a string, specified at runtime by the user, is passed as a
> command line argument. Unfortunately, this quoted exec still has
> ' as a special character and so can't fully handle this case.
This is wrong! Of course, this case is handled!
The point is, that *at most one* quote is removed,
so if *you* add one, every previously existing one
is preserved.
foreach arg [list <hello> 'hello '<hello 2>hello] {
exec -quote echo '$arg
}
does (with the patch applied) give you:
<hello>
'hello
'<hello
2>hello
For immediate strings (like "echo" in the example above) you
don't need a quote, but you could also add one:
exec -quote 'echo '$arg
would have been equivalent.
> I had been toying with a similar idea in which all the arguments
> following the switches are taken as literal strings. You'd then need
> another switch to support redirection, e.g. '-file stdout /tmp/myOutputFile'.
With the patch included you could write a proc to wrap exec and
support all the fancy options.
For those, who can't apply the patch:
It is roughly equivalent to executing each prog (of the pipeline)
through an external wrapper-prog which strips (at most!) one quote
from each argument. Unfortunately, such a wrapper prog cannot be
written in current tcl, for recursively the same reason.
steve bold
"Andreas Leitgeb" <Andreas...@siemens.at> wrote in message news:slrnbgqb8j.eo....@pcc524.gud.siemens.at...
Donal K. Fellows
> I'm now coming up with a patch, that I do not yet consider
> mature for a TIP, because it contains a very ugly (but sound)
> hack. I hope to find a nicer alternative with your help.
>
> The patch (for tcl8.4.3-src.tgz) can be found at:
> <http://www.logic.at/people/avl/tcltk/tcl-4.3.0-exec.patch>
Let me point out at this point that you do not have to have a mature
implementation to start writing the TIP. It is the start point of the
formal process, not the end of it. If you want some help getting a
TIP up and going on this matter, email the kernel of it to me.
Indeed, you've got most of what you'd need in your article already!
The main thing I'd need extra is a copyright statement that permits
editing/publishing/etc (most TIP authors, including myself, just use
the public domain for everything.)
Mind you, if you send the message while I'm at the Tcl Conference it
might take me a while to respond. :^)
> PS: currently, the patch does not yet include user-doku but only
> local comments. Once this patch is made TIP-ready, I'll add the
> patches for user-doku. (exec.n and OpenFileChnl.3)
Documenting the alterations is probably easier than writing the tests
(which you *will* have to do to get the code into the core.) But
doesn't need to be done before writing the TIP. I hardly ever write
the code before the TIP, and in fact the past few times I've actually
co-developed TIP and code, with the TIP forming the basis for the
documentation. One of the main purposes of the TCT is to help with
the engineering of changes to the core; there might be aspects of your
changes which we can help with and which you've not considered yet.
Where on earth did this "community assumption" (that the TIP comes
after the implementation) come from? IT IS NOT TRUE. If we take the
example of the Dict TIP (#111, http://purl.org/tcl/tip/111.html), I
wrote that TIP *before* I wrote the implementation. (I did revise the
TIP after writing the code, I'll admit. But that happened later on.
The TIP came first.) You[*] can do it too. Don't be shy!
Donal.
[* Generic "you" here, and not "you" as referring specifically to any
one person. ]
Andreas Leitgeb
> Let me point out at this point that you do not have to have a mature
> implementation to start writing the TIP. It is the start point of the
> formal process, not the end of it.
A TIP without an implementation likely falls into forgotten-ness.
Probably same for a lousy implementation.
> If you want some help getting a
> TIP up and going on this matter, email the kernel of it to me.
> Indeed, you've got most of what you'd need in your article already!
> The main thing I'd need extra is a copyright statement that permits
> editing/publishing/etc (most TIP authors, including myself, just use
> the public domain for everything.)
Ha, if only every problem was as easily solveable :-)
Yes, of course it all goes to public domain. (usually I'm
more a GPL'er but for Tcl-core-related stuff, public domain
is just fine with me)
> Documenting the alterations is probably easier than writing the tests
> (which you *will* have to do to get the code into the core.) But
> doesn't need to be done before writing the TIP.
Ok, I'll prepare for that.
By the way: Bug # 768678 is directly related to this patch:
solving it most likely has big influence on a possible clean
implementation of it.
> I hardly ever write
> the code before the TIP, and in fact the past few times I've actually
> co-developed TIP and code, with the TIP forming the basis for the
> documentation.
If you, Donal, write a TIP, everyone knows, that a good
implementation will soon follow. :-)
This is not necessarily the case if I wrote a TIP.
> One of the main purposes of the TCT is to help with
> the engineering of changes to the core; there might be aspects of your
> changes which we can help with and which you've not considered yet.
well, yes, I'm looking forward...
> Where on earth did this "community assumption" (that the TIP comes
> after the implementation) come from? IT IS NOT TRUE.
Let's say its a perceived correlation, that a TIP without
at least a usable prototype of an implementation tends to
be forgotten. It's just that often no one else than the TIPper
gathers the energy to implement it, even if the usefulness seems
widely accepted.