fexecve()

[Kenny McCormack]

I've been playing with fexecve(). Here's a program called "lgo.c":

% cat lgo.c
#include <stdio.h>
#include <unistd.h>

int main(int argc, char **argv)
{
extern char **environ;
(void) argc;
fexecve(0,argv,environ);
perror("fexecve");
return 1;
}

Now, I have a previously compiled program a.out, which we can run thusly:
% ./lgo < a.out
sizeof(int) = 4

Now, we try to do the same thing using the output of gcc (I have verified
that sending gcc's output to /dev/stdout does indeed send the compiled
binary there) Note to C purists: Ignore the warnings - that's not the
point here:
% echo 'main() { printf("sizeof(int) = %d\\n",sizeof(int)); }' | gcc -xc - -o /dev/stdout | ./lgo
fexecve: Permission denied
<stdin>: In function 'main':
<stdin>:1:10: warning: incompatible implicit declaration of built-in function 'printf' [enabled by default]
<stdin>:1:1: warning: format '%d' expects argument of type 'int', but argument 2 has type 'long unsigned int' [-Wformat]
%

Now, not that this particularly surprises me - and there could be any
number of reasons why it doesn't work - but I'm curious. Is there anything
I can add to lgo.c to make it work on a pipe as above?

--
Modern Conservative: Someone who can take time out from flashing her
wedding ring around and bragging about her honeymoon to complain that a
fellow secretary who keeps a picture of her girlfriend on her desk is
"flauting her sexuality" and "forcing her lifestyle down our throats".

[Rainer Weikusat]

> I can add to lgo.c to make it work on a pipe as above?

fexecve doesn't read any data from the passed file descriptor, it just
tries to execute the file it refers to. And on its own, 'a pipe' is just
an IPC endpoint --- it doesn't contain anything which could be
interpreted as program.

[Volker Birk]

Kenny McCormack <gaz...@shell.xmission.com> wrote:
> Now, not that this particularly surprises me - and there could be any
> number of reasons why it doesn't work - but I'm curious. Is there anything
> I can add to lgo.c to make it work on a pipe as above?

Your file descriptor does not point to anything which has the x bit set.
I never tried, but you could try to execute fchmod(2) on the file
descriptor first setting S_IXUSR.

Yours,
VB.
--
“Vor Snowden war das ein Blog mit Verschwörungstheorien.
Nach Snowden ist das ein Security-Newsticker.
Bei unverändertem Inhalt...”
Marc Stibane über Fefes Blog

[Rainer Weikusat]

Volker Birk <bum...@dingens.org> writes:
> Kenny McCormack <gaz...@shell.xmission.com> wrote:
>> Now, not that this particularly surprises me - and there could be any
>> number of reasons why it doesn't work - but I'm curious. Is there anything
>> I can add to lgo.c to make it work on a pipe as above?
>
> Your file descriptor does not point to anything which has the x bit set.
> I never tried, but you could try to execute fchmod(2) on the file
> descriptor first setting S_IXUSR.

That's easy to try but an obviously pointless exercise. Below is the
helper function the Linux execve uses to open a file before execution:

struct file *open_exec(const char *name)
{
struct file *file;
int err;
static const struct open_flags open_exec_flags = {
.open_flag = O_LARGEFILE | O_RDONLY | __FMODE_EXEC,
.acc_mode = MAY_EXEC | MAY_OPEN,
.intent = LOOKUP_OPEN
};

file = do_filp_open(AT_FDCWD, name, &open_exec_flags, LOOKUP_FOLLOW);
if (IS_ERR(file))
goto out;

err = -EACCES;
if (!S_ISREG(file->f_path.dentry->d_inode->i_mode))
goto exit;

if (file->f_path.mnt->mnt_flags & MNT_NOEXEC)
goto exit;

fsnotify_open(file);

err = deny_write_access(file);
if (err)
goto exit;

out:
return file;

exit:
fput(file);
return ERR_PTR(err);
}

Assuming it wouldn't reject the 'non regular file', it would then fail
ETXTBSY because the pipe is writable and even assuming this check wasn't
there, it would hit a brick wall when the execve-proper would try to
mmap the supposedly executable file.

[Joe Pfeiffer]

Interesting! I would have expected the input redirection to have wound
up hiding the input file. With a little playing around with fstat, it
clearly does not:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h> /* Definition of AT_* constants */

#include <stdio.h>
#include <unistd.h>

int main(int argc, char **argv)
{

int stat;
struct stat statbuf;

extern char **environ;
(void) argc;

// let's see if we can perform a stat on FD 0
stat = fstat(0, &statbuf);
if (stat < 0) {
perror("fstat");
return 1;
}
printf("st_dev %d\n", statbuf.st_dev);
printf("st_ino %d\n", statbuf.st_ino);
printf("st_mode %o\n", statbuf.st_mode);
printf("st_nlink %d\n", statbuf.st_nlink);
printf("st_uid %d\n", statbuf.st_uid);
printf("st_gid %d\n", statbuf.st_gid);
printf("st_rdev %d\n", statbuf.st_rdev);
printf("st_size %d\n", statbuf.st_size);
printf("st_blksize %d\n", statbuf.st_blksize);
printf("st_blocks %d\n", statbuf.st_blocks);

fexecve(0,argv,environ);
perror("fexecve");
return 1;
}

snowball:588$ bogus < hello
st_dev 38
st_ino 22820087
st_mode 100755
st_nlink 1
st_uid 1248
st_gid 1005
st_rdev 0
st_size 6695
st_blksize 4096
st_blocks 16
Hello World
snowball:589$

[Kaz Kylheku]

On 2014-06-05, Kenny McCormack <gaz...@shell.xmission.com> wrote:
> % echo 'main() { printf("sizeof(int) = %d\\n",sizeof(int)); }' | gcc -xc - -o /dev/stdout | ./lgo
> fexecve: Permission denied

This is almost certainly because the output is a pipe, and not a file.

The exec system call requires a real file.

The reason for that would be that executables are mapped into memory
using mmap; they are not sequentially read and buffered.

><stdin>: In function 'main':
><stdin>:1:10: warning: incompatible implicit declaration of built-in function 'printf' [enabled by default]
><stdin>:1:1: warning: format '%d' expects argument of type 'int', but argument 2 has type 'long unsigned int' [-Wformat]
> %
>
> Now, not that this particularly surprises me - and there could be any
> number of reasons why it doesn't work - but I'm curious. Is there anything
> I can add to lgo.c to make it work on a pipe as above?

No. Obviously, reading stdin and putting it into a temporary file
does not count as "make it work on a pipe".

You can do the kernel work to support the execution of pipes.

A possible way would be to support mmap on a pipe. This is not inconceivable.
It would simply buffer the contents of the pipe from byte 0 as far as it
has been accessed, and map to virtual memory.

The mapping would obviously be like MAP_ANON; it cannot be "backed" by the pipe.

[Volker Birk]

Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> Volker Birk <bum...@dingens.org> writes:
>> Kenny McCormack <gaz...@shell.xmission.com> wrote:
>>> Now, not that this particularly surprises me - and there could be any
>>> number of reasons why it doesn't work - but I'm curious. Is there anything
>>> I can add to lgo.c to make it work on a pipe as above?
>> Your file descriptor does not point to anything which has the x bit set.
>> I never tried, but you could try to execute fchmod(2) on the file
>> descriptor first setting S_IXUSR.
> That's easy to try but an obviously pointless exercise.

My remark was targeting the error message Kenny got from the system:

| fexecve: Permission denied

> helper function the Linux execve uses to open a file before execution:

We're talking about fexecve(2) of POSIX.1-2008, don't we? From there:

<http://pubs.opengroup.org/onlinepubs/9699919799/>
| The exec functions, except for fexecve(), shall fail if:
|
| [EACCES]
| Search permission is denied for a directory listed in the new
| process image file's path prefix, or the new process image file denies
| execution permission.
| […]
| The fexecve() function shall fail if:
|
| [EBADF]
| The fd argument is not a valid file descriptor open for executing.

“Obvious” is the workaround to write data from a pipe to a temp file and
chmod(2) that for having S_IXUSR – or just using a temp file in the
first place.

[Rainer Weikusat]

Volker Birk <bum...@dingens.org> writes:
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> Volker Birk <bum...@dingens.org> writes:
>>> Kenny McCormack <gaz...@shell.xmission.com> wrote:
>>>> Now, not that this particularly surprises me - and there could be any
>>>> number of reasons why it doesn't work - but I'm curious. Is there anything
>>>> I can add to lgo.c to make it work on a pipe as above?
>>> Your file descriptor does not point to anything which has the x bit set.
>>> I never tried, but you could try to execute fchmod(2) on the file
>>> descriptor first setting S_IXUSR.
>> That's easy to try but an obviously pointless exercise.
>
> My remark was targeting the error message Kenny got from the system:
>
> | fexecve: Permission denied
>
>> helper function the Linux execve uses to open a file before execution:
>
> We're talking about fexecve(2) of POSIX.1-2008, don't we?

For practical purposes, that's almost certainly going to be 'the Linux
fexecve' (which is a C library wrapper doing an execve with /proc/fd
filename) and 'POSIX' cannot execute a pipe, either: That corresponds
with a temporarily allocated memory buffer in the kernel of some fixed,
relatively small size (PIPE_BUF POSIX default is 4096 bytes) and even if
the exectuable in question was small enough to fit into it before the
writing application is blocked, it still couldn't be mmapped for
execution like a regular file.

The whole idea is no more sensible than trying to 'execute' a TCP
socket.

[Rainer Weikusat]

[example program]

How could it possibly do that? 'File descriptor 0' is not special to the
kernel and somewhat simplified, the shell just performs a

close(0);
open("/some/path/to/something", O_RDNLY, 0);

in order to accomplish the redirection.

[Rainer Weikusat]

Kaz Kylheku <k...@kylheku.com> writes:
> On 2014-06-05, Kenny McCormack <gaz...@shell.xmission.com> wrote:
>> % echo 'main() { printf("sizeof(int) = %d\\n",sizeof(int)); }' | gcc -xc - -o /dev/stdout | ./lgo
>> fexecve: Permission denied

[...]

> A possible way would be to support mmap on a pipe. This is not inconceivable.
> It would simply buffer the contents of the pipe from byte 0 as far as it
> has been accessed, and map to virtual memory.

Conceptually, a pipe is an unidirectional byte-stream and there is no
way to execute a bytestream as there's always just 'the current byte',
all previous bytes are gone from the pipe and all future bytes are not
yet accessible. Granted, that's not how implementations usually work and
pipes could be implemented on top of some kind of 'shared memory
filesystem' providing regular files residing in kernel memory where a
writing application could copy 'all of the input data' into kernel
memory atomically and the kernel could then do an execve once it sees an
input EOF on the write descriptor but that would bascially mean that the
kernel would provide an implicit 'temporary file' just to enable
'execution of pipes', a rather bizarre idea.

[Barry Margolin]

In article <87ppinw...@sable.mobileactivedefense.com>,

Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:

> The whole idea is no more sensible than trying to 'execute' a TCP
> socket.

If you really wanted to do this kind of thing, I suppose you could
implement a FUSE filesystem that supports it. After all, FUSE allows you
to execute a file on an SSH or FTP server.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Rainer Weikusat]

Barry Margolin <bar...@alum.mit.edu> writes:
> In article <87ppinw...@sable.mobileactivedefense.com>,
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>
>> The whole idea is no more sensible than trying to 'execute' a TCP
>> socket.
>
> If you really wanted to do this kind of thing, I suppose you could
> implement a FUSE filesystem that supports it. After all, FUSE allows you
> to execute a file on an SSH or FTP server.

The kernel also allows execution of files residing on 'other computers'
accessible by talking to call kinds of 'other servers', eg, NFS, CIFS,
AFS and $whatnot, not to mention 'SCSI servers connected to a USB' (USB
is a LAN), that's just a matter of integrating a suitable 'protocol
module' with the page/ buffer cache. But pipes/ FIFOs (or TCP sockets or
ptys[*]) are not 'integrated with the page cache' by default, they're
all abstractions for 'byte-at-a-time IPC' (I'd gladly use a better term
for that if I knew one ...) and thus, fundamentally incompatible with
'random access files'. In contrast to that, FTP is a protocol providing
'random access to files' residing on some filesystem on a different
computer which uses TCP for transport.

[*] Having someone typewrite an executable to the kernel for direct
execution sounds like an interesting idea ...

[Kaz Kylheku]

On 2014-06-05, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> The whole idea is no more sensible than trying to 'execute' a TCP
> socket.

It sure is a lot simpler than making the poor end user on the other end have to
find a buffer overflow where to inject his executable code.

[Kaz Kylheku]

On 2014-06-05, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:

> Kaz Kylheku <k...@kylheku.com> writes:
>> On 2014-06-05, Kenny McCormack <gaz...@shell.xmission.com> wrote:
>>> % echo 'main() { printf("sizeof(int) = %d\\n",sizeof(int)); }' | gcc -xc - -o /dev/stdout | ./lgo
>>> fexecve: Permission denied
>
> [...]
>
>> A possible way would be to support mmap on a pipe. This is not inconceivable.
>> It would simply buffer the contents of the pipe from byte 0 as far as it
>> has been accessed, and map to virtual memory.
>
> Conceptually, a pipe is an unidirectional byte-stream and there is no
> way to execute a bytestream as there's always just 'the current byte',

This is true, but in addition to the current byte there is the historic
bytes that came before the current byte.

> all previous bytes are gone from the pipe and all future bytes are not
> yet accessible.

When you read a random access file using a 512 byte buffer, and lseek all over
the place between reads, all the 512 byte blocks you ever read before are also
gone.

Yet a memory mapping can integrate the pages that are touched and keep them
persistent.

> Granted, that's not how implementations usually work and
> pipes could be implemented on top of some kind of 'shared memory
> filesystem' providing regular files residing in kernel memory where a

All you need is to implement the mmap VFS method for pipes. As the data
is arriving, allocate pages for it and map them.

The restriction could be (or would have to be) imposed that the mapping must be
arranged prior to the first page being read.

Like say you've read 500 megs of data from a pipe and suddenly get the crazy
idea to mmap it. Of course, the 500 megs is gone. A pipe can't hoard all the
data just in case someone might mmap it. (Or, maybe it can: crazy idea #2: have
an O_MMAP_PIPE flag to request this, which can be passed into pipe2).

From the time a mmap is made, any data which lands into the mapped range
can be stashed away by the pipe, and incorporated to the mapping.

Alignment is an issue. Suppose you've read 41 bytes from a pipe. Do we say
that those bytes 0-40 are "water under the bridge", and the new mmap now
being requested aligns byte 41 on the start of a page?

It doesn't seem reasonable to enforce pager alignment based on absolute stream
position.

> writing application could copy 'all of the input data' into kernel
> memory atomically and the kernel could then do an execve once it sees an
> input EOF on the write descriptor but that would bascially mean that the
> kernel would provide an implicit 'temporary file' just to enable
> 'execution of pipes', a rather bizarre idea.

The exec does not have to block until an EOF is seen. Basically, the pipe
is mapped upfront. The exec format handler will touch some early part of the
mapping for the ELF header or whatever, and so it is blocked until those pages
are read from the pipe and integrated into the mapping. Then it will arrange
for users space to branch to some location in the executable, so again, that
has to block until enough of the executable is mapped to cover that page, and
so on. Something like that.

The pipe does not have to be read through to EOF, if some tail part of it is
never touched during execution.

I have a pretty clear idea on how this would work; except that it would
be the laughing stock of kernel.org.

[Volker Birk]

Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> Volker Birk <bum...@dingens.org> writes:
>> We're talking about fexecve(2) of POSIX.1-2008, don't we?
> For practical purposes, that's almost certainly going to be 'the Linux
> fexecve'

So Solaris does not exist in your small world?

[Rainer Weikusat]

Volker Birk <bum...@dingens.org> writes:
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> Volker Birk <bum...@dingens.org> writes:
>>> We're talking about fexecve(2) of POSIX.1-2008, don't we?
>> For practical purposes, that's almost certainly going to be 'the Linux
>> fexecve'
>
> So Solaris does not exist in your small world?

I doubt that Kenny was testing this on Solaris and "in the small [but
surely very profitable] world of Oracle", fexecve/ execve is documented
to return an EACCES error if

The new process file is not an ordinary file.
http://docs.oracle.com/cd/E23824_01/html/821-1463/exec-2.html

which is the exactly the same as the behaviour of Linux for this case.

BTW, you seem to be moving away from the topic again and towards
"discussing me". Since I'm generally tired of this, that's the end of
this 'subdiscussion' for me.

[Kaz Kylheku]

On 2014-06-05, Volker Birk <bum...@dingens.org> wrote:
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> Volker Birk <bum...@dingens.org> writes:
>>> We're talking about fexecve(2) of POSIX.1-2008, don't we?
>> For practical purposes, that's almost certainly going to be 'the Linux
>> fexecve'
>
> So Solaris does not exist in your small world?

LOL. I have version 10 of that running in a VirtualBox.
Good grief, step back into the 1980's --- and not the cool
Anthrax-and-Judas-Priest 1980's either.

POSIX.1-2008?

The /bin/sh doesn't even understand $(command substitution),
and /usr/xpg4/bin/sh fails on "$@" when there are no arguments;
you have to use the ${@+"$@"} trick.

POSIX clearly says "If there are no positional parameters, the expansion of
'@' shall generate zero fields, even when '@' is double-quoted."
and this requirement is years old.

Most people would rather have things like that fixed, than fexecve.

The only thing that makes Solaris tolerable is www.opencsw.org.

I installed a bunch of that, and then did this:

# cd /opt/csw/bin
# for x in g* ; ln -s $x ${x#g} ; done # few false positives here, who cares
# PATH=/opt/csw/bin:$PATH # look for non-broken utils first

I'm going to try removing /bin/sh and replacing it with bash to see what
breaks, if anything.

[Volker Birk]

Kaz Kylheku <k...@kylheku.com> wrote:
> I'm going to try removing /bin/sh and replacing it with bash to see what
> breaks, if anything.

Or with dash?

VB, blaspheming ;-)

[Casper H.S. Dik]

Kaz Kylheku <k...@kylheku.com> writes:

>LOL. I have version 10 of that running in a VirtualBox.
>Good grief, step back into the 1980's --- and not the cool
>Anthrax-and-Judas-Priest 1980's either.

A 10 year old OS wouldn't likely to be feel "modern".

>POSIX.1-2008?

>The /bin/sh doesn't even understand $(command substitution),
>and /usr/xpg4/bin/sh fails on "$@" when there are no arguments;
>you have to use the ${@+"$@"} trick.

There is nothing in the standard says what the name of the POSIX
shell is /bin/sh; it is /usr/xpg4/bin/sh

Your test method must have failed as both /bin/sh and /usr/xpg4/bin/sh
work fine when confronted with "$@" when no arguments have been passed.

I think, but I can no longer verify that, that even SunOS 4.x did
this properly.

>POSIX clearly says "If there are no positional parameters, the expansion of
>'@' shall generate zero fields, even when '@' is double-quoted."
>and this requirement is years old.

And Solaris does do it properly but you shouldn't be able test that.

>Most people would rather have things like that fixed, than fexecve.

>The only thing that makes Solaris tolerable is www.opencsw.org.

>I installed a bunch of that, and then did this:

> # cd /opt/csw/bin
> # for x in g* ; ln -s $x ${x#g} ; done # few false positives here, who cares
> # PATH=/opt/csw/bin:$PATH # look for non-broken utils first

>I'm going to try removing /bin/sh and replacing it with bash to see what
>breaks, if anything.

So why not use a more modern version of Solaris?

In Solaris 11, you get a much more recent /bin/sh. (Some version of
ksh93)

Casper

[Rainer Weikusat]

Kaz Kylheku <k...@kylheku.com> writes:
> On 2014-06-05, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> Kaz Kylheku <k...@kylheku.com> writes:
>>> On 2014-06-05, Kenny McCormack <gaz...@shell.xmission.com> wrote:
>>>> % echo 'main() { printf("sizeof(int) = %d\\n",sizeof(int)); }' | gcc -xc - -o /dev/stdout | ./lgo
>>>> fexecve: Permission denied
>>
>> [...]
>>
>>> A possible way would be to support mmap on a pipe. This is not inconceivable.
>>> It would simply buffer the contents of the pipe from byte 0 as far as it
>>> has been accessed, and map to virtual memory.
>>
>> Conceptually, a pipe is an unidirectional byte-stream and there is no
>> way to execute a bytestream as there's always just 'the current byte',
>
> This is true, but in addition to the current byte there is the historic
> bytes that came before the current byte.

Only if it is buffered outside of the pipe.


[...]

>> Granted, that's not how implementations usually work and
>> pipes could be implemented on top of some kind of 'shared memory
>> filesystem' providing regular files residing in kernel memory where a
>
> All you need is to implement the mmap VFS method for pipes. As the data
> is arriving, allocate pages for it and map them.
>
> The restriction could be (or would have to be) imposed that the mapping must be
> arranged prior to the first page being read.
>
> Like say you've read 500 megs of data from a pipe and suddenly get the crazy
> idea to mmap it. Of course, the 500 megs is gone. A pipe can't hoard all the
> data just in case someone might mmap it. (Or, maybe it can: crazy idea #2: have
> an O_MMAP_PIPE flag to request this, which can be passed into pipe2).

It is doubtlessly possible to change the/ a pipe implementation such
that a pipe can functions as something other than a IPC channel,
however, why would this then still be called 'a pipe'?

The original program didn't really use the pipe to pass data from one
process to another process but really just passed a file descriptor
referring to the pipe to the other process which then passed it into the
kernel (it actually passed something the kernel could use to locate the
corresponding i-node, but this doesn't really matter), ie, except that
some data generated by the program sending program was likely copied
into a kernel buffer associated with the pipe, nothing ever read 'the
sent program' (and hence, it can't possibly be executed).

[...]

>> writing application could copy 'all of the input data' into kernel
>> memory atomically and the kernel could then do an execve once it sees an
>> input EOF on the write descriptor but that would bascially mean that the
>> kernel would provide an implicit 'temporary file' just to enable
>> 'execution of pipes', a rather bizarre idea.
>
> The exec does not have to block until an EOF is seen. Basically, the pipe
> is mapped upfront. The exec format handler will touch some early part of the
> mapping for the ELF header or whatever, and so it is blocked until those pages
> are read from the pipe and integrated into the mapping. Then it will arrange
> for users space to branch to some location in the executable, so again, that
> has to block until enough of the executable is mapped to cover that page, and
> so on. Something like that.
>
> The pipe does not have to be read through to EOF, if some tail part of it is
> never touched during execution.

There are all kinds of amusing corner cases. Two I thought of so far:

- Assume a daemon process is 'sent via pipe to the kernel for
execution' in this way and some of the data is never
read. This would then block the sending process forever. What
if somebody kills it?

- The sent application could try to eat itself by reading from
its standard input until EOF

[Scott Lurndal]

Why not just use ksh instead of /bin/sh?

[Gordon Burditt]

>> fexecve: Permission denied
>
> This is almost certainly because the output is a pipe, and not a file.
>
> The exec system call requires a real file.

I think you're trying to say that it requires a *SEEKABLE* file.
That would also exclude certain old tape devices which don't have
a fixed blocksize (as well as punched-card readers, paper tape
readers, and TCP/IP connections).

If you can support fexecve() on pipes, you can probably also support
it on punched-card readers and TCP/IP connections. Seems like an
excellent feature to support virus writers.

[Barry Margolin]

In article <TqOdnQkORv8oDA_O...@posted.internetamerica>,

Gordon Burditt <gor...@hammy.burditt.org> wrote:

> If you can support fexecve() on pipes, you can probably also support
> it on punched-card readers and TCP/IP connections. Seems like an
> excellent feature to support virus writers.

That would be nice. I'm really getting tired of having to copy my punch
card decks into temporary files so I can run them.

DECtape is seekable, maybe I'll transfer my cards to that.

[Rainer Weikusat]

As Kaz Kylheku already remarked, the kernel can't really "execute files"
either. It can get the CPU to execute code which has been loaded into
memory and since regular files are (also) intended to be used to store
programs, the necessary mechanisms for doing so already exist. A
'seekable' backing store mechanism for programs can also support to
other useful features, namely, that code can be loaded on demand and
that code which isn't currently execute can be discarded to free memory
since it can always be demand-loaded from the correponding file again.

The original example was essentially a sham because it was composed of a
program set up to write some other program to a FIFO and another program
which had access to the read end of this FIFO which didn't ever read
anything from it but used the FIFO file descriptor as argument for an
fexecve call. This is obvious nonsense because the kernel doesn't read
from this FIFO, either, as that is an IPC 'device' usually supposed to be
utilized by applications who want to exchange data accross an address
space boundary with the help of the kernel which can make this happen by
providing 'controlled' access to a kernel memory buffer to both
applications, and not some facility for copying an arbitrarily and
possibly, even infinitively large amount of data into kernel memory.

[Kenny McCormack]

In article <87k38re...@sable.mobileactivedefense.com>,
Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
...

>The original example was essentially a sham because it was composed of a

Which of these definitions of 'sham' do you think the example exemplifies?


sham [sham] Show IPA
noun
1. something that is not what it purports to be; a spurious imitation;
fraud or hoax.
2. a person who shams; shammer.
3. a cover or the like for giving a thing a different outward appearance:
a pillow sham.

--
The scent of awk programmers is a lot more attractive to women than
the scent of perl programmers.

(Mike Brennan, quoted in the "GAWK" manual)

[Rainer Weikusat]

gaz...@shell.xmission.com (Kenny McCormack) writes:
> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> ...
>>The original example was essentially a sham because it was composed of a
>
> Which of these definitions of 'sham' do you think the example exemplifies?
>
>
> sham [sham] Show IPA
> noun
> 1. something that is not what it purports to be; a spurious imitation;
> fraud or hoax.

Obviously this one, as explained in the text I wrote: A pipe was set up
and a process writing some data to it was started but the process on the
other end of the pipe never read anything. Hence, no data was ever
passed over the pipe and even if this had been different, the data had
now existed somewhere in the address space of the process connected to
the reading end of the pipe and the pipe file descriptor couldn't
possibly be used to access it anymore.

[Kaz Kylheku]

On 2014-06-08, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> gaz...@shell.xmission.com (Kenny McCormack) writes:
>> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> ...
>>>The original example was essentially a sham because it was composed of a
>>
>> Which of these definitions of 'sham' do you think the example exemplifies?
>>
>>
>> sham [sham] Show IPA
>> noun
>> 1. something that is not what it purports to be; a spurious imitation;
>> fraud or hoax.
>
> Obviously this one, as explained in the text I wrote: A pipe was set up
> and a process writing some data to it was started but the process on the
> other end of the pipe never read anything. Hence, no data was ever

The process doing an execvp() also never reads anything, although opening
a file is involved.

[Barry Margolin]

In article <201406081...@kylheku.com>,

Right. fexecvp() implicitly reads the file. So there's no inherent
reason why it couldn't implicitly read the pipe.

The simple answer is that no one saw the need to allow exec'ing anything
but ordinary files. Operating systems long had the notion of
memory-mapping files, and this is used for executing files and dynamic
linking. All the extra work to execute a pipe requires resurring the
concepts from the old days where code was copied from files into memory
rather than being mapped.

[Rainer Weikusat]

There's no reason why it would have to do so because a regular file has
content which exists independently of any I/O operation performed on
it. A pipe (or any other kind of 'serial something stream IPC
channel') hasn't: While there's a (fairly) small kernel buffer
associated with it, this is 'an implementation optimization' and the
kernel could as well block a writer until a reader tries to get some
data and copy it directly from one address space to the other.

The idea to 'execute a pipe' isn't really different from the idea to
connect a microphone to a set of speakers in order to 'play' a song
nobody is singing: The result will be silence/ static noise because the
microphone doesn't record content, it just transports something 'put
into it' in realtime: If there's no input, nothing will ever come out of
it. A better analogy would be to give someone a microphone and request
him to sing something but without connecting it to anything: Strangely,
no amplified output will happen beause the data isn't really going
anywhere. Even assuming that everything happens to be connected, it is
not possible to jump five minutes back in time in order to listen to
something again by pressing some button because the microphone connected
to some amplifier and speakers doesn't preserve the data which is sent
through it. This generally requires creating a recording of some sort
and devices which do so exist.

[Rainer Weikusat]

Barry Margolin <bar...@alum.mit.edu> writes:
> In article <201406081...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2014-06-08, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> > gaz...@shell.xmission.com (Kenny McCormack) writes:
>> >> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> >> ...
>> >>>The original example was essentially a sham because it was composed of a
>> >>
>> >> Which of these definitions of 'sham' do you think the example exemplifies?
>> >>
>> >>
>> >> sham [sham] Show IPA
>> >> noun
>> >> 1. something that is not what it purports to be; a spurious imitation;
>> >> fraud or hoax.
>> >
>> > Obviously this one, as explained in the text I wrote: A pipe was set up
>> > and a process writing some data to it was started but the process on the
>> > other end of the pipe never read anything. Hence, no data was ever
>>
>> The process doing an execvp() also never reads anything, although opening
>> a file is involved.
>
> Right. fexecvp() implicitly reads the file. So there's no inherent
> reason why it couldn't implicitly read the pipe.

fexecvp does not 'implictly read the file': It arranges for the
_recorded_ content to be made available as needed in order to execute
the program. But a pipe has no recorded content, hence, this cannot
possibly be done with it. It is possible to create a trivial 'execution
server' which reads data from a pipe, records it, and executes this
recorded content. It is obviously also possible to to extend the kernel
to create a permanently allocated, that is, leaked, in-kernel memory
buffer containing a copy of the data written to the pipe and enable
execution of this seriously functionally impaired sort-of 'mini
filesystem'. But that still won't 'execute the pipe' but enable and
application to copy an principally unbounded amount of data to kernel
memory for possible execution (say, in the event the application stops
writing data before the system runs out of memory).]

> The simple answer is that no one saw the need to allow exec'ing anything
> but ordinary files. Operating systems long had the notion of
> memory-mapping files, and this is used for executing files and dynamic
> linking. All the extra work to execute a pipe requires resurring the
> concepts from the old days where code was copied from files into memory
> rather than being mapped.

'Being mapped' just means it is copied on demand by the page fault
handler.

[Volker Birk]

Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
> fexecvp does not 'implictly read the file': It arranges for the
> _recorded_ content to be made available as needed in order to execute
> the program.

(Assuming we're talking about fexecve():) I see nothing in POSIX what
leads to that conclusion. Can you help me there?

As far as I'm reading the spec, fexecve() can be implemented like Linux
and Solaris do, but it would be standards compliant to implement it for
pipes, too. From the Linux man page (as you seem to prefer that):

| The file descriptor fd must be opened read-only, and the caller must
| have permission to execute the file that it refers to.

The second constraint includes the necessity to do fchmod() on pipes.
A possibility to have an x bit set is already implemented for FIFOs,
though.

Yours,
VB.

[Rainer Weikusat]

Barry Margolin <bar...@alum.mit.edu> writes:
> In article <201406081...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2014-06-08, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> > gaz...@shell.xmission.com (Kenny McCormack) writes:
>> >> Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:
>> >> ...
>> >>>The original example was essentially a sham because it was composed of a
>> >>
>> >> Which of these definitions of 'sham' do you think the example exemplifies?
>> >>
>> >>
>> >> sham [sham] Show IPA
>> >> noun
>> >> 1. something that is not what it purports to be; a spurious imitation;
>> >> fraud or hoax.
>> >
>> > Obviously this one, as explained in the text I wrote: A pipe was set up
>> > and a process writing some data to it was started but the process on the
>> > other end of the pipe never read anything. Hence, no data was ever
>>
>> The process doing an execvp() also never reads anything, although opening
>> a file is involved.
>
> Right. fexecvp() implicitly reads the file. So there's no inherent
> reason why it couldn't implicitly read the pipe.
>
> The simple answer is that no one saw the need to allow exec'ing anything
> but ordinary files.

I think this is basically turned upside down: Pipes exist because
somebody saw a reason for an I/O facility enabling real-time
communication among cooperating applications in addition to the already
existing facilites for 'recording data for possible, future use'. That's
consistent with the fact that they didn't appear in UNIX(*) until 1972
and were basically forced into the system by the lone person who
understood the difference and was concvinced that it would enable useful
things not hitherto possible (that's my understanding of the 'Pipes'
section of http://cm.bell-labs.com/cm/cs/who/dmr/hist.html).

Another wild analogy I came up with in the meantime: Shouldn't it be
possible to use a lawnmower as single person helicopter? After all, both
lawnmowers and helicopters have a motor and this horizontally mounted,
rotating thing. Granted, the helicopter has it on the top and the
lawnmower on the bottom but that's easily fixed by turning the lawnmower
around. Any remaining difficulties can surely be overcome by Gyro
Gearloose in no time!

[Rainer Weikusat]

Rainer Weikusat <rwei...@mobileactivedefense.com> writes:
> Kaz Kylheku <k...@kylheku.com> writes:
>> On 2014-06-08, Rainer Weikusat <rwei...@mobileactivedefense.com> wrote:

[...]

>>> A pipe was set up
>>> and a process writing some data to it was started but the process on the
>>> other end of the pipe never read anything. Hence, no data was ever
>>
>> The process doing an execvp() also never reads anything, although opening
>> a file is involved.
>
> There's no reason why it would have to do so because a regular file has
> content which exists independently of any I/O operation performed on
> it. A pipe (or any other kind of 'serial something stream IPC
> channel') hasn't: While there's a (fairly) small kernel buffer
> associated with it, this is 'an implementation optimization' and the
> kernel could as well block a writer until a reader tries to get some
> data and copy it directly from one address space to the other.

Quote from a paper titled "The Unix Time-Sharing System" which apparently
came together with 6th ed UNIX:

5.2 Pipes

Processes may communicate with related processes using the same
system read and write calls that are used for file system
I/O. The call

filep = pipe ( )

returns a file descriptor filep and creates an inter-process
channel called a pipe. This channel, like other open files,
is passed from parent to child process in the image by the
fork call.A read using a pipe file descriptor waits until
another process writes using the file descriptor for the same
pipe. At this point, data are passed between the images of
the two processes.
[p.8, http://wwwlehre.dhbw-stuttgart.de/~helbig/os/v6/doc/unix.ps]

'Image' refers to data necessary for describing a process
'frozen in time', especially the memory areas used by it. And this pipe
is really just a mechanism for performing a synchronous block memory
copy from the 'core image' of one process to the 'core image' of another
without requiring either process to be aware of the innards of the
other.