[RfC] (fd)open files on other layers than the default layer.
Juergen Boemmels
currently there is no (simple) way to open a file on an other layer
than the default layer. But this is necessary if we want to take
advantage from the layered approach.
So i added two new functions: pio_open_with_layer and
pio_fdopen_with_layer which create IO-Objects on different layers. I'm
not keen on the names, but I didn't find better ones.
So to test this I need another layer on which I can open files: stdio
should be available on all systems. So far so good, now I can open
files on with stdio and fdopen existing handles.
But there are some issues:
* During layer initialisation the layers want to register the handles
for stdin, stdout and stderr. I went around this by conditionally
only compiling the handle init if stdio is the base IO systems.
Is this a good solution?
* I needed many casts from PIOHANDLE to FILE * and vice versa. I'm not
sure if this one fits all approach of PIOHANDLE is the right way.
Maybe its better to make PIOHANDLE a union. But what to do then
with the fdopen and getfd ops. It is not guaranteed that a union
can be casted to an INTVAL and back.
* Modifying the same file through different layers might lead to very
unpredictable results. But thats already a problem with
unix-handles/stdio-handles.
* PIO_isatty and PIO_getblksize are macros which are coded at
compile-time to some layer, and not as read, write etc go through a
LayerAPI.
Comments
bö
Melvin Smith
Here are a couple comments,
At 03:18 PM 11/4/2003 +0100, Juergen Boemmels wrote:
>currently there is no (simple) way to open a file on an other layer
>than the default layer. But this is necessary if we want to take
>advantage from the layered approach.
>
>So i added two new functions: pio_open_with_layer and
>pio_fdopen_with_layer which create IO-Objects on different layers. I'm
>not keen on the names, but I didn't find better ones.
An alternative is to add an optional layer parameter to PIO_open and company.
If it is null you can use the default.
>But there are some issues:
>* During layer initialisation the layers want to register the handles
> for stdin, stdout and stderr. I went around this by conditionally
> only compiling the handle init if stdio is the base IO systems.
> Is this a good solution?
It is probably time to have the layer specific init (PIO_unit_init, etc.)
pass back an array of standard handles rather than explicilty
setting them into the interpreter as it does now? This will allow
us to ignore them at the top level if need be. You are right,
currently it isn't really easy to mix "terminal" layers because of this.
>* I needed many casts from PIOHANDLE to FILE * and vice versa. I'm not
> sure if this one fits all approach of PIOHANDLE is the right way.
> Maybe its better to make PIOHANDLE a union. But what to do then
> with the fdopen and getfd ops. It is not guaranteed that a union
> can be casted to an INTVAL and back.
I think it works with casts, unless there are cases where our INTVAL
cannot hold a pointer. There are currently 2 cases:
PIOHANDLE is a low level OS handle (UNIX or Win32) and we to
create a ParrotIO around it.
PIOHANDLE is a stdio FILE * and we need to create a ParrotIO around it.
I'm really ok if the io_stdio.c is a mega hack since STDIO is only there
for convenience and bootstrap where there is no other IO alternative.
In either case, it is the bytecode op that is the unsure part, and whether
we have to use the INTVAL type or we have to change it to a STRING buffer.
>* Modifying the same file through different layers might lead to very
> unpredictable results. But thats already a problem with
> unix-handles/stdio-handles.
What do you mean by modifying here?
>* PIO_isatty and PIO_getblksize are macros which are coded at
> compile-time to some layer, and not as read, write etc go through a
> LayerAPI.
Those need to be changed as well. They break the whole layer plan,
as you point out.
Actually isatty() can be called inside PIO_fdopen and the result
stored into the PIO header as a bit flag. Then PIO_isatty() can simply
be a flag check against the PIO.
Finally, the rest of the compile time macros definitely need to go away
since they make it impossible to use 2 terminal layers at the same time.
-Melvin
Melvin Smith
>At 03:18 PM 11/4/2003 +0100, Juergen Boemmels wrote:
>>* I needed many casts from PIOHANDLE to FILE * and vice versa. I'm not
>> sure if this one fits all approach of PIOHANDLE is the right way.
>> Maybe its better to make PIOHANDLE a union. But what to do then
>> with the fdopen and getfd ops. It is not guaranteed that a union
>> can be casted to an INTVAL and back.
>
>I think it works with casts, unless there are cases where our INTVAL
>cannot hold a pointer. There are currently 2 cases:
>
>PIOHANDLE is a low level OS handle (UNIX or Win32) and we to
> create a ParrotIO around it.
>PIOHANDLE is a stdio FILE * and we need to create a ParrotIO around it.
One more comment. I don't even see the possibility of calling fdopen
from the bytecode level with a FILE *. From the bytecode level, either
we know some OS constants that are typically streams (UNIX 0,1,2)
or the call probably won't come from the bytecode. For the STDIO
case, there is currently no way to even call fdopen() from bytecode
with a valid parameter, it only works for the low level OS handles.
It may not make sense to expose it, but I don't see how we can do without
exposing it for UNIX and low level Win32 without exposing it for STDIO.
1 possible solution is: remove PIO_stdio_fdopen altogether and move the code
to PIO_stdio_unit() which actually needs it.
-Melvin
Juergen Boemmels
> Hey Juergen,
>
> Here are a couple comments,
>
> At 03:18 PM 11/4/2003 +0100, Juergen Boemmels wrote:
> >currently there is no (simple) way to open a file on an other layer
> >than the default layer. But this is necessary if we want to take
> >advantage from the layered approach.
> >
> >So i added two new functions: pio_open_with_layer and
> >pio_fdopen_with_layer which create IO-Objects on different layers. I'm
> >not keen on the names, but I didn't find better ones.
>
> An alternative is to add an optional layer parameter to PIO_open and company.
> If it is null you can use the default.
Ok, but this means that all uses of PIO_open needs to be changed
too. But greping through the source, they are not that many.
> >But there are some issues:
> >* During layer initialisation the layers want to register the handles
> > for stdin, stdout and stderr. I went around this by conditionally
> > only compiling the handle init if stdio is the base IO systems.
> > Is this a good solution?
>
> It is probably time to have the layer specific init (PIO_unit_init, etc.)
> pass back an array of standard handles rather than explicilty
> setting them into the interpreter as it does now? This will allow
> us to ignore them at the top level if need be. You are right,
> currently it isn't really easy to mix "terminal" layers because of this.
Yeah, might work. It can be an garbage collected array, there is
already a PIO_data_mark.
> >* I needed many casts from PIOHANDLE to FILE * and vice versa. I'm not
> > sure if this one fits all approach of PIOHANDLE is the right way.
> > Maybe its better to make PIOHANDLE a union. But what to do then
> > with the fdopen and getfd ops. It is not guaranteed that a union
> > can be casted to an INTVAL and back.
>
> I think it works with casts, unless there are cases where our INTVAL
> cannot hold a pointer. There are currently 2 cases:
>
> PIOHANDLE is a low level OS handle (UNIX or Win32) and we to
> create a ParrotIO around it.
> PIOHANDLE is a stdio FILE * and we need to create a ParrotIO around it.
>
> I'm really ok if the io_stdio.c is a mega hack since STDIO is only there
> for convenience and bootstrap where there is no other IO alternative.
Is it always guaranteed that an pointer fits in an INTVAL?
> In either case, it is the bytecode op that is the unsure part, and whether
> we have to use the INTVAL type or we have to change it to a STRING buffer.
Or a PMC. Either an PerlInt or an UnmanagedStruct.
> >* Modifying the same file through different layers might lead to very
> > unpredictable results. But thats already a problem with
> > unix-handles/stdio-handles.
>
> What do you mean by modifying here?
Two layers (write-)buffering independently. If one writes but not
flushes, then the other on writes. But that problem is not new:
fwrite("a", 1, 1, stdout);
write(1, "b", 1);
fwrite("c", 1, 1, stdout);
> >* PIO_isatty and PIO_getblksize are macros which are coded at
> > compile-time to some layer, and not as read, write etc go through a
> > LayerAPI.
>
> Those need to be changed as well. They break the whole layer plan,
> as you point out.
>
> Actually isatty() can be called inside PIO_fdopen and the result
> stored into the PIO header as a bit flag. Then PIO_isatty() can simply
> be a flag check against the PIO.
>
> Finally, the rest of the compile time macros definitely need to go away
> since they make it impossible to use 2 terminal layers at the same time.
Will do that.
boe
--
Juergen Boemmels boem...@physik.uni-kl.de
Fachbereich Physik Tel: ++49-(0)631-205-2817
Universitaet Kaiserslautern Fax: ++49-(0)631-205-3906
PGP Key fingerprint = 9F 56 54 3D 45 C1 32 6F 23 F6 C7 2F 85 93 DD 47
Juergen Boemmels
> >PIOHANDLE is a low level OS handle (UNIX or Win32) and we to
> > create a ParrotIO around it.
> >PIOHANDLE is a stdio FILE * and we need to create a ParrotIO around it.
>
> One more comment. I don't even see the possibility of calling fdopen
> from the bytecode level with a FILE *. From the bytecode level, either
> we know some OS constants that are typically streams (UNIX 0,1,2)
> or the call probably won't come from the bytecode. For the STDIO
> case, there is currently no way to even call fdopen() from bytecode
> with a valid parameter, it only works for the low level OS handles.
The standard handles are already special cased in the ParrotIO, so
they should not be a reason for fdopen. The main goal why I wanted to
keep fdopen is in cases were parrot is embedded and the bytecode gets
called with an external FILE * pointer. This might be for example
yyin. But is it needed that this must be an opcode or can it be a
method of some layer-object.
> It may not make sense to expose it, but I don't see how we can do without
> exposing it for UNIX and low level Win32 without exposing it for STDIO.
>
> 1 possible solution is: remove PIO_stdio_fdopen altogether and move the code
> to PIO_stdio_unit() which actually needs it.
Or just set the FDOpen slot in ParrotIOLayerAPI to NULL. The function
PIO_stdio_fdopen is static and should not leak out of io_stdio.c
bye
Melvin Smith
>they should not be a reason for fdopen. The main goal why I wanted to
>keep fdopen is in cases were parrot is embedded and the bytecode gets
>called with an external FILE * pointer. This might be for example
>yyin. But is it needed that this must be an opcode or can it be a
>method of some layer-object.
fdopen does make sense, especially in the UNIX world as it is common
to have a file descriptor that you want to open a standard stream on.
It could be a known constant fd, or one passed by some other API, another
processes, etc.. More obscure cases include execing() yourself without
closing any files, which is possible, but probably not in Parrot
since Parrot would just reload bytecode; no need to exec().
But I don't know whether it makes sense as an opcode since it is
platform dependant.
I am glad you are having a go at it, I've been avoiding the issue. :)
-Melvin
Melvin Smith
> > I think it works with casts, unless there are cases where our INTVAL
> > cannot hold a pointer. There are currently 2 cases:
>
I'm not aware of any platform where size_t cannot hold a pointer,
but I can't speak for INTVAL since I don't remember exactly
how we determine it without going back to the config code.
We could change PIOHANDLE to size_t, leave the bytecode op as INTVAL,
but I'm not sure where that leaves us. I say punt and wait until later
to worry about it. :)
-Melvin