Reading files from device driver.

[kvleonhardt]

Hi

I found another problem writing a VGA driver.

I want to make file access from the driver for multiple purpose.

But when I use the fopen function, following is written in the
logfile:

May 12 10:49:08 minix kernel: VFS: only the PFS device can make nested
VFS calls

I believe that it must be possible to have file access from a device
driver. But how?

Thanks

kvleonhardt

[BeakGwangMoon]

Hi kvleonhardt, I'm moon ; ).

I think OPEN call would be fine not FOPEN.
and have a very testing vesa tty source(minix 3.2.0), and 16 color VGA
source (MINIX 3.1.9)
if you want to, please, mail me.(moonl...@naver.com)

VESA don't use VGA memory, see this site.
http://www.monstersoft.com/tutorial1/VESA_intro.html
http://wiki.osdev.org/Drawing_In_Protected_Mode

i hope it will help.
Thanks

[BeakGwangMoon]

On 5월12일, 오후5시59분, kvleonhardt <kvleonha...@gmail.com> wrote:

Hi, kvleonhardt
I' moon ;)

I think open call can do.

and I have very testing and buggy VESA tty source (MINIX 3.2.0) and
16color VGA source (MINIX 3.1.9)

if you want to, please mail me (moonl...@naver.com)

more info about VESA is here
http://wiki.osdev.org/Drawing_In_Protected_Mode
http://www.monstersoft.com/tutorial1/VESA_intro.html

I hope it help you

[kvleonhardt]

Hi moon

Thanks for trying.

But it did not work. "open" returns -1 when trying, and the kernel

still says:

VFS: only the PFS device can make nested VFS calls

I have no problem using the SVGA mode (only some performence problems
discussed in other thread), but I want my driver to read images files
directly from disk, instead og sending all data through the device
node.



kvleonhardt

[BeakGwangMoon]

Hi kvleonhardt
Happy to hear you found your solution.
I'm sorry confuse you about reading a file in device.
and your code will be helpfull.

I read this server source of Minix 3.2.0
http://git.minix3.org/?p=minix.git;a=blob_plain;f=servers/inet/inet_config.c;hb=HEAD
at line 268, void read_conf(void).

so think that the Driver can read a file by open call .

but device driver in boot image can not because it is in boot image, I
think

moon.

[kvleonhardt]

Hi moon

I don't get it.

The only difference I can see between the inet-server and my VGA-
driver is in fact that inet is a server, and my VGA driver is . . . of
cause a device driver.

I have indeed looked for a device driver with file access, but I have
not found anything useful.

I will try again. Like the problem writing directly to the video
memory, it helped waiting to the next day.


kvleonhardt

[BeakGwangMoon]

Hi kvleonhardt.

In Minix a driver, a server are same.

but the tty driver is in a boot image and the inet server is not.

I think this is very important.

In Minix 3.1.8 a boot image has all in one file.
I hope this doc help understanding boot image.

http://wiki.minix3.org/en/DevelopersGuide/VmInternals

and this thread
http://groups.google.com/group/minix3/browse_thread/thread/c21a65c577fe1d1f

moon.
Thanks for your reading.

p.s.

In minix 3.1.8 Memory allocation is not possible in boot device
driver.
But 3.2.0 can do.

If you succeed in 3.1.8, Let me know, Please.

[kvleonhardt]

To visualize the problem please look at following test code (for minix
3.1.8):

Please look at the open command in the vga_transfer function and the
output listed below.

File: vga.c
[code]
#include <minix/drivers.h>
#include <minix/driver.h>
#include <minix/sysutil.h>
#include <minix/safecopies.h>
#include <minix/ds.h>
#include <minix/dmap.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>

/*
* Function prototypes for the vga driver.
*/
_PROTOTYPE( PRIVATE char * vga_name, (void) );
_PROTOTYPE( PRIVATE int vga_open, (struct driver *d, message
*m) );
_PROTOTYPE( PRIVATE int vga_close, (struct driver *d, message
*m) );
_PROTOTYPE( PRIVATE struct device * vga_prepare, (int device) );
_PROTOTYPE( PRIVATE int vga_transfer, (int procnr, int opcode, u64_t
position, iovec_t *iov, unsigned nr_req) );
_PROTOTYPE( PRIVATE void vga_geometry, (struct partition *entry) );

/* SEF functions and variables. */
_PROTOTYPE( PRIVATE void sef_local_startup, (void) );
_PROTOTYPE( PRIVATE int sef_cb_init_fresh, (int type, sef_init_info_t
*info) );
_PROTOTYPE( PRIVATE int sef_cb_lu_state_save, (int) );

/* Entry points to the vga driver. */
PRIVATE struct driver vga_tab =
{
vga_name,
vga_open,
vga_close,
nop_ioctl,
vga_prepare,
vga_transfer,
nop_cleanup,
vga_geometry,
nop_alarm,
nop_cancel,
nop_select,
nop_ioctl,
do_nop,

};

PRIVATE struct device vga_device;

/
*===========================================================================*
*
vga_name
*

*===========================================================================*/
PRIVATE char * vga_name(void)
{
printf("vga_name()\n");
return "vga";

}

/
*===========================================================================*
*
vga_open
*

*===========================================================================*/
PRIVATE int vga_open(d, m)
struct driver *d;
message *m;
{
printf("vga_open()\n");
return OK;

}

/
*===========================================================================*
*
vga_close
*

*===========================================================================*/
PRIVATE int vga_close(d, m)
struct driver *d;
message *m;
{
printf("vga_close()\n");
return OK;
}

/
*===========================================================================*
* vga_prepare *

*===========================================================================*/
PRIVATE struct device * vga_prepare(dev)
int dev;
{
vga_device.dv_base.lo = 0;
vga_device.dv_base.hi = 0;
vga_device.dv_size.lo = 0;
vga_device.dv_size.hi = 0;
return &vga_device;
}

/
*===========================================================================*
* vga_transfer *

*===========================================================================*/
PRIVATE int vga_transfer(proc_nr, opcode, position, iov, nr_req)
int proc_nr;
int opcode;
u64_t position;
iovec_t *iov;
unsigned nr_req;
{
static char toggle = 0;
int bytes, ret;

printf("vga_transfer()\n");
switch (opcode)
{
case DEV_GATHER_S:
break;
case DEV_SCATTER_S:
{
int fp;
if ((fp = open("/usr/src/drivers/vga/Farver.bmp", O_RDONLY)) <=
0)
{
printf("Can't open file. (%d)\n", fp);
return 0;
}
}
break;
default:
return EINVAL;
}
return ret;
}

/
*===========================================================================*
*
vga_geometry
*

*===========================================================================*/
PRIVATE void vga_geometry(entry)
struct partition *entry;
{
printf("vga_geometry()\n");
entry->cylinders = 0;
entry->heads = 0;
entry->sectors = 0;

}

/*************************************************************
*
* Function sef_cb_lu_state_save
*
* Descriptions:
* Handler for device driver
*
* Returns: OK
*
*************************************************************/
PRIVATE int sef_cb_lu_state_save(int state)
{
return OK;
}

/*************************************************************
*
* Function lu_state_restore
*
* Descriptions:
* Handler for device driver
*
* Returns: OK
*
*************************************************************/
PRIVATE int lu_state_restore(void)
{
return OK;
}


/
*===========================================================================*
*
sef_local_startup
*

*===========================================================================*/
PRIVATE void sef_local_startup(void)
{
/* Register init callbacks. */
sef_setcb_init_fresh(sef_cb_init_fresh);
sef_setcb_init_lu(sef_cb_init_fresh); /* treat live updates
as fresh inits */
sef_setcb_init_restart(sef_cb_init_fresh); /* treat restarts as
fresh inits */

/* Register live update callbacks. */
sef_setcb_lu_prepare(sef_cb_lu_prepare_always_ready); /*
agree to update immediately when a LU request is received in a
supported state */
sef_setcb_lu_state_isvalid(sef_cb_lu_state_isvalid_standard); /*
support live update starting from any standard state */
/* - Register a custom routine to save the state. */
sef_setcb_lu_state_save(sef_cb_lu_state_save);

/* Let SEF perform startup. */
sef_startup();
}

/
*===========================================================================*
*
sef_cb_init_fresh
*

*===========================================================================*/
PRIVATE int sef_cb_init_fresh(int type, sef_init_info_t *info)
{
int do_announce_driver = TRUE;
open_counter = 0;

/* Initialize the vga driver. */
switch(type)
{
case SEF_INIT_FRESH:
printf("%s", VGA_MESSAGE);
break;
case SEF_INIT_LU:
lu_state_restore();
do_announce_driver = FALSE;
printf("%sHey, I'm a new version!\n", VGA_MESSAGE);
break;
case SEF_INIT_RESTART:
printf("%sHey, I've just been restarted!\n", VGA_MESSAGE);
break;
}

if(do_announce_driver)
driver_announce();

/* Initialization completed successfully. */
return(OK);
}

/
*===========================================================================*
*
main
*

*===========================================================================*/
PUBLIC int main(int argc, char **argv)
{
/*
* Perform initialization.
*/
sef_local_startup();

/*
* Run the main loop.
*/

driver_task(&vga_tab, DRIVER_STD);

return OK;
}

To test the driver i just send new line to the device handle

echo "" > /dev/vga

and the output is:

May 14 10:36:56 minix kernel: vga_transfer()
May 14 10:36:56 minix kernel: VFS: only the PFS device can make nested
VFS calls
May 14 10:36:56 minix kernel: Can't open file. (-1)
May 14 10:36:56 minix kernel: vga_close()

[BeakGwangMoon]

I'm wrong. Sorry about that.

I think You made a TTY driver not a VGA Driver.

My TTY code is too long to visualize the problem.

moon

p.s

Why MINIX team say NOTHING about it ?

[BeakGwangMoon]

Why MINIX team say NOTHING about it ?

I hope the team find a solution.

include PnP, Memory.

[kvleonhardt]

Moon,

Thank you for trying, anyway.

kvleonhardt

[Erik van der Kouwe]

Hi,

> I believe that it must be possible to have file access from a device
> driver. But how?

It is neither possible nor desirable to access files from device
drivers. Since device drivers have elevated privilege levels, they
should be kept as simple as possible. This means pulling in all POSIX
interfaces is not a good idea, as this is a lot of code and complexity
(especially for example the signals). In addition, there is the
problem that VFS implements files and that the device driver is being
called by VFS. This creates a potential for deadlocks that can only be
solved with more complexity. In some cases, such as FUSE, this was
worth it as this is crucial for its functioning. For other drivers it
is not allowed.

If you need to pull in data from a file, one way to do it is to make a
user-level program supply that data over the read/write interface or
using ioctls.

With kind regards,
Erik

[kvleonhardt]

Hi

Found out something interesting. The error message in the log file
says:

May 12 10:49:08 minix kernel: VFS: only the PFS device can make nested
VFS calls

Of cause it is related to the fact that I try to open a file, caused
by a data transfer to the device handle /dev/vga.

But the I got the idea, what will happen if the actual file access is
done later?

First I tried to just set a flag in the vga_transfer function above,
and the handle the file access in a interrupt device routine hooked up
on the system timer interrupt (IRQ0).

It has a remarkable effect, but some times if the communication is
hold to the device handle while the timer interrupt occurs, I receive
tis error message:

May 15 08:59:09 minix kernel: deadlock between these processes:
May 15 08:59:09 minix kernel: vga vfs
May 15 08:59:09 minix kernel:
May 15 08:59:09 minix kernel: 49: vga 109723 prio 7 time 11/0 cycles
0x0005e8a44 cr3 0xf731000 rts RTS_PREEMPTED misc MF_REPLY_PEND
MF_FULLVM sched sched sigmgr rs(2)
May 15 08:59:09 minix kernel: vga 109723 0x70e9 0x6cb7 0x7359
0x207f 0x3369 0x2324 0x100a
May 15 08:59:09 minix kernel: 1: vfs 1 prio 5 time 394/0 cycles
0x05e895486 cr3 0x0 rts RTS_SENDING RTS_RECEIVING misc MF_REPLY_PEND
sched KERNEL sigmgr rs(2) blocked on: vga(109723)
May 15 08:59:09 minix kernel: vfs 1 0x11371 0x66dd 0x64f2
0x5d4a 0x9397 0x39b2 0x397d 0x2219 0x100a

But the program continues, and closes the file again as expected. Then
I trued to make a delay, to be sure that the communication with the
device handle is closed. Then all worked without errors.

The function is as followed. OpenFile is set to 10 in the vga_transfer
function.

PRIVATE int vga_interrupt( struct driver *d, message *m)
{
if (OpenFile)
{
int fp;
OpenFile--;
if(OpenFile)
return;
printf("vga_interrupt: Trying to open file\n");
OpenFile = FALSE;
if ((fp = open("/etc/passwd", O_RDONLY)) < 0)
{
printf("vga_interrupt: Can't open file. (%d)\n", fp);
}
else
{
int r;
char buf[11];
printf("vga_interrupt: File open. (%d)\n", fp);
read(fp, buf, 10);
buf[10] = 0;
printf("%s\n", buf);
r = close(fp);
printf("vga_interrupt: File closed. (%d)\n", r);
}
}
}

Then the output is as this:

May 15 09:03:27 minix kernel: vga_open()
May 15 09:03:27 minix kernel: vga_transfer()
May 15 09:03:27 minix kernel: vga_close()
May 15 09:03:27 minix kernel: vga_interrupt: Trying to open file
May 15 09:03:27 minix kernel: vga_interrupt: File open. (0)
May 15 09:03:27 minix kernel: root:##roo
May 15 09:03:27 minix kernel: vga_interrupt: File closed. (0)

BUT
A device driver that cannot act without disconnecting the connection
while working is not very useful :-(

And then my questions.
What deadlock is detected, and what are the result. It seems to work
anyway)
And how can I avoid it?

I hope that somebody has a clue. It will be much appreciated. Thank
you in advance.

kvleonhardt

[Erik van der Kouwe]

Hi,

> And then my questions.
> What deadlock is detected, and what are the result. It seems to work
> anyway)
> And how can I avoid it?
>
> I hope that somebody has a clue. It will be much appreciated. Thank
> you in advance.

Drivers are not supposed to access files in MINIX, so even if you do
get it to work it won't be reliable. For example, if additional checks
to prevent this are added in the future then it would stop working
altogether.

I recommend that you open and parse the file in userspace and then
send only the relevant data to the driver. This keeps the driver as
simple as possible and hence the system as reliable as possible.

BTW if the file always contains the same data you can also embed it in
the driver binary of course.

With kind regards,
Erik

[Thomas Veerman]

Hi kvleonhardt,

What you've observed is indeed what is to be expected. That is, when VFS
talks to a driver and the driver subsequently does a system call to VFS
(i.e., a nested call), the call is supposed to fail. The reason is
simple; nested calls are very sensitive to deadlocks. There is an
exception for PFS, but the calls PFS is allowed to make are very well
defined and cannot lead to deadlocks. The current code should be more
clear about this and I'll fix that as soon as I can.

Also, when a driver initiates a system call to VFS that is not nested
this will work. However, in my opinion Erik is right in suggesting the
driver should not do any system calls at all. The goal of a driver is to
send provided data to the hardware or retrieve data from hardware and
store it in a provided buffer. In other words, you should use ioctls to
steer your driver. ;)

--
Thomas

[kvleonhardt]

Hi Thomas and Erik

Thank you to both of you for your comments.

It was an easy way to load graphics to the VGA memory, just to pass a
pointer to the file together with the position to put the graphic.
Furthermore it would give a minimal load of the system, as passing a
lot of graphical data data through the device handle would perform a
lot of overhead. And embedding half a megabyte of data in the
executables of a device driver does not sound good in my ears.

I have to choose between plague an cholera as we say here in Denmark.

Before I read your answers i tried to open files for writing in the /
tmp directory, but that does not work at all.

But thanks

kvleonhardt

[Antoine LECA]

kvleonhardt wrote:
> It was an easy way to load graphics to the VGA memory, just to pass a
> pointer to the file together with the position to put the graphic.

But why should your _driver_ does that stuff?

It very much seems to me the workload of a userland "system
administration" utility to open that file, parse or expand it if needed
to, then "upload" the content to the driver through either normal
write(2) or dedicated ioctl(2).

BTW, look at loadkeys(1) or loadfont(1) utilities, they are doing
exactly that.

> Furthermore it would give a minimal load of the system, as passing a
> lot of graphical data data through the device handle would perform a
> lot of overhead. And embedding half a megabyte of data in the
> executables of a device driver does not sound good in my ears.

Embedding the whole POSIX interface for files do not make sense either.


Antoine

[BeakGwangMoon]

I'm not a heavy user.

I Just think a SIMPLE solution about the problem.

If it is possible to kernel can make device driver into two sort, one
for a file(ex. Hard disk), one for a normal (ex. TTY),

we can make a new syscall for a file access of a normal device
driver.
(like a OOP method ; ))

A normal driver --> VFS --> A file driver.

What do you think about this concept ?

thanks.

moon.

[Erik van der Kouwe]

Hi,

> If it is possible to kernel can make device driver into two sort, one
> for a file(ex. Hard disk), one for a normal (ex. TTY),
>
> we can make a new syscall for a file access of  a normal device
> driver.
> (like a OOP method ; ))
>
> A normal driver --> VFS --> A file driver.
>
> What do you think about this concept ?

Every driver is always accessed through VFS because it implements
system calls like open, reda, write and ioctl. Therefore, this would
not work.

In addition, we really don't want drivers to deal with files at all
for reliability reasons. Reliability is achieved through creation of
small isolated modules with a minimal level of privilege.

In this particular case, there is actually no need to access files.
read/write or ioctl can be used to pass the data.

With kind regards,
Erik

[Antoine LECA]

BeakGwangMoon wrote:
> I Just think a SIMPLE solution about the problem.
>
> If it is possible to kernel can make device driver into two sort, one
> for a file(ex. Hard disk), one for a normal (ex. TTY),

Does it match the already existing distinction between block-device
drivers and char-device drivers?

> we can make a new syscall for a file access of a normal device
> driver. (like a OOP method ; ))
>
> A normal driver --> VFS --> A file driver.
>
> What do you think about this concept ?

That calls dealt with in TTY (or any "normal driver" I can think of)
already go through VFS. So the full draw looks like

Userland -> VFS -> A normal driver -> VFS -> A file driver

Do you see the problem?


Antoine

[BeakGwangMoon]

Hi

Erik van der Kouwe wrote:
> In addition, we really don't want drivers to deal with files at all
> for reliability reasons. Reliability is achieved through creation of
> small isolated modules with a minimal level of privilege.

Yes, Because of that, I like Minix rather than Linux

> In this particular case, there is actually no need to access files.
> read/write or ioctl can be used to pass the data.
>

I'll try that call in my TTY code.

With kind regards,
moon

[BeakGwangMoon]

First of all, Thanks Antoine LECA and Erik van der Kouwe for reading
and writing.

Antoine LECA wrote:
> Does it match the already existing distinction between block-device
> drivers and char-device drivers?

I mean a driver to use a file syscall not merely distinction.

> That calls dealt with in TTY (or any "normal driver" I can think of)
> already go through VFS. So the full draw looks like
>
>    Userland -> VFS -> A normal driver -> VFS -> A file driver
>
> Do you see the problem?

Userland -> VFS -> A normal driver -> VFS -> A file driver

^^^^ ^^^^
Yes, I see the problem.


With kind regards,
moon