O_DIRECT and tcl
ikeon
I'm writing a pure TCL application that one program is reading
commands from a file in NFS (network file share).
I'm using "open" command and "gets" command, than "seek" to the first
line and "gets" again.
One scripts is writing to the file locally and other script reads it
from NFS.
the file is being changed in 100 ms rate.
When i read the file from NFS I see only per second changes.
I was told it has to do with NFS cache and stuff and after reading NFS
behavior the cache can be reduced but not disabled.
One recommendation was to open the file with O_DIRECT flag which is an
option like blocking or non_blocking - this way there is a direct I/O
to the file and no cache/buffers.
Can it be done in TCL ? Somehow control the flags related to the file
opening.
Thank.
Nick Hounsome
If you think about it a bit more you will realise that the compiler/
library/OS cannot implement anything like O_DIRECT for remote files
and that buffering is unavoidable. This isn't a limitation of Tcl.
NFS is just not suitable for the sort of communication that you are
trying to do.
I think that you really need sockets.
If you really have to write to a file then maybe you could try named
pipes, possibly with a pipe to cocket converter app on the writing
machine.
Donal K. Fellows
> behavior the cache can be reduced but not disabled.
> One recommendation was to open the file with O_DIRECT flag which is an
> option like blocking or non_blocking - this way there is a direct I/O
> to the file and no cache/buffers.
There is no way that that could ever be made to work over a network,
even if Tcl supported passing that flag through (which it doesn't, as
it happens). The issue is that network systems have buffers *anyway*
in the OS and the fileserver probably buffers stuff too (to say
nothing of the other machine). Sorry, but it won't ever work.
Instead, you need a different system architecture (e.g., a dedicated
server that acts as a control point, or maybe a database server).
Changing from one way of doing things to another is non-trivial, but
can't be helped as you're hitting the point where simple hacks don't
cut it.
Donal.
ikeon
Thank for your help !
I did some workaround that work...
Instead of reading remotley and writeing locally i've turned it the
other way around -
Reading locally and writing remotely using "flush" so that way the
file get updated through the network immediately and the reading is
locally so it solved my problems.
Thanks again.
Shay.
Alexandre Ferrieux
Still, you've been given a strong signal that it was a flawed
approach. Any reason to ignore better interprocess communication
primitives (like sockets) ?
-Alex
Nick Hounsome
Good.
But "network" and "immediately" are incompatible so you might as well
forget about the flush.
ikeon
wrote:
>
> - Show quoted text -
Gentelmans,
I'm sure sockets is a much better approch.
The thing is that I'm in a bit of a rush and my twisted solution is
good enough for now.
I'll try to implement sockets later on.
Thank you all for the good advises.
Shay.