Disk write shell script
Fabio Giannotti
That is to say, if you write a "regular" shell script that writes 100
lines to a file, it may only hit the disk 1 or 2 times. I assuming it's
buffering or caching a bunch, then
writing a bunch all at once.
What I want is for each line written to "go right to disk" and not be
buffered/cached.
Any ideas?
Thanks,
Fabio
The comments/views expressed here are my own, and not of my company.
Jean-Pierre Radley
| Is there a way using bourne shell to write directly to the hard disk?
|
| That is to say, if you write a "regular" shell script that writes 100
| lines to a file, it may only hit the disk 1 or 2 times. I assuming it's
| buffering or caching a bunch, then
| writing a bunch all at once.
|
| What I want is for each line written to "go right to disk" and not be
| buffered/cached.
man sync(ADM)
--
Jean-Pierre Radley <j...@jpr.com> XC/XT Custodian Sysop, CompuServe SCOForum
Radek Tomis
> Sent: Thursday, August 05, 1999 12:51 AM
> Is there a way using bourne shell to write directly to the hard disk?
>
> That is to say, if you write a "regular" shell script that writes 100
> lines to a file, it may only hit the disk 1 or 2 times. I assuming it's
> buffering or caching a bunch, then writing a bunch all at once.
>
> What I want is for each line written to "go right to disk" and not be
> buffered/cached.
>
> Any ideas?
If you want transparent synchronous output, then download this 35 KB binary:
http://members.xoom.com/rts101/tmp/fildes
and use it inside (or outside) of your script in one or more of the
following ways:
1. Inside your script for individual output commands:
fildes 1=~,A+S : echo "line" >> /tmp/file
This will append "line" to "/tmp/file" synchronously (i.e. the line will
be physically written to the disk drive before the command 'echo'
returns).
Consider the following example using different output command:
fildes 1=~,S : tail -300 /usr/adm/messages > /tmp/file
This will synchronously append last 300 lines of "/usr/adm/messages"
to "/tmp/file. However, 'tail' (and almost all other commands/utilities)
uses user-level standard buffered I/O (from standard C library), and if
the output is directed to something other than a terminal device, then
the output is buffered by constant block size, usually 1 KB (depends on
implementation). So, the 'tail' above actually writes (calls kernel
routine write(S)) to "/tmp/file" only once per 1 KB (no matter whether
this happens to be in the middle of the read line or not). In this case,
each such 1K block gets written synchronously (the 1K block physically
appears on the disk drive before the write(S) call returns).
2. Inside your script for several output commands at once:
fildes 1=/tmp/file,A+S : sh -c '(
set -- comp unix sco misc
echo "`date`: arguments:"
for arg
do
echo " arg: ${arg}"
sleep 1 # some processing
done
)'
This will append several lines to "/tmp/file". Each line will be written
synchronously (the line will be physically written to the disk drive
before the command 'echo' returns).
3. Outside of your script for the entire script:
fildes 1=/tmp/file,S : <your_script>
This will run your script with standard output redirected to "/tmp/file".
Every standard output from your script (and all its children, i.e. all
external commands used within your script) will be synchronous, i.e. all
underlying write(S) calls completes after all data in the given write(S)
call have been physically written to the disk drive.
You can also use standard system command 'sync' after each line output,
however, it will write out not only this last line to the disk, but also all
other dirty file system blocks from system memory buffers for all file
systems. Depending on the recent write activity in the system, it may of
course take much longer than actually needed for your one-line write.
--
Radek Tomis
r...@mediumsoft.cz
Radek Tomis
> Sent: Thursday, August 05, 1999 3:01 PM
[...]
> 1. Inside your script for individual output commands:
>
> fildes 1=~,A+S : echo "line" >> /tmp/file
>
> This will append "line" to "/tmp/file" synchronously (i.e. the line
> will be physically written to the disk drive before the command 'echo'
> returns).
>
> Consider the following example using different output command:
>
> fildes 1=~,S : tail -300 /usr/adm/messages > /tmp/file
>
> This will synchronously append last 300 lines of "/usr/adm/messages"
> to "/tmp/file.
Sorry, this will not append, but create/overwrite instead.
For append, add "+A" to "1=~,S" (">>" instead of ">" isn't necessary).
--
Radek Tomis
r...@mediumsoft.cz
Radek Tomis
> Sent: Thursday, August 05, 1999 3:01 PM
[...]
> 3. Outside of your script for the entire script:
>
> fildes 1=/tmp/file,S : <your_script>
>
Sorry, using merely 'S' in this example will neither create "/tmp/file" if
it doesn't already exist nor truncate "/tmp/file" if it already exists (and
its size is greater than zero). To get the same effect as with shell
redirection '>', you will want to add '+C+T' to 'S'.
> This will run your script with standard output redirected to
> "/tmp/file". Every standard output from your script (and all its
> children, i.e. all external commands used within your script) will be
> synchronous, i.e. all underlying write(S) calls completes after all
> data in the given write(S) call have been physically written to the
> disk drive.
[...]
--
Radek Tomis
r...@mediumsoft.cz