Locking a file to avoiding it been read/written when it's just being created.

[Hongyi Zhao]

Hi all,

I've a huge file which is being created by some commands / cripts, say mv/
cp and etc., I want to lock the file to avoiding it been read/written
till the file is completely created.

How to achieve this job?

Regards
--
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.

[Kaz Kylheku]

On 2016-11-04, Hongyi Zhao <hongy...@gmail.com> wrote:
> Hi all,
>
> I've a huge file which is being created by some commands / cripts, say mv/
> cp and etc., I want to lock the file to avoiding it been read/written
> till the file is completely created.
>
> How to achieve this job?

The easiest thing is to use a temporary file which has a properly
"anonymized" name.

If the file being created is a name like /tmp/tmp.uEqAKULVJ7
or whatever, then only a maliciously crafted program (running
as the same user or root) will interfere.

The legitimate programs which are looking for "outputfile"
will not see anything until your script executes

mv /tmp/tmp.uEqAKULVJ7 outputfile

the file is then "published".

You probably don't even need a fancy name in /tmp; just any
name that the other programs aren't looking for, like
"outputfile.inprogress" which gets renamed to "outputfile"
on completion.

[Hongyi Zhao]

On Fri, 04 Nov 2016 02:31:38 +0000, Kaz Kylheku wrote:

> The legitimate programs which are looking for "outputfile"
> will not see anything until your script executes
>
> mv /tmp/tmp.uEqAKULVJ7 outputfile

Are all of the cp / mv / rename atomic?

>
> the file is then "published".

[Kaz Kylheku]

On 2016-11-04, Hongyi Zhao <hongy...@gmail.com> wrote:

> On Fri, 04 Nov 2016 02:31:38 +0000, Kaz Kylheku wrote:
>
>> The legitimate programs which are looking for "outputfile"
>> will not see anything until your script executes
>>
>> mv /tmp/tmp.uEqAKULVJ7 outputfile
>
> Are all of the cp / mv / rename atomic?

Only renaming (mv utility, rename C function) is atomic.

mv is only atomic if you don't use it to move objects
from one filesystem to another; then it emulates simulates a rename
by copying.

Renaming multiple files isn't atomic: i.e. "mv a.tmp a; mv b.tmp b"
obviously. If you want both to be published at the same time, put them
in a directory first and rename that. Or find a way to put the data
into one file.

[gerg]

Hongyi Zhao <hongy...@gmail.com> wrote:
>
>I've a huge file which is being created by some commands / cripts, say mv/
>cp and etc., I want to lock the file to avoiding it been read/written
>till the file is completely created.
>
>How to achieve this job?
>

The approach I used to solve this was to write my scripts so the
new file is created in a one subdirectory, and when the creation
is finished (successfully), the mv command is used to add the file
to another subdirectory. (on the same filesystem, naturally)

The scripts that want to work with the completed file only watch
the other subdirectory. They don't see the file until it's complete.

This isn't the only way. Rsync defaults to creating a 'hidden' file
during transfer and renames it afterward.

-Greg
--
::::::::::::: Greg Andrews ::::: ge...@panix.com :::::::::::::
I have a map of the United States that's actual size.
-- Steven Wright