Question about umount and sync

[Jan Panteltje]

Question about umount and sync

In the long ago past you could type 'sync' 3 times to make sure a disk
was umounted..
Then later I used umount, and it returned when all data was written.

But I notice on my new 16GB USB sticks that the busy light just keeps flashing
for up to a minute after umount and sync have long returned.
So what is a reliable way to umount these things from for exmaple a script,
and be sure before allowing the user to unplug such an USB stick?

[Joe Beanfish]

umount has always been and still is the way to unmount. It will call
sync then detach the filesystem from the OS in a safe way so there's
no lost data. Sync just flushes the buffers. No amount of syncing does
or ever did properly detach a filesystem from the OS. Pure luck kept
you out of trouble using that technique.

The ongoing activity is probably the USB bus probing etc. Don't worry
about that. Umount is definitive.

[Jan Panteltje]

On a sunny day (Fri, 29 Jun 2012 15:47:23 -0400) it happened Joe Beanfish
<joebe...@nospam.duh> wrote in <jsl0od$tlb$1...@dont-email.me>:

Thank you.

[Richard Kettlewell]

Joe Beanfish <joebe...@nospam.duh> writes:
> Jan Panteltje wrote:

>> Question about umount and sync
>>
>> In the long ago past you could type 'sync' 3 times to make sure a
>> disk was umounted.. Then later I used umount, and it returned when
>> all data was written.
>>
>> But I notice on my new 16GB USB sticks that the busy light just keeps
>> flashing for up to a minute after umount and sync have long returned.
>> So what is a reliable way to umount these things from for exmaple a
>> script, and be sure before allowing the user to unplug such an USB
>> stick?
>
> umount has always been and still is the way to unmount. It will call
> sync then detach the filesystem from the OS in a safe way so there's
> no lost data. Sync just flushes the buffers. No amount of syncing does
> or ever did properly detach a filesystem from the OS. Pure luck kept
> you out of trouble using that technique.

umount (the syscall) does block until the kernel has no more writes to
do. However umount (the program) can invoke a helper program instead of
calling the syscall directly and if that helper goes wrong then the
umount program can terminate while there are still writes pending.

You should at least see some kind of error message in this case though.

> The ongoing activity is probably the USB bus probing etc. Don't worry
> about that. Umount is definitive.

I've noticed similar behavior with spinning rust (and no helpers):
umount completes but then disk activity starts up, and audibly so. I
don't know what's going on here.

--
http://www.greenend.org.uk/rjk/

[Joe Beanfish]

Perhaps the disk flushing it's buffer to the platter?

[amel...@gmail.com]

But this raises the question: what good is sync if it does not properly
flush filesystem-related data too?
What's the use for sync then?

[Joe Beanfish]

On Mon, 18 Aug 2014 12:58:35 -0700, ameliusje wrote:
> On Friday, June 29, 2012 9:47:23 PM UTC+2, Joe Beanfish wrote:
>> On 06/29/2012 12:44 PM, Jan Panteltje wrote:
>> > Question about umount and sync
>> >
>> > In the long ago past you could type 'sync' 3 times to make sure a
>> > disk
>> > was umounted..
>> > Then later I used umount, and it returned when all data was written.
>> >
>> > But I notice on my new 16GB USB sticks that the busy light just keeps
>> > flashing
>> > for up to a minute after umount and sync have long returned.
>> > So what is a reliable way to umount these things from for exmaple a
>> > script,
>> > and be sure before allowing the user to unplug such an USB stick?
>>
>> umount has always been and still is the way to unmount. It will call
>> sync then detach the filesystem from the OS in a safe way so there's
>> no lost data. Sync just flushes the buffers. No amount of syncing does
>> or ever did properly detach a filesystem from the OS. Pure luck kept
>> you out of trouble using that technique.
>>
>> The ongoing activity is probably the USB bus probing etc. Don't worry
>> about that. Umount is definitive.
>
> But this raises the question: what good is sync if it does not properly
> flush filesystem-related data too?
> What's the use for sync then?

It does sync the data. But until unmounted the filesystem is still in use
and may have more data and meta data written at any time. So depending on
the filesystem type and timing you may get away with sync-and-yank but
it's highly inappropriate. "Sync 3 times" was never appropriate even if
some people did it before because they were too lazy to umount correctly.

[Jan Panteltje]

On a sunny day (Tue, 19 Aug 2014 13:29:05 +0000 (UTC)) it happened Joe
Beanfish <joebe...@nospam.duh> wrote in <lsvjf0$dlm$1...@dont-email.me>:

'I've stopped using filesystems altogether,
and now just dd my data directly to the media,
be it stick, card, DVD-R or BR-R-25.
Do I still need sync before removing those media?

[Joe Beanfish]

Linux has caching on block devices. Make sure you use a raw device.
See http://www.tldp.org/HOWTO/SCSI-2.4-HOWTO/rawdev.html

[Jan Panteltje]

On a sunny day (Thu, 21 Aug 2014 13:22:29 +0000 (UTC)) it happened Joe
Beanfish <joebe...@nospam.duh> wrote in <lt4rqk$k6v$1...@dont-email.me>:

>Linux has caching on block devices. Make sure you use a raw device.
>See http://www.tldp.org/HOWTO/SCSI-2.4-HOWTO/rawdev.html

Interesting, did not know about 'raw',
but used dd with the correct sector size.

Maybe in some cases 'raw' is faster.

What I have done a lot of times is just tar a directory,
and then
growisofs [options] -Z /dev/dvd=big_tar.gz

dvd can be dvd or cd or bluray or some other media
This used for backups.

Then when you want to get everything back:
tar -zxvf /dev/dvd

Or write a whole movie to a dvd-r.
cat /dev/dvd | mplayer - for the cat people.
dd if=/dev/dvd bs=10000000 skip=1000 | mplayer - to start halfway in the movie.
Saves time authoring disks.
Actually I do that from main PC to laptop to watch movies via netcat too.
works great (recording HD movies on the main PC via USB sat receiver) and watching
in other room on laptop..
scripted of course.

About not using filesystems, I recentley designed a GPS based litte radiation counter, GPS logger,
that writes to SDcard,
and a simple C program to display the data (and show the location in google maps).
read_gmp_card-0.3.c
See:
http://panteltje.com/panteltje/pic/gm_pic2/
Who needs filesystems...

Maybe I can use 'raw' for some fast video stuff.
one of the advantages of writing directly is that all sectors are sequential...
reducing seek times, at least for old media it was, modern media controllers do whatever they like
like bad sector management, write limiting, etc (cards).