bup repo on cifs/samba mountpoint

[Mike Nunn]

Hi I have bup v 0.29-2-gb75d996 on an up to date debian jessie

make test passes ok

I get the following stack trace when doing a bup save with the bup repo on a cifs/samba mountpoint:

Indexing: 2, done (3195 paths/s).

Reading index: 1, done.

Saving: 100.00% (0/0k, 1/1 files), done.

Traceback (most recent call last):

  File "/usr/local/lib/bup/cmd/bup-save", line 460, in <module>

    w.close()  # must close before we can update the ref

  File "/usr/local/lib/bup/bup/git.py", line 790, in close

    return self._end(run_midx=run_midx)

  File "/usr/local/lib/bup/bup/git.py", line 776, in _end

    os.fsync(self.parentfd)

OSError: [Errno 22] Invalid argument

when the bup repo is on local disk it is fine.

mount output for the relevant cifs mountpoint is:

//u144509.your-backup.de/backup on /backup type cifs (rw,relatime,vers=1.0,cache=strict,username=u144509,domain=ROOTBACKUP78,uid=34,forceuid,gid=1003,forcegid,addr=2a01:04f8:0b15:1000:0000:0000:0000:0016,unix,posixpaths,serverino,mapposix,acl,rsize=1048576,wsize=65536,echo_interval=60,actimeo=1)

Thanks

Mike

[Rob Browning]

Mike Nunn <bonn...@gmail.com> writes:

> I get the following stack trace when doing a bup save with the bup repo on
> a cifs/samba mountpoint:
>
> Indexing: 2, done (3195 paths/s).
> Reading index: 1, done.
> Saving: 100.00% (0/0k, 1/1 files), done.
> Traceback (most recent call last):
> File "/usr/local/lib/bup/cmd/bup-save", line 460, in <module>
> w.close() # must close before we can update the ref
> File "/usr/local/lib/bup/bup/git.py", line 790, in close
> return self._end(run_midx=run_midx)
> File "/usr/local/lib/bup/bup/git.py", line 776, in _end
> os.fsync(self.parentfd)
> OSError: [Errno 22] Invalid argument

Hmm, perhaps cifs is cranky about attempts to fsync directories. I suppose
one option might be to ignore (or simply report) EINVAL when bup is
acting on a directory, i.e.

try:
os.fsync(self.parentfd)
except OSError as ex:
if ex.errno != errno.EINVAL:
raise
log(...)
...

We'd need to think about that though, i.e. are there cases where that
would be inappropriate.

You could probably test the error directly via:

python -c "import os; os.fsync(os.open('.', os.O_RDONLY))"

If the guess above is correct, that should work when the current
directory is say ext4, but not when it's cifs.

Thanks
--
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[Mike Nunn]

On 11 February 2017 at 16:25, Rob Browning <r...@defaultvalue.org> wrote:

> You could probably test the error directly via:
>
> python -c "import os; os.fsync(os.open('.', os.O_RDONLY))"
>
> If the guess above is correct, that should work when the current
> directory is say ext4, but not when it's cifs.

Yes just as you say exits cleanly on ext4 and the following on the cifs mount:

python -c "import os; os.fsync(os.open('.', os.O_RDONLY))"

Traceback (most recent call last):

File "<string>", line 1, in <module>

OSError: [Errno 22] Invalid argument

is there anything I can do about it? I don't have access to the nas
server so not sure what it is.

Thanks for your help.

Mike

[Rob Browning]

Mike Nunn <bonn...@gmail.com> writes:

> is there anything I can do about it? I don't have access to the nas
> server so not sure what it is.

Well as a quick hack, you could perhaps add the try/catch to your copy
of bup around the parent fsync (and around any other dir syncs that
crash).

In the longer run, we need to decide if we think that's generally OK.
If so, then I'll add something similar to master.

My first impression is that ignoring (or just logging) EINVAL for fsync
on a dir sounds plausible, but of course we want to be cautious where
fsync is concerned.

Greg - any opinion?

[poulikp...@gmail.com]

Hi, it has been 2 years and it seems the problem is still there. But thanks for the information how to fix it.

Dne neděle 12. února 2017 0:23:17 UTC+1 Rob Browning napsal(a):

[Greg Troxel]

I may have been paged out and not answered earlier. As always, I
searched for

site:opengroup.org fsync 2017

and found

https://pubs.opengroup.org/onlinepubs/9699919799/functions/fsync.html

which talks about "files" and doesn't mention directories.

My take is that an OS should allow fsync on a dir, as getting directory
contents to stable storage is sensible, and a directory is a file.

It sounds like there is a NAS that when accessed over some remote
filesystem protocol, is returning EINVAL for fsync, instead of (doing
something) and returning 0?

I would say that given that reality, it is totally ok for bup to
continue when getting EINVAL on fsync, but it should complain about it
the first time. Basically s/error/warning/.

Ideally bup would have a way to count warnings so that the first 10
would be logged and then at exit the total count would be printed.

But, printing the first fsync/dir failure and then being quiet seems
pretty reasonable. as does printing all of them, which I'd prefer if
the number that happens in real situation is not so high as to be
overwhelming.