bup save fails with Error 22 Invalid Argument

[Axelle Apvrille]

Hello,
I am using bup, recently pulled from git. Make check worked with
success, no error.
Then, I initialized (bup init) and indexed my home and ran "bup save
-n myhome /home/axelle". It crashes bup :

Reading index: 462630, done.
/home/axelle/softs/openjdk/sources/openjdk8/jaxws/.hg/store/data/src/share/classes/com/sun/xml/internal/ws/developer/_uses_j_a_x_b_context_feature.java.i:
[Errno 22] Invalid argument
Traceback (most recent call last):
File "/usr/local/lib/bup/cmd/bup-save", line 416, in <module>
keep_boundaries=False)
File "/usr/local/lib/bup/bup/hashsplit.py", line 243, in split_to_blob_or_tree
files, keep_boundaries, progress))
File "/usr/local/lib/bup/bup/hashsplit.py", line 231, in split_to_shalist
for (sha,size,level) in sl:
File "/usr/local/lib/bup/bup/hashsplit.py", line 185, in split_to_blobs
sha = makeblob(blob)
File "/usr/local/lib/bup/bup/git.py", line 785, in new_blob
return self.maybe_write(b'blob', blob)
File "/usr/local/lib/bup/bup/git.py", line 780, in maybe_write
self.just_write(sha, type, content)
File "/usr/local/lib/bup/bup/git.py", line 770, in just_write
self._write(sha, type, content)
File "/usr/local/lib/bup/bup/git.py", line 747, in _write
self.breakpoint()
File "/usr/local/lib/bup/bup/git.py", line 752, in breakpoint
id = self._end(self.run_midx)
File "/usr/local/lib/bup/bup/git.py", line 859, in _end
packbin)
File "/usr/local/lib/bup/bup/git.py", line 900, in _write_pack_idx_v2
assert(count == self.count)
AssertionError

[axelle....@gmail.com]

FYI, the same error occurs when using branch 0.30.

/home/axelle/softs/openjdk/sources/openjdk8/build/linux-x86_64-normal-server-release/hotspot/linux_amd64_compiler2/product/verifier.o: [Errno 22] Invalid argument

Traceback (most recent call last):                                              
  File "/usr/local/lib/bup/cmd/bup-save", line 416, in <module>                
    keep_boundaries=False)                                                      
  File "/usr/local/lib/bup/bup/hashsplit.py", line 243, in split_to_blob_or_tree
    files, keep_boundaries, progress))                                          
  File "/usr/local/lib/bup/bup/hashsplit.py", line 231, in split_to_shalist    
    for (sha,size,level) in sl:      

          

I noticed during bup index that I had 2 warnings ("permission denied") on some directories (not accessible to this account). Those warnings seem "normal" however so I don't think they are related with the python crash.

I am using Linux Mint 19.1. My file system is a mixture of ext4 and ZFS. The file on which bup crashes is on ext4. The BUP_DIR is located on a CIFS filesystem (and I have permissions to write on the dir).

Thanks

Axelle.

[Johannes Berg]

On Thu, 2020-02-06 at 13:25 -0800, axelle....@gmail.com wrote:
> FYI, the same error occurs when using branch 0.30.

Maybe if it keeps recurring, change the code to something like

assert count == self.count, "%d vs. %d" % (count, self,count)

so we know the values.

I'm sort of thinking perhaps there's a short write or something, and the
code doesn't handle that well? Not sure though, didn't look much.

johannes

[Johannes Berg]

No, wait ...

> I am using Linux Mint 19.1. My file system is a mixture of ext4 and
> ZFS. The file on which bup crashes is on ext4.

FWIW, it doesn't really crash on this file. It just happens to be the
last warning that occurred before the crash.

The crash actually happens when it's finishing a pack file because it
got full, and then wants to write the index for it.

> The BUP_DIR is located on a CIFS filesystem (and I have permissions to
> write on the dir).

But maybe that's it? We do a lot of (actually pretty unnecessary) mmap
tricks here, so perhaps CIFS just doesn't like that very much -
especially msync() appears to force going to the server (unless you
mounted with "nostrictsync") which may fail or something here?

Do you know what options this was mounted with?

johannes

[Axelle Apvrille]

So, if I modified the assertion line and I get: "AssertionError: 1 vs. 200001"

The CIFS partition is mounted with options :
defaults,credentials=/etc/fstab.d/cred.timecaps,sec=ntlm,uid=axelle,gid=users,noauto,vers=1.0

I tried with a BUP_DIR on another partition (ext3) and it worked without errors.

-- Axelle

[Johannes Berg]

On Mon, 2020-02-10 at 22:25 +0100, Axelle Apvrille wrote:
> So, if I modified the assertion line and I get: "AssertionError: 1 vs. 200001"

So ... I've been thinking about this, but I cannot understand how this
is even possible.

> The CIFS partition is mounted with options :
> defaults,credentials=/etc/fstab.d/cred.timecaps,sec=ntlm,uid=axelle,gid=users,noauto,vers=1.0
>
> I tried with a BUP_DIR on another partition (ext3) and it worked without errors.

Good to know, so it really is CIFS specific.

But unfortunately I have no idea as to what might be done to fix this,
sorry.

johannes

[Johannes Berg]

On Tue, 2020-02-11 at 22:59 +0100, Johannes Berg wrote:
>
> > The CIFS partition is mounted with options :
> > defaults,credentials=/etc/fstab.d/cred.timecaps,sec=ntlm,uid=axelle,gid=users,noauto,vers=1.0
> >
> > I tried with a BUP_DIR on another partition (ext3) and it worked without errors.
>
> Good to know, so it really is CIFS specific.

FWIW, I couldn't even reproduce this with a CIFS filesystem.

But my options were something like

rw,relatime,vers=1.0,cache=strict,username=...,domain=...,
uid=0,noforceuid,gid=0,noforcegid,addr=...,file_mode=0755,
dir_mode=0755,soft,nounix,mapposix,rsize=61440,wsize=65536,
bsize=1048576,echo_interval=60,actimeo=1

and I didn't try to make a large backup, just a very small one.

johannes