"{svn | svnadmin}: Decompression of svndiff data failed" and "svn: Malformed representation header", "svn: Corrupt representation" messages
Bastian Ugimachi
I recently installed a subversion (version 1.6.11) repository on out RHEL 5 system, that I access using ssh+svn protocol. Already after some very few commits, I ran into the problem of getting the error "svnadmin: Decompression of svndiff data failed" when performing "svnadmin dump /path/to/rep > /path/to/backup/repo.dump". I get the same error "svn: Decompression of svndiff data failed" when trying to check out the entire repository using the command line with "svn checkout svn+ssh://localhost/path/to/rep /path/to/co".
I did quite a lot of searches on the internet and tried "svnadmin verify /path/to/rep", also returning the same error.
* Verified revision 1.
* Verified revision 2.
svnadmin: Decompression of svndiff data failed
It seems to be revision 3 which is corrupted, as a revision specific verifcation with "svnadmin verify /path/to/rep -r x" succeeds with x=1 and x=2:
* Verified revision 0.
resp.
* Verified revision 1.
Verifying revision 3 (and any subsequent revision) returns the well-known error again:
svnadmin: Decompression of svndiff data failed
Strangely, "svnadmin recover /path/to/rep" successfully succeeds:
svnadmin recover /path/to/rep
Repository lock acquired.
Please wait; recovering the repository may take some time...
Recovery completed.
The latest repos revision is 8.
I came accorss some posts, suggesting to use fsfsverify.py and apply it to the revisions. Applying "./fsfsverify.py -f /opt/maps/svn/maps/db/revs/0/x" it to revisions x=0 and x=1 successfully terminates but applying it to revision 2 (and all subsequent ones) terminates with the error:
Traceback (most recent call last):
File "./fsfsverify.py", line 1120, in ?
for noderev in strategy:
File "./fsfsverify.py", line 839, in _nodeWalker
for x in self._nodeWalker():
File "./fsfsverify.py", line 839, in _nodeWalker
for x in self._nodeWalker():
File "./fsfsverify.py", line 832, in _nodeWalker
noderev = NodeRev(self.f, self.currentRev)
File "./fsfsverify.py", line 678, in __init__
(rev, offset, length, size, digest) = value.split(' ')
ValueError: too many values to unpack
I furthermore integrated svn into my IDE (Eclipse Helios) and also tried a checkout of the project, of which revision 2 (the corrupt one) consisted. The process aborts, showing the following error:
Get content for 'svn+ssh://server.fqdn.com/path/to/rep/ProjectName/.project' failed.
svn: Malformed representation header
svn: Corrupt representation '2 325 387 370 d66781ba03f9571d044bf8f257601485 0fd3590a02c19961e78d0ef76efa710d9d6ef194 1-2/_c'
I guess that these errors are related to each, aren't they?
It would be great if someone out there could help me out - there are not many revisions within the repo currently, but I need to report about the success of the test.
Thank anyone for reading and helping!
Bastian
Daniel Shahaf
> Strangely, "svnadmin recover /path/to/rep" successfully succeeds:
> svnadmin recover /path/to/rep
> Repository lock acquired.
> Please wait; recovering the repository may take some time...
> Recovery completed.
> The latest repos revision is 8.
>
'svnadmin recover' in FSFS recreates 'current', and maybe a few other
things, but it most definitely doesn't fix broken svndiff deltas. (How
would it do so? The filesystem may not have the redundant storage from
which to reconstruct them...)
>
> I furthermore integrated svn into my IDE (Eclipse Helios) and also tried a
> checkout of the project, of which revision 2 (the corrupt one) consisted.
> The process aborts, showing the following error:
> Get content for 'svn+ssh://server.fqdn.com/path/to/rep/ProjectName/.project'
> failed.
> svn: Malformed representation header
> svn: Corrupt representation '2 325 387 370 d66781ba03f9571d044bf8f257601485
> 0fd3590a02c19961e78d0ef76efa710d9d6ef194 1-2/_c'
>
That's the interesting part. It tells you that the representation
(either the full plaintext or a binary delta) which starts at byte 325
of the r2 revision file is corrupt. You can continue by looking at that
offset of that revision and see what bytes you find there. ('xxd -s 325
-l 488 /path/to/revs/0/2' for example.)
A representation header is either "PLAIN\n" or "DELTA\n" or "DELTA $some
$numeric $parameters\n". A representation terminates in the word
"ENDREP\n". The representation's length (excluding header and trailer)
should be 387 bytes. Details in the file subversion/libsvn_fs_fs/structure
in the source tree.
Here's an example:
[[[
DELTA
SVN[9 chars]This is the file 'iota'.
ENDREP
...
text: 1 605 37 25 2d18c5e57e84c5b8a5e9a6e13fa394dc 2c0aa9014a0cd07f01795a333d82485ef6d083e2 0-0/_14
]]]
Happy digging,
Daniel
Daniel Shahaf
You probably know that, but:
You should have taken a backup before running this.
Bastian Ugimachi
>That's the interesting part. It tells you that the representation
> I furthermore integrated svn into my IDE (Eclipse Helios) and also tried a
> checkout of the project, of which revision 2 (the corrupt one) consisted.
> The process aborts, showing the following error:
> Get content for 'svn+ssh://server.fqdn.com/path/to/rep/ProjectName/.project'
> failed.
> svn: Malformed representation header
> svn: Corrupt representation '2 325 387 370 d66781ba03f9571d044bf8f257601485
> 0fd3590a02c19961e78d0ef76efa710d9d6ef194 1-2/_c'
>
(either the full plaintext or a binary delta) which starts at byte 325
of the r2 revision file is corrupt. You can continue by looking at that
offset of that revision and see what bytes you find there. ('xxd -s 325
-l 488 /path/to/revs/0/2' for example.)
OK, I performed "xxd -s 325 -l 488 2"
0000145: 7465 6d2e 6f75 742e 7072 696e 746c 6e28 tem.out.println(
0000155: 2256 6965 7274 6572 2057 7572 6622 293b "Vierter Wurf");
0000165: 0a09 0953 7973 7465 6d2e 6f75 742e 7072 ...System.out.pr
0000175: 696e 746c 6e28 2246 c3bc 6e66 7465 7220 intln("F..nfter
0000185: 5775 7266 2229 3b0a 0909 5379 7374 656d Wurf");...System
0000195: 2e6f 7574 2e70 7269 6e74 6c6e 2822 5365 .out.println("Se
You can imagine - this is just some Java code...
By the way... the source file is UTF8 encoded and where xxd says "F..nfter", it should be "Fünfter" - I did some Special-Character-Testing here. Might this be some problem?
A representation header is either "PLAIN\n" or "DELTA\n" or "DELTA $some
$numeric $parameters\n". A representation terminates in the word
"ENDREP\n". The representation's length (excluding header and trailer)
should be 387 bytes. Details in the file subversion/libsvn_fs_fs/structure
in the source tree.
Here's an example:
[[[
DELTA
SVN[9 chars]This is the file 'iota'.
ENDREP
...
text: 1 605 37 25 2d18c5e57e84c5b8a5e9a6e13fa394dc 2c0aa9014a0cd07f01795a333d82485ef6d083e2 0-0/_14
]]]
The intersting thing is that the ID in my error message belongs does not belong to the same representation as the part extracted by xxd around the specified bytes...?!
Daniel Shahaf
> this be some problem?
>
No, that's okay: the bytes there (0xC3 0xB3) are one UTF-8 encoding of "�".
>
> > A representation header is either "PLAIN\n" or "DELTA\n" or "DELTA $some
> > $numeric $parameters\n". A representation terminates in the word
> > "ENDREP\n". The representation's length (excluding header and trailer)
> > should be 387 bytes. Details in the file subversion/libsvn_fs_fs/structure
> > in the source tree.
> >
> > Here's an example:
> >
> > [[[
> > DELTA
> > SVN[9 chars]This is the file 'iota'.
> > ENDREP
> >
> > ...
> >
> > text: 1 605 37 25 2d18c5e57e84c5b8a5e9a6e13fa394dc
> > 2c0aa9014a0cd07f01795a333d82485ef6d083e2 0-0/_14
> > ]]]
> >
>
> The intersting thing is that the ID in my error message belongs does not
> belong to the same representation as the part extracted by xxd around the
> specified bytes...?!
Well, it's corrupted. The correct representation might be somewhere
nearby --- perhaps a few hundred bytes away (in either direction)?
Bastian Ugimachi
No, that's okay: the bytes there (0xC3 0xB3) are one UTF-8 encoding of "ü".> it should be "Fünfter" - I did some Special-Character-Testing here. Might
> this be some problem?
>
Well, it's corrupted. The correct representation might be somewhere
>
> > A representation header is either "PLAIN\n" or "DELTA\n" or "DELTA $some
> > $numeric $parameters\n". A representation terminates in the word
> > "ENDREP\n". The representation's length (excluding header and trailer)
> > should be 387 bytes. Details in the file subversion/libsvn_fs_fs/structure
> > in the source tree.
> >
> > Here's an example:
> >
> > [[[
> > DELTA
> > SVN[9 chars]This is the file 'iota'.
> > ENDREP
> >
> > ...
> >
> > text: 1 605 37 25 2d18c5e57e84c5b8a5e9a6e13fa394dc
> > 2c0aa9014a0cd07f01795a333d82485ef6d083e2 0-0/_14
> > ]]]
> >
>
> The intersting thing is that the ID in my error message belongs does not
> belong to the same representation as the part extracted by xxd around the
> specified bytes...?!
nearby --- perhaps a few hundred bytes away (in either direction)?
Is there any chance to make this consistent again?
And where could this inconsistence possibly come from?
By the way, I just remarked, that "svnadmin verify ...-r x" is wrong ONLY for x=2, not for x>=2 as I wrote before. So correcting this one should make the repository be OK again, I guess?
Daniel Shahaf
> 2011/4/21 Daniel Shahaf <dani...@elego.de>
> > Bastian Ugimachi wrote on Thu, Apr 21, 2011 at 14:17:00 +0200:
> > > The intersting thing is that the ID in my error message belongs does not
> > > belong to the same representation as the part extracted by xxd around the
> > > specified bytes...?!
> >
> > Well, it's corrupted. The correct representation might be somewhere
> > nearby --- perhaps a few hundred bytes away (in either direction)?
> >
>
> Is there any chance to make this consistent again?
Yes, with a bit of skill and luck. Do you have a working copy of r2?
Does the rev file contain all the information (just possibly in the
wrong place)? Do you have backups?
The solutions I'm getting at are "manual revision file surgery/reconstruction".
(You may be able to do it yourself if you study the 'structure' file and
have some patience.) There may be other classes of solutions.
> And where could this inconsistence possibly come from?
>
"Bug in libsvn_fs_fs". (See issue #3705.) Or bug in any of N other
things, including but not limited to hardware bugs.
By the way: did you say what version of svn your server was running?
> By the way, I just remarked, that "svnadmin verify ...-r x" is wrong ONLY
> for x=2, not for x>=2 as I wrote before. So correcting this one should make
> the repository be OK again, I guess?
Yes. I'd expect verify to fail only for r2, but checkout to fail for
(at least some) later revisions as well.
Daniel Shahaf
trunk), which Julian Foad added a few weeks ago.
I have not used that tool or reviewed it.