Issue while loading the SVN Dump SVN version 1.9.7

[Santosh Kondapuram]

HI All,

 

I am running in to issues while loading the SVN dump to one of the newly created SVN repository and the error happens only while loading specific revision 724413.

Please find the error message below. Appreciate your help !!

Source SVN version : 1.9.5

Target SVN version : 1.9.7

 

 

FYI,

 

[root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos < incdump_724413.txt

<<< Started new transaction, based on original revision 724413

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_businessEntityRelationsHistory.sql ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityApplicationUserHistory.sql ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityRoleUserHistory.sql ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_userMaintenance_audit_Labels.sql ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.hbm ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.hbm ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.hbm ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseBusinessEntityRelationHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityApplicationUsrHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityRoleUserHist.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityService.java ... done.

     * editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityServiceImpl.java ...svnadmin: E160000: SHA1 of reps '669684 7 1143 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ

svnadmin: E160004: Filesystem is corrupt

svnadmin: E200014: Checksum mismatch while reading representation:

   expected:  efbdb058ce857b2860cfa245f014f0b9

     actual:  04a53277f405bcbab8a3b1fff9f8c6e0

 

Thanks,

Santosh.

 

This e-mail message and any files transmitted with it may contain confidential and proprietary information and are intended solely for the use of the individual or entity to which they are addressed. Any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this e-mail in error please notify the sender by reply email and destroy all copies of the original message. Thank you for your cooperation.

[Johan Corveleyn]

Hello,

This looks similar to this problem (which was present in 1.8.19, but
probably the same bug is still present in 1.9.7):

http://mail-archives.apache.org/mod_mbox/subversion-users/201707.mbox/%3C5aae07a9-bb6d-6340-206a-dafcf91e35e6%40apache.org%3E

Performing the load with the option "-M 0" (disabling the in-memory
cache during load, making it a little bit slower) should work around
it.

HTH,
--
Johan

[Santosh Kondapuram]

Hi Johan,

I have tried reloading the dump as you suggested with -M 0 option but still I am running in to the same issue.
Seems like the svn admin could not load the dump due to sha1 collision files. So the question is how do I identify the sha1 colliding files ? does the error stack trace say something about it ?
As per the sha1-advisory note if we create a Subversion permission rule(authz) that block access to the one or both of the files, then tools like svnsync will not send the colliding content.
So If someone can help me in identifying the colliding files then I would like to block them and sync my mirror repository. Below is the error I am hitting.
Any help much appreciated and I am very much new to the subversion.

https://subversion.apache.org/security/sha1-advisory.txt

FYI,

svnadmin: E160000: SHA1 of reps '669684 7 1143 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ
svnadmin: E160004: Filesystem is corrupt
svnadmin: E200014: Checksum mismatch while reading representation:
expected: efbdb058ce857b2860cfa245f014f0b9
actual: 04a53277f405bcbab8a3b1fff9f8c6e0

Thanks,
Santosh.

-----Original Message-----
From: Johan Corveleyn [mailto:jco...@gmail.com]
Sent: Monday, January 22, 2018 5:50 PM
To: Santosh Kondapuram <SKond...@vitechinc.com>
Cc: us...@subversion.apache.org
Subject: Re: Issue while loading the SVN Dump SVN version 1.9.7

On Thu, Jan 18, 2018 at 1:34 PM, Santosh Kondapuram <SKonud...@vitechinc.com> wrote:
> HI All,
>
>
>
> I am running in to issues while loading the SVN dump to one of the
> newly created SVN repository and the error happens only while loading
> specific revision 724413.
>
> Please find the error message below. Appreciate your help !!
>
> Source SVN version : 1.9.5
>
> Target SVN version : 1.9.7
>
>
>
>
>
> FYI,
>
>
>
> [root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos
> < incdump_724413.txt
>
> <<< Started new transaction, based on original revision 724413
>
> * editing path :
> v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception
> /web_dml_admin_businessEntityRelationsHistory.sql
> ... done.
>
> * editing path :
> v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception
> /web_dml_admin_securityApplicationUserHistory.sql
> ... done.
>
> * editing path :
> v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception

> /web_dml_admin_securityRoleUserHistory.sql

> ... done.
>
> * editing path :
> v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception

> /web_dml_admin_userMaintenance_audit_Labels.sql

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/BusinessEntityRelationHist.hbm

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/BusinessEntityRelationHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/SecurityApplicationUsrHist.hbm

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/SecurityApplicationUsrHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/SecurityRoleUserHist.hbm

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/SecurityRoleUserHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/base/BaseBusinessEntityRelationHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/base/BaseSecurityApplicationUsrHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/model/base/BaseSecurityRoleUserHist.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/
> admin/services/SecurityService.java

> ... done.
>
> * editing path :

> v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/

> admin/services/SecurityServiceImpl.java
> ...svnadmin: E160000: SHA1 of reps '669684 7 1143 195972
> efbdb058ce857b2860cfa245f014f0b9
> 9db457be74545c184242e57208bf1d56db1f15b2
> 724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9
> 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches
> (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ
>
> svnadmin: E160004: Filesystem is corrupt
>
> svnadmin: E200014: Checksum mismatch while reading representation:
>
> expected: efbdb058ce857b2860cfa245f014f0b9
>
> actual: 04a53277f405bcbab8a3b1fff9f8c6e0
>

Hello,

This looks similar to this problem (which was present in 1.8.19, but probably the same bug is still present in 1.9.7):

http://mail-archives.apache.org/mod_mbox/subversion-users/201707.mbox/%3C5aae07a9-bb6d-6340-206a-dafcf91e35e6%40apache.org%3E

Performing the load with the option "-M 0" (disabling the in-memory cache during load, making it a little bit slower) should work around it.

HTH,
--
Johan

[Johan Corveleyn]

On Thu, Jan 25, 2018 at 10:40 AM, Santosh Kondapuram
<SKond...@vitechinc.com> wrote:
> Hi Johan,
>
> I have tried reloading the dump as you suggested with -M 0 option but still I am running in to the same issue.
> Seems like the svn admin could not load the dump due to sha1 collision files. So the question is how do I identify the sha1 colliding files ? does the error stack trace say something about it ?
> As per the sha1-advisory note if we create a Subversion permission rule(authz) that block access to the one or both of the files, then tools like svnsync will not send the colliding content.
> So If someone can help me in identifying the colliding files then I would like to block them and sync my mirror repository. Below is the error I am hitting.
> Any help much appreciated and I am very much new to the subversion.
>
> https://subversion.apache.org/security/sha1-advisory.txt
>
> FYI,
> svnadmin: E160000: SHA1 of reps '669684 7 1143 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ
> svnadmin: E160004: Filesystem is corrupt
> svnadmin: E200014: Checksum mismatch while reading representation:
> expected: efbdb058ce857b2860cfa245f014f0b9
> actual: 04a53277f405bcbab8a3b1fff9f8c6e0
>
> Thanks,
> Santosh.

Hi Santosh,

Hmmm, I chatted a bit about this on irc with Andreas Stieger. It seems
unlikely that you have a real sha-1 collision in your repository.
Unless someone committed those on purpose (for instance by committing
files from https://shattered.io/). It's also possible that there is
another bug in the "sha-1 collision detection code" (one that doesn't
get bypassed with the -M0 trick).

So, are you sure someone committed such sha-1-colliding files?

Also: the revision where it fails, is that the last revision in the dumpstream?

Something you could try to get further, or to get more information
about what's going on: if you disable rep-sharing SVN should be able
to store the sha-1 collision (you won't be able to check out both
colliding files in a working copy though). So you could 'svnadmin
load' until the revision right before the problematic one, then
disable rep-sharing (putting the option enable-rep-sharing = false in
db/fsfs.conf), then load the problematic revision. Then you might be
able to find out at least the name of the file that's causing the
problem.

HTH,
--
Johan

[Santosh Kondapuram]

Hi Johan,

Sorry for the delayed response as I was in long vacation.
Yes, I don’t think we are hitting the real sha-1 collision in our repository and as you said it might be another bug in the sha-1 collision detection code.
I am sure that no one committed the sha-1-colliding files from https://shattered.io
Our repository is very huge with 1 Million + revisions and I am seeing the issue only while loading the specific revision 724413. I am able to load the revisions before and after it.
As you suggested I have now loaded the dump till 724412 which is the revision before the problematic one with rep sharing enabled.
It's not allowing me to load the problematic revision with rep-sharing disabled and getting below message.

FYI,

[root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos < incdump724413.txt
svnadmin: E200002: Error while parsing config file: /u01/svn/repos/db/fsfs.conf:
svnadmin: E200002: line 39: Option expected
[root@vitech-svn-slave-test-01 svndump]#


If I re-enable the rep-sharing and trying to load the problematic revision, I get the error while loading the specific file"SecurityServiceImpl.java" of that revision number. Please let me know if you need any further details.
Just to reiterate my source SVN is 1.9.5 and target is 1.9.7 version

FYI,

[root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos < incdump724413.txt

<<< Started new transaction, based on original revision 724413

* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_businessEntityRelationsHistory.sql ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityApplicationUserHistory.sql ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_securityRoleUserHistory.sql ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/Scripts/DML_Exception/web_dml_admin_userMaintenance_audit_Labels.sql ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.hbm ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/BusinessEntityRelationHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.hbm ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityApplicationUsrHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.hbm ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/SecurityRoleUserHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseBusinessEntityRelationHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityApplicationUsrHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/model/base/BaseSecurityRoleUserHist.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityService.java ... done.
* editing path : v3-web-product/branches/8.3.0.279_IPERS/benefits/src/com/vitechinc/v3/admin/services/SecurityServiceImpl.java ...svnadmin: E160000: SHA1 of reps '669684 7 1143 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' and '-1 0 929 195972 efbdb058ce857b2860cfa245f014f0b9 9db457be74545c184242e57208bf1d56db1f15b2 724412-fiyk/_16' matches (9db457be74545c184242e57208bf1d56db1f15b2) but contents differ

svnadmin: E160004: Filesystem is corrupt
svnadmin: E200014: Checksum mismatch while reading representation:
expected: efbdb058ce857b2860cfa245f014f0b9
actual: 04a53277f405bcbab8a3b1fff9f8c6e0

Thanks,
Santosh.

-----Original Message-----
From: Johan Corveleyn [mailto:jco...@gmail.com]
Sent: Thursday, January 25, 2018 4:27 PM
To: Santosh Kondapuram <SKond...@vitechinc.com>
Cc: us...@subversion.apache.org
Subject: Re: Issue while loading the SVN Dump SVN version 1.9.7

[Johan Corveleyn]

On Wed, Jan 31, 2018 at 9:28 AM, Santosh Kondapuram
<SKond...@vitechinc.com> wrote:
> Hi Johan,
>

> Sorry for the delayed response as I was in long vacation.
> Yes, I don’t think we are hitting the real sha-1 collision in our repository and as you said it might be another bug in the sha-1 collision detection code.
> I am sure that no one committed the sha-1-colliding files from https://shattered.io
> Our repository is very huge with 1 Million + revisions and I am seeing the issue only while loading the specific revision 724413. I am able to load the revisions before and after it.
> As you suggested I have now loaded the dump till 724412 which is the revision before the problematic one with rep sharing enabled.
> It's not allowing me to load the problematic revision with rep-sharing disabled and getting below message.
>
> FYI,
>
> [root@vitech-svn-slave-test-01 svndump]# svnadmin load /u01/svn/repos < incdump724413.txt
> svnadmin: E200002: Error while parsing config file: /u01/svn/repos/db/fsfs.conf:
> svnadmin: E200002: line 39: Option expected

Hm, that indicates that there was a syntax error on line 39 in the
db/fsfs.conf file you edited (when you were trying to disable
rep-sharing). Can you take another look and try again with rep-sharing
disabled? That line in fsfs.conf should be (without leading spaces):

enable-rep-sharing = false

Now, I'm not sure what we'll be able to do next to figure this out.
First step is probably find out which two files are colliding, and
what their contents is exactly. It's extremely unlikely that there is
a real sha1 collision, but we have to rule it out I suppose.

--
Johan

[Johan Corveleyn]

Santosh,

Can you double-check that using -M 0 for the 'svnadmin load' operation
doesn't solve the problem (while keeping rep-sharing enabled)?

So:
svnadmin load -M 0 /u01/svn/repos < incdump724413.txt

It's just that this works around the only currently known bug in the
sha1-collision-detection code, so I want to be really sure that this
workaround doesn't help in your case and we need to look further.

--
Johan

[Santosh Kondapuram]

Hi Johan,

As you suggested I have removed the leading spaces from line 39 (enable-rep-sharing=false) and this time it worked and was able to successfully load the problematic revision.
So does this conclude I have the sha-1 colliding files in my repo ? Also what are the next steps to catchup the latest revisions from the master node ?
Appreciate all your help and great working with you on this issue !!!

FYI,

By the way I have tried running the following command " svnadmin load -M 0 /u01/svn/repos < incdump724413.txt "with rep-sharing enabled and still see the same issue. I have tried doing this before the above work around which worked.

Thanks,
Santosh.

-----Original Message-----
From: Johan Corveleyn [mailto:jco...@gmail.com]
Sent: Wednesday, January 31, 2018 8:59 AM
To: Santosh Kondapuram <SKond...@vitechinc.com>
Cc: us...@subversion.apache.org
Subject: Re: Issue while loading the SVN Dump SVN version 1.9.7

[Johan Corveleyn]

On Wed, Jan 31, 2018 at 4:12 PM, Santosh Kondapuram
<SKond...@vitechinc.com> wrote:
> Hi Johan,
>

> As you suggested I have removed the leading spaces from line 39 (enable-rep-sharing=false) and this time it worked and was able to successfully load the problematic revision.
> So does this conclude I have the sha-1 colliding files in my repo ? Also what are the next steps to catchup the latest revisions from the master node ?
> Appreciate all your help and great working with you on this issue !!!
>
> FYI,
>
> By the way I have tried running the following command " svnadmin load -M 0 /u01/svn/repos < incdump724413.txt "with rep-sharing enabled and still see the same issue. I have tried doing this before the above work around which worked.

Okay, thanks for re-testing that.

What to do next? I think it depends on whether or not this is a real
collision, or why the collision-detection code went wrong. Normally
you can catchup with the original repo by creating another incremental
dump of the remaining revisions, and loading that into the new
repository. You can re-enable rep-sharing before doing so, so the
additional revisions are using the rep-sharing functionality.

However, I'm still wondering what went wrong here. If there is a real
sha1 collision, you won't be able to checkout a working copy which
contains both colliding files (though it's certainly possible that
both files would normally not appear together in a working copy --
perhaps the "first file" is already long deleted, so it's only part of
ancient history in your repository).

To find out a little bit more about the (alleged) collision, can you
do the following, by using the sqlite3 executable (perhaps it's
installed standard on your system)?

go to the db subdir of your repository
sqlite3 rep-cache.db "select * from rep_cache where
hash='9db457be74545c184242e57208bf1d56db1f15b2'"

I think you'll get back at least two rows. The schema of the table is:

( hash TEXT NOT NULL PRIMARY KEY, revision INTEGER NOT NULL, offset
INTEGER NOT NULL, size INTEGER NOT NULL, expanded_size INTEGER NOT
NULL )

The revision columns that you get back might be interesting for
further investigation (perhaps by looking at them in the original
repo). Maybe you can 'svn log -v' those revisions, and run 'svn cat
URL_OF_FILE@REV' for each of the affected files (and the corresponding
revisions) to see their contents (and perhaps sha1sum them with a
commandline tool).

--
Johan