subversion load fails with “no such revision”

[Harlan Harris]

Hi. This is crossposted from ServerFault. I'm not subscribed to this list, so a cc would be appreciated. (Or feel free to just respond on ServerFault. So far it's got a couple of upvotes, but no responses.)

I'm trying to learn how to migrate a Subversion repo, and am running into an issue that doesn't make sense to me. I've used `svndumpfilter` to split out a sub-project, and have removed some path prefixes. Several hundred commits now import correctly, but then I'm getting the following error:

    <<< Started new transaction, based on original revision 19190

         * editing path : branches/features/DynamicSource ... done.

         * editing path : branches/features/DynamicSource/src/build.properties ... done.

         * editing path : branches/features/DynamicSource/src/client/default.htm ...done.

         * editing path : branches/features/DynamicSource/src/client/js/AdHocController.js ... done.

         * editing path : branches/features/DynamicSource/src/client/js/Report.js ... done.

    svnadmin: E160006: No such revision 19098

         * adding path : branches/features/DynamicSource/src/client/js/Enums.js ...

OK, so I go into the dump file to look at revisions 19190 and 19098. First of all, revision 19098 _does_ exist in the dump file and was imported without a problem. Revision 19190 is a merge. Within 19190, here's that last file's info, which seems to be causing the issue:

    Node-copyfrom-rev: 19100

    Node-copyfrom-path: trunk/src/client/js/Enums.js

    Text-copy-source-md5: 2db7f8d9c0ba4750d88ce0722731aad6

    Node-path: branches/features/DynamicSource/src/client/js/Enums.js

    Node-action: add

    Text-copy-source-sha1: 8f930509f8dbc17c5e82cd40aa5a76454d3d812c

    Node-kind: file

    Content-length: 0

Confusingly, revision 19100 does NOT exist in this filtered file. But the error's not referring to 19100, it's referring to 19098!

What do I do to get this file to load?

Thanks!

[Harlan Harris]

I still haven't been able to make any progress on this. If ServerFault and this mailing list aren't going to be adequate to solve the problem, could someone suggest an alternative? Thanks,

 -Harlan

[Thorsten Schöning]

Guten Tag Harlan Harris,
am Freitag, 30. August 2013 um 16:06 schrieben Sie:

> I still haven't been able to make any progress on this. If
> ServerFault and this mailing list aren't going to be adequate to
> solve the problem, could someone suggest an alternative? Thanks,

The easiest workaround most of the time is to simply not use
svndumpfilter, but create as many repos as you need with all the
dumped data and afterwards delete and move directories within the new
repos using subversion clients.

Mit freundlichen Grüßen,

Thorsten Schöning

--
Thorsten Schöning E-Mail:Thorsten....@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow

[Philip Martin]

Harlan Harris <harlan...@gmail.com> writes:

> I still haven't been able to make any progress on this. If ServerFault and
> this mailing list aren't going to be adequate to solve the problem, could
> someone suggest an alternative? Thanks,

Splitting up repositories is not trivial and it is not clear, to me at
least, exactly what you are doing.

>
> On Thu, Aug 29, 2013 at 8:21 AM, Harlan Harris <har...@harris.name> wrote:
>

>> Hi. This is crossposted from ServerFault<http://serverfault.com/questions/534553/subversion-load-fails-with-no-such-revision>.

>> I'm not subscribed to this list, so a cc would be appreciated. (Or feel
>> free to just respond on ServerFault. So far it's got a couple of upvotes,
>> but no responses.)
>>
>>
>> I'm trying to learn how to migrate a Subversion repo, and am running into
>> an issue that doesn't make sense to me. I've used `svndumpfilter` to split
>> out a sub-project, and have removed some path prefixes. Several hundred
>> commits now import correctly, but then I'm getting the following error:
>>
>> <<< Started new transaction, based on original revision 19190
>> * editing path : branches/features/DynamicSource ... done.
>> * editing path :
>> branches/features/DynamicSource/src/build.properties ... done.
>> * editing path :
>> branches/features/DynamicSource/src/client/default.htm ...done.
>> * editing path :
>> branches/features/DynamicSource/src/client/js/AdHocController.js ... done.
>> * editing path :
>> branches/features/DynamicSource/src/client/js/Report.js ... done.
>> svnadmin: E160006: No such revision 19098
>> * adding path :
>> branches/features/DynamicSource/src/client/js/Enums.js ...
>>
>> OK, so I go into the dump file to look at revisions 19190 and 19098. First
>> of all, revision 19098 _does_ exist in the dump file and was imported
>> without a problem.

That error refers to r19098 in the repository, does that revision exist?
Which revision was created by loading r19098?

>> Revision 19190 is a merge. Within 19190, here's that
>> last file's info, which seems to be causing the issue:
>>
>> Node-copyfrom-rev: 19100
>> Node-copyfrom-path: trunk/src/client/js/Enums.js
>> Text-copy-source-md5: 2db7f8d9c0ba4750d88ce0722731aad6
>> Node-path: branches/features/DynamicSource/src/client/js/Enums.js
>> Node-action: add
>> Text-copy-source-sha1: 8f930509f8dbc17c5e82cd40aa5a76454d3d812c
>> Node-kind: file
>> Content-length: 0
>>
>> Confusingly, revision 19100 does NOT exist in this filtered file. But the
>> error's not referring to 19100, it's referring to 19098!

Perhaps there is a bug in the load renumbering, but it's hard to say
because it's not clear what you are doing.

You mention "several hundred" but you are dealing with revisons much
higher. How did you produce the dump file? How did you filter it? How
many revisions in the dumpfile? Are they sequential? How many
revisions in the destination repository?

Aside from the renumbering problem, you say that r19098 exists in the
dumpfile but r19100 does not. I don't think there is any way you can
load r19190 if it refers to r19100. A dump may refer to revisions
before the start of the dump but that is not the case here. You have a
reference to a missing revision withing the dump range. What do you
expect load to do?

--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*

[Harlan Harris]

Thanks, Thorsten. That makes sense. I guess disk space is cheap now...!

 -Harlan

[Tony Sweeney]

There are at least four different versions of dumpfilter floating about the aether.  It would help to know which one you used, and how you used it.

 

Tony.

---

From: Harlan Harris [mailto:harlan...@gmail.com]
Sent: 30 August 2013 15:06
To: us...@subversion.apache.org
Subject: Re: subversion load fails with “no such revision”

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

---

No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.3392 / Virus Database: 3211/6618 - Release Date: 08/28/13

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

[Thorsten Schöning]

Guten Tag Harlan Harris,
am Freitag, 30. August 2013 um 17:10 schrieben Sie:

> I guess disk space is cheap now...!

It's especially cheaper than your time if you only need to version "a
bit of web application stuff" and, depending on the version of your
old repos, newer repos may even reduce disk space because of
representation sharing and some improvements in storing directory
structures in Subversion 1.8.

[Harlan Harris]

Tony, I used the seemingly-standard version of dumpfilter, not the 2 or 3 versions. But then I also did the search/replace trick that others had suggested to shift paths around. I don't think that's the issue, but I'm not sure. 

Thorsten, I wish it was just "a bit of web application stuff", but it's actually a 5-year-old enterprise repo that was horribly abused, including checkins of very large binary and data files. The dump file of the whole thing is about 25 GB in size. That's part of why I want to split it up and filtering things out.

I'm going to try just exporting the last month worth of revisions, and see if that works better. We may have to sacrifice most history in the interest of actually getting this working.

 -Harlan

[Thorsten Schöning]

Guten Tag Harlan Harris,
am Freitag, 30. August 2013 um 20:03 schrieben Sie:

> Thorsten, I wish it was just "a bit of web application stuff", but
> it's actually a 5-year-old enterprise repo that was horribly abused,
> including checkins of very large binary and data files.

Unless your users did commit very distinct data, representation
sharing can save you quite a lot of disk space. It's worth trying it.

[Harlan Harris]

An FYI to all,

I'm back trying to deal with this migration, and it's still a giant mess. The consistent problem seem to be that revision numbering is just flat wrong when files were renamed in a revision. I'm fixing "svnadmin load" errors one-at-a-time by the following process:

1. Find in the dump file the "add" that's breaking. Note that there is a Node-copyfrom-rev that refers to a revision that doesn't exist in the filtered file (and is irrelevant in the pre-filtered file); that is, the revision number is incorrect. Also note that the svnadmin load error message refers to a third revision number that has nothing to do with the problem either -- it's NOT the number in the Node-copyfrom-rev. 

2. Find the first previous revisionof that file that was a change. Note the correct revision number.

3. Return to the add that's breaking and manually change the revision number to refer to that commit. 

4. Wipe the repo and restart. Go back to step 1.

I'm doing the load with svnadmin 1.7.13, but the dump was from svnadmin 1.6.6. Upgrading the latter is not an option, which is why we're moving and splitting the repo.

[Andreas Mohr]

Hi,

On Thu, Sep 26, 2013 at 12:18:30PM -0400, Harlan Harris wrote:
> An FYI to all,
> I'm back trying to deal with this migration, and it's still a giant mess.
> The consistent problem seem to be that revision numbering is just flat
> wrong when files were renamed in a revision. I'm fixing "svnadmin load"
> errors one-at-a-time by the following process:
> 1. Find in the dump file the "add" that's breaking. Note that there is a
> Node-copyfrom-rev that refers to a�revision�that doesn't exist in the
> filtered file (and is irrelevant in the pre-filtered file); that is, the
> revision number is incorrect. Also note that the svnadmin load error
> message refers to a third revision number that has nothing to do with the

> problem either -- it's NOT the number in the Node-copyfrom-rev.�

I'm not entirely convinced that manual corrections are the way to go.
From the sounds of it, this is a painfully huge effort for you,
with considerable complications. IOW: ouch.

So, AFAICS (you hinted at that) these dumps were split out via svndumpfilter.

What about tending towards trying to fix the suspected root cause
(problematic algorithms in svndumpfilter) rather than huge manual efforts
of trying to fix each such "broken rename" case?

I don't know how difficult/challenging it would be to try to fix
svndumpfilter algorithms to correctly handle/take into account such renames
(I don't have any experience with svndumpfilter),
but to me it sounds like this would be much more lucrative.

If one managed to do some investigations about the differences between
the 4 different svndumpfilter "offsprings" and codify this into a nice
well-manageable upstream project (SourceForge, github, ...)
while fixing the revisions-with-renames issues,
then this would be a huge win AFAICS.


I'm trying to teach the TFS Plain-Original-Software some new SVN tricks,
so I know that there's quite some effort required to achieve proper SCM
handling, but fixing one isolated issue (let's hope it's only one issue...)
hopefully wouldn't be too hard (in the TFS support case there were many
low-hanging fruits, too).

Or try to ask someone to have a look at what would be required
to get these svndumpfilter issues improved...

HTH,

Andreas Mohr